Grading timetable (June/July 2013)

General discussions about ratings.
Andrew Collins
Posts: 66
Joined: Sun Jun 10, 2012 3:58 pm

Re: Grading timetable (June/July 2013)

Post by Andrew Collins » Sun Jul 28, 2013 8:03 pm

I did notice that on the previous grading list, I looked at the top players and it went up from around 200 grade, although I admittedly can't remember how many players it covered. However on this top players list it starts at 210. So perhaps there are now a higher distribution of players graded above 210?
I'm not sure what it all means, but providing it doesn't hinder my ambitions to continue increasing my grade, I'm unconcerned. I had +30 on this list, however I would like to think it is due to me being a greatly improving adult player, although I'm sure some of my friends would just say I'm now at my true playing strength.

+-5 is an interesting point, I looked at the top improving players and the majority with large increases are juniors in any case

Angus French
Posts: 2151
Joined: Thu May 15, 2008 1:37 am

Re: Grading timetable (June/July 2013)

Post by Angus French » Sun Jul 28, 2013 8:05 pm

A few comments, for what they are worth, on the new grading web pages:

- I preferred the old presentation. It seemed slicker (better css) and I guess I liked the pie charts.
- I think the split panel - with lookup stuff on the left and results on the right - is an improvement (now there's no need to hit the 'back' button to get back to the lookup page).
- I miss not seeing grade estimates for ungraded opponents. To my mind this is a omission as it isn't now possible see that a grade is correctly derived.
- I'm sorry but I think the crosstables are naff. It seems to me that someone is trying too hard to impress.

User avatar
Christopher Kreuzer
Posts: 8806
Joined: Fri Aug 06, 2010 2:34 am
Location: London

Re: Grading timetable (June/July 2013)

Post by Christopher Kreuzer » Sun Jul 28, 2013 8:26 pm

MartinCarpenter wrote:The limit to the most recent 30 can definitely make things more random - my -22 this time (193 to 171) would I think have been 'merely' ~-15 (190 - 175) on full yearly grades. (Which would be wel deserved as I played pretty awfully last season.).

I guess one interesting thing to test would be if players with >30 games a year (X grades I think?) were getting more volatile results than people playing less than that. If they are then wouldn't that be fairly strong evidence that countback isn't a good idea?
I think this is a misunderstanding. Countback to the most recent 30 games only occurs if you play less than 30 games. If you play more than 30 games in a grading period, then all the games are included. At least that is my understanding unless I missed something somewhere. I have seen some posts, though, where people seemed to think that the grades are only based on your most recent 30 games regardless of whether you have played more than 30 games or not, so if some people do think that and it is incorrect, it is worth correcting that misunderstanding.
Angus French wrote:I miss not seeing grade estimates for ungraded opponents. To my mind this is a omission as it isn't now possible see that a grade is correctly derived.
I agree. If you have only played one or two such players, you can still estimate their grade estimates to some extent, but it does seem silly that this information has been omitted.

Richard Bates
Posts: 3338
Joined: Fri Nov 14, 2008 8:27 pm

Re: Grading timetable (June/July 2013)

Post by Richard Bates » Mon Jul 29, 2013 8:13 am

Sean Hewitt wrote:I've asked Alex Holowczak, as the Director responsible for grading, if he can get the team to run some simulations. In particular, what effect has the count back to 30 had. I've not seen any evidence that this change was properly tested in advance. They'll also look at whether there is any inflation or stretch in the system.
I feel there has to be something logically dubious about a methodology that can result in significant changes in grade for people who play few games in any single grading period. The extreme case being someone who, say, plays 50 games in one period and none in the next. Also if you have a grade of 200, and you perform at 200 in the next period then your grade should stay the same, regardless of the number of games played. This is a) bound to increase volatility throughout the list and b) means that all games aren't equal with games played at the end of a grading period much more important than those at the start. With the effect potentially continuing over several lists. For the vast majority chess strength simply doesn't change enough to justify this. This simply wouldn't happen if the sensible approach IMO of a weighted average was used.

LawrenceCooper
Posts: 7173
Joined: Tue Dec 20, 2011 8:13 am

Re: Grading timetable (June/July 2013)

Post by LawrenceCooper » Mon Jul 29, 2013 8:25 am

Richard Bates wrote:
Sean Hewitt wrote:I've asked Alex Holowczak, as the Director responsible for grading, if he can get the team to run some simulations. In particular, what effect has the count back to 30 had. I've not seen any evidence that this change was properly tested in advance. They'll also look at whether there is any inflation or stretch in the system.
I feel there has to be something logically dubious about a methodology that can result in significant changes in grade for people who play few games in any single grading period. The extreme case being someone who, say, plays 50 games in one period and none in the next. Also if you have a grade of 200, and you perform at 200 in the next period then your grade should stay the same, regardless of the number of games played. This is a) bound to increase volatility throughout the list and b) means that all games aren't equal with games played at the end of a grading period much more important than those at the start. With the effect potentially continuing over several lists. For the vast majority chess strength simply doesn't change enough to justify this. This simply wouldn't happen if the sensible approach IMO of a weighted average was used.
I know one player who had played 20 games in the last year, making a plus score against an average of 140 but had dropped from 136 to 132. He was less than impressed when I observed that his grade was calculated on his last 30 games so the poor tournament he'd had in May 2012 was still coming back to haunt him.

Michael Flatt
Posts: 1235
Joined: Tue Jul 02, 2013 7:36 am
Location: Hertfordshire

Re: Grading timetable (June/July 2013)

Post by Michael Flatt » Mon Jul 29, 2013 8:31 am

Category
With each grade is a Category that indicates the extent to which games from earlier periods have been incorporated. Grades are based on the most recent 36 months maximum, and all Categories except X (the most active) are based on the most recent 30 games in that period, where 30 are available. Categories D and E may indicate that fewer than 30 games are available.

The Categories were formerly different for adult and junior players, but this distinction has been dropped.

Category definitions:
X - Grade based on all games (at least 30) played in the latest halfyear
A - Grade based on the 30 most recent games, all played in the latest 12 months
B - Grade based on the 30 most recent games, all played in the latest 24 months, including at least 20 in the latest 12 months
C - Grade based on the 30 most recent games in the latest 36 months, of which at least 10 were played in the latest 12 months
D - Grade based on 15 or more games in the latest 36 months, of which at least 5 were played in the latest 12 months
E - Grade based on 9 or more games in the latest 36 months, of which at least 1 was played in the latest 12 months

Players not meeting any of these requirements are ungraded.
The whole discussion on volatility of grades seems to be a complete red herring.

To get an accurate grade - just play more games!

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: Grading timetable (June/July 2013)

Post by Roger de Coverly » Mon Jul 29, 2013 8:45 am

Michael Flatt wrote:To get an accurate grade - just play more games!
Alternatively don't play as many. So if you are on form for half the season and off form for the other half, by playing 15 games in each period you get a stable grade. If you play 30 in each half, you get a high grade for the on form part of the season and a low grade for the off form part. My recent six month sequences have been 175, 193, 183, 189. Averaged over a whole year it would have been something like 184 and 186. Still at least it means you get to set new personal highs.

Michael Flatt
Posts: 1235
Joined: Tue Jul 02, 2013 7:36 am
Location: Hertfordshire

Re: Grading timetable (June/July 2013)

Post by Michael Flatt » Mon Jul 29, 2013 9:08 am

Roger, that is an interesting point.

Can players really complain that the grading system picks up short term variations in a players performance?

The question may be whether we should restrict the calculation to the most recent 30 games when taking an average over more than one six month grading period?

Would grades calculated over a full season (12 months) using all games played that season be more meaningful than an average based on the last 30 games?

David Guthrie
Posts: 9
Joined: Mon Jul 29, 2013 8:48 am

Re: Grading timetable (June/July 2013)

Post by David Guthrie » Mon Jul 29, 2013 9:55 am

I'm not a statistician, but I think I could easily be convinced that the system of +(-) 50 points for a win(loss) produces a stable set of gradings as long as a large majority of the players play against a reasonably even distribution of opponents by grade (i.e. they play approximately as many people graded above them as below them). However, for players who play uneven distributions of opponents by grade, I think some strange effects can appear.

For example, I think some players might be classed as "flat track bullies". Against a normal distribution of opponents, their grades would be stable. However, they score well in excess of their predicted score against opponents graded below them, and much worse than predicted against opponents graded above them. So if the strength of their opponents changes from grading period to grading period (e.g. because they play league chess, where a season playing weaker opponents results their grade going up, resulting in promotion to a higher board for the next season, where they play much stronger opponents, resulting in their grade plummeting back down again, resulting in being demoted down the board order, resulting in a season playing weaker opponents, etc) their grade will yo-yo up and down, without any real change in their playing strength.

This type of effect is definitely volatility, but on its own it is not inflation. I can think of ways that you might try and fix it (e.g. applying some sort of weighting to results so that a player's grade is always based on 50% on games against players graded above and 50% against players graded below them, regardless of the actual distribution of opponents) but obviously this might create move problems than it solves...

Alan Walton
Posts: 1394
Joined: Mon Nov 02, 2009 8:33 pm
Location: Oldham

Re: Grading timetable (June/July 2013)

Post by Alan Walton » Mon Jul 29, 2013 10:26 am

A couple of good statistics that could be should would be the following

1) Percentage score of games graded
2) Average grade of opponent graded

This would show the variations on how people achieved their grade (as David aludes to)

Taking my grade as an example, it has come out at 198, the above stats were an % score of 54.8, with an average grade of 193 (42 game sample)

If another person has a grade of 198, but has the a % score of 75, then his average opponents grade would be 173

With the inflation in players grades a couple of years a 173 average would relate to an approx "old grade" of 160ish, whereas before the inflation and old 193 didn't receive much inflation

Therefore this example shows as David said a "flat track bullies" scenario

Sean Hewitt
Posts: 2193
Joined: Sat Mar 10, 2012 8:18 pm

Re: Grading timetable (June/July 2013)

Post by Sean Hewitt » Mon Jul 29, 2013 11:23 am

Richard Bates wrote:
Sean Hewitt wrote:I've asked Alex Holowczak, as the Director responsible for grading, if he can get the team to run some simulations. In particular, what effect has the count back to 30 had. I've not seen any evidence that this change was properly tested in advance. They'll also look at whether there is any inflation or stretch in the system.
I feel there has to be something logically dubious about a methodology that can result in significant changes in grade for people who play few games in any single grading period. The extreme case being someone who, say, plays 50 games in one period and none in the next. Also if you have a grade of 200, and you perform at 200 in the next period then your grade should stay the same, regardless of the number of games played. This is a) bound to increase volatility throughout the list and b) means that all games aren't equal with games played at the end of a grading period much more important than those at the start. With the effect potentially continuing over several lists. For the vast majority chess strength simply doesn't change enough to justify this. This simply wouldn't happen if the sensible approach IMO of a weighted average was used.
On a personal level I'm inclined to agree with you. However, I'd want to see the testing of the alternative approach that was carried out by the grading team before implementation. I wasn't involved when the change was implemented, so don't know what they thought would happen but it seems sensible to test if the assumptions that they did have were right or not.

Alan Walton
Posts: 1394
Joined: Mon Nov 02, 2009 8:33 pm
Location: Oldham

Re: Grading timetable (June/July 2013)

Post by Alan Walton » Mon Jul 29, 2013 11:32 am

I would of thought the best way to calculate grades for those players who don't play enough games was to use the weighted average from previous periods to make up the games, rather than just the games in calendar day order

User avatar
Mike W. Richardt
Posts: 94
Joined: Wed Mar 23, 2011 10:37 am
Location: Abu Dhabi & Taunton

Re: Grading timetable (June/July 2013)

Post by Mike W. Richardt » Mon Jul 29, 2013 11:48 am

What I can't understand why the new grades have to be published only once or twice a year? Looking at my ratings: I have three different ones: My German national rating (Deutsche Wertungs Zahl), my ECF and my FIDE rating.

Looking at the FIDE and my German national rating they get updated as soon the tournament I played is send for grading, which mean that both ratings up to date with your current level of play.
In Germany for league purposes they use the rating at the beginning of the season and for grading purpose they use the one at the end of the league season.

To be honest I do think that if we all had the same ratings type (i.e. we all have FIDE ratings or else) I think we would be better off and don't compare pears with apples.
I am in the process of sorting out a move to NZ and I bet when I start playing chess there I will get another rating which would be my forth one!!!! :shock: :lol:
Nullius addictus iurare in verba magistri - I am not bound to believe in the word of any master
http://www.mikerichardt.co.uk

MartinCarpenter
Posts: 3044
Joined: Tue May 24, 2011 10:58 am

Re: Grading timetable (June/July 2013)

Post by MartinCarpenter » Mon Jul 29, 2013 12:59 pm

No, no misundertanding. (I hope :))

It is vanishingly rare for people to play >30 games in half a year, but far from unknown to do so over a single season. I say season and not year because serious club level chess is nearly all ~Sept <-> ~April with a non trivial summer break. So if you play 30 games a season you get to count the most recent two half seasons and your July grade is your entire previous season.

If you move up to playing say 40 games a season - which really isn't hugely uncommon - then each grade is the twenty games from that six month period combined with a time limited sample from the previous time period.

Given the tendency to cluster abnormally bad/good results in specific tournaments/short time periods, there is clear reason to think that this might produce more volatile grades than those for people whose July grades do represent an entire season. Whether it does or not in practice is another matter - its the sort of thing which you can check easily enough. Nice to see it happening :)

It might also affect people playing less each year but somehow seems especially perverse in this case, because you'd logically expect the July grade to include every game from September to April. Discarding data less than a year old seems a little peverse. After all, in July, the principal task for most players is predicting how they're going to play in ~3/4 months time not next week.

User avatar
Christopher Kreuzer
Posts: 8806
Joined: Fri Aug 06, 2010 2:34 am
Location: London

Re: Grading timetable (June/July 2013)

Post by Christopher Kreuzer » Mon Jul 29, 2013 1:15 pm

MartinCarpenter wrote:No, no misundertanding. (I hope :))

It is vanishingly rare for people to play >30 games in half a year, but far from unknown to do so over a single season. I say season and not year because serious club level chess is nearly all ~Sept <-> ~April with a non trivial summer break. So if you play 30 games a season you get to count the most recent two half seasons and your July grade is your entire previous season.
Ah, I see what you mean now. I think I was thinking of what you said here back at the beginning of July:

http://www.ecforum.org.uk/viewtopic.php ... 41#p117906

Possibly it was what someone else said that I thought was a misunderstanding, but I don't think I'd be able to find it again now.

(You can get a list from the public-facing part of the database of the most active players. Click on 'Top players' and then re-sort by 'Standard activity' - the top 120 list returned shows a range from 269 to 84 games in 2012-13 - quite why the columns for 2009 and 2010 are still there, I don't know).