July 2012 grading list now live

[LawrenceCooper]

I agree with mleonard above. As a grader I have come across some cases where the results are quite sensitive to the "last 30 games" method. A much better system for adults would be to always use the last 12 months of results (and add weighted averages of previous years to bring it up to 30 if necessary). Some constituent organisations need to put down motions to Council to this effect, though, if it is ever to be properly considered.

You have completely ignored my post above.

The Board frequently complains about Council going into too much detail and not allowing the Board to do its job. However, your post demonstrates that action at Council is often necessary to try and get the Board to do what it should be doing anyway.

Will someone from the Board now please explain why the year available for considering these issues and testing alternatives was completely wasted?

I'm not able to but I've copied the link to this thread to the rest of the board for their comments. Grading comes under the Home Directorship so I hope an answer from that area will be forthcoming.

[Paul Cooksey]

I discussed the grading with Sean Hewitt in an earlier thread, and I am trying not to moan all the time, since it isn’t constructive. But on this point about 30 game count-back, I can’t help covering some old ground:.

I agree with most of David Sedgwick’s comments, but I’ll quibble slightly with; “So all the problems and issues which could and should have been sorted out before implementation are only now becoming apparent” since it could be interpreted as saying the issues were hidden until after implementation. These were things being discussed in advance, as Paul Mc has just pointed out.

I’ve said before I suspect 30 game count-back is a conspiracy, although it seems one that Mike Gunn is not party too. 30 games is an important number, as I understand it is the number of games Clarke felt necessary for players to be accurate placed in 8 point bands with reasonable accuracy. Given he was a good deal clever than me, and a lot more familiar with the statistical model, I am certainly not going to argue.

However 30 is not a magic number, if 30 games are played in a period, then that period could be considered to give a reasonably accurate result. But it does not define what is a relevant period. If it is necessary to include another period, then clearly including an average from that period gives a more reliable result. It is pretty easy to get 15 point swings if 2 or 3 games are taken from a previous period, if the player finished with a few wins or loses. Indeed you can get a 7 point swing if you only need one game, depending on whether the player won in the morning and lost in the afternoon, or lost in the morning and won in the afternoon.

The reason I believe in a conspiracy is that the grading team are not stupid, and presumably knew this long before I pointed it out. The only reason I could see to use exactly 30 games is if you intended to move to more frequent updates, where the periods involved were sufficiently short that they were not statistically reliable for most players (eg months, in which few people play 30 games). At that point, using the average of previous periods breaks down too. Your grade represents the last 30 games you played, however long that took, regardless of whether than is a logical group.

To my mind, monthly ECF grades are a bit silly. The methodology was defined to give someone a grade for a logical period, like a season. If you want to update grades regularly as you go along, Elo and related methodologies were designed for that purpose. But I’m guessing that somewhere there is an entrenched view we should keep the system designed on our own shores. Maybe if we divided by 10, the culture shock would be lessened?

[Adam Raoof]

I am not a statistician. However it would be useful if someone, probably Paul, gave some examples with working out!

If there are genuine queries about grades they should be addressed to Chris Majer or Richard Haddrell, of course.

[Roger de Coverly]

I’ve said before I suspect 30 game count-back is a conspiracy, although it seems one that Mike Gunn is not party too. 30 games is an important number, as I understand it is the number of games Clarke felt necessary for players to be accurate placed in 8 point bands with reasonable accuracy. Given he was a good deal clever than me, and a lot more familiar with the statistical model, I am certainly not going to argue.

It may be a conspiracy, but it shows a lack of long term planning. Whatever approach, you use towards moving to monthly grades, the thing you want least is to be continually re-estimating players. So why did they take that approach for Juniors?

The better way of doing monthly grades is to do what people have done in their score books for forty years, namely to work out what the grade for the season would be if you played no further games. So everyone has two grades, a frozen one used to calculate other peoples grades, the other being a dynamic one which measures recent form and results.

[Mike Gunn]

David, if I didn't answer your post (with which I happen to mostly agree!) it was because I had no answer. Discussions of grading issues have been fairly brief in my time on the board partly because we have lots of other things on our plate and partly because there does not seem to be a great appetite for discussing the minutiae of the grading system there.

I believe the fact is that 90% of members, Council and the board are happy to leave these matters up to the "experts". For the other 10% of us to make any impression somebody needs to put together a short paper/ argument on the subject. It is only some sort of coherent theoretical/ statistical case which stands any chance of making progress (in my view).

[Roger de Coverly]

I believe the fact is that 90% of members, Council and the board are happy to leave these matters up to the "experts".

Those making the decisions are conspicuous by their absence in making their case, even announcing what they've decided would be a help.

[Mike Gunn]

There are two strong reasons for sticking with the 12 month principle:

1. About 50 or 60 games worth of data is going to give you a more accurate answer than the last 30 games (this applies to the more active players who play more than 30 games a year).

2. The last 30 games method means that extra bias is given to games played just before the cutoff date - these can be included twice in a grading calculation at the end of successive 6 month periods. Thus a new bias is introduced which wasn't present in the old system.

If you want to move to monthly grades then Elo is the only sensible way to go.

[Roger de Coverly]

There are two strong reasons for sticking with the 12 month principle:

It's already been partly abandoned. If you play more than 30 games in a six month period, you get an X grade based only on the last 6 months. If you play almost 30 games, you get an X grade and a bit. In other words, all of the most recent six months and enough of the previous six months to make up the game count to 30. So players playing between 30 and 60 games, or not able to play at least 30 games in a half season will have their results skewed by their performances in games that were counted twice.

[E Michael White]

I think Stewart Reuben is probably the source of misinformation on frequent grading lists as in this post from 2009 : -

My reason for thinking only one rating list a year leads to deflation is because it has led to deflation.
Juniors are given a, one size fits all, of a certain age group, grading bonus at the end of the calculation. This remains with them for the whole year. If the bonus was inadequate, then that particular player will deflate the grades of his opponents. The type of player for whom the bonus is inadequate is likely to be one who carries on playing chess, thus playing more games. The type of player for whom the bonus is too high is much less likely to continue playing. Thus one has deflation.
Of course the bonus should be individualised to each person, not just juniors. A player has gone up 20 points in the year, give him a bigger bonus than one who has only gone up 10.

But if you had say quarterly grading lists and continued to roll over the last 30 games, then the problem would be far less acute. In addition it would help popularise chess, lead to more activity as internationally, in the US and London when they had monthly lists.

Having frequent grading lists or rolling the last 30 games won’t fix deflation/inflation without other changes in fact they will exacerbate any issues.

[David Sedgwick]

David, if I didn't answer your post (with which I happen to mostly agree!) it was because I had no answer. Discussions of grading issues have been fairly brief in my time on the board partly because we have lots of other things on our plate and partly because there does not seem to be a great appetite for discussing the minutiae of the grading system there.

That's undeniably true. Those of us who tried to get the Board to discuss the problems and issues surrounding the introduction of "new" grades a few years ago ran up against a similar brick wall.

The Board would do well to remember that the ECF provides one and only one service that matters to the vast majority of players.

That service is the grading system.

[Paul McKeown]

I am not a statistician. However it would be useful if someone, probably Paul, gave some examples with working out!

Adam, Mike Gunn's reply, point 2, is a very strong argument, I think:

2. The last 30 games method means that extra bias is given to games played just before the cutoff date - these can be included twice in a grading calculation at the end of successive 6 month periods. Thus a new bias is introduced which wasn't present in the old system.

That certainly has been my experience when considering my performance last season.

In 2010-11 I was 185 based on 19 games, with 11 games taken from 2009-2010 when I was graded 183.

The 6 monthly grading cycle - and the implementation of the 30 game cut-off - started with the first half of 2011-12. I broke my elbow, had my right arm in a sling and was having to take strong painkillers. Naturally my results were poor for the affected games, I could barely write the game scores and moving the pieces and stopping/starting the clock with my left hand was slow and awkward, just to start. So my result for the 5 games in that grading period was an abnormally low 165, giving a grade of 182 when previous results were added in from 2010-11 and 2009-10.

Okay, arm out of sling, my results bounced back in the next six month period. I didn't lose to anyone below 190 for the whole period and was handing out some rather severe cuffings to lots of those sub 190 players. But then I noticed some rather odd things happening in a spreadsheet I had set up to work out my grade. My grade was starting to go down, even when I was entering new won games! Investigation showed that what was happening was an artefact of the 30 game cut-off. Rather than smoothing small dips or peaks, which is what the grading system did when adding whole previous seasons' results, with the 30 game cut-off, at some point the 5 crappy games from the autumn of 2011 were starting to dominate the resulting grade.

What I found was I could calculate (with reasonable accuracy in hindsight) exactly what my grade would be based on a specific number of games played in the current 6 month period. In the end I decided to play some games that I hadn't anticipated on playing, including in my club's championship, which I hadn't done in a number of years, simply to get exactly 30.

Okay, 30 games played, 193 achieved. I played 6 games that I wouldn't otherwise have played. From memory, the 24 I would have played were also at 193, but if I hadn't played the extra 6, my grade would have been 188. Under the old system without the 30 game cut-off, it would have been 192.

I know people back in the days of annual gradings and without the 30 game cut-off, who played 30 to get completely flush out old results, but usually the effect wasn't a drastic as it now is with the 30 game cut-off.

I think the grading system is more volatile, spikier, less smooth with the 30 game cut-off than before. I also think it is more vulnerable to manipulation. If people wish to move to monthly grading, then they really out to move to an Elo based system. RWB Clarke's system wasn't designed for this. If people want to move to an Elo based system and want to do so so gradually that no one notices, then at some point they will destabilise the system so that they have to take a large jump the rest of the way.

[Paul Cooksey]

The Board would do well to remember that the ECF provides one and only one service that matters to the vast majority of players.

That service is the grading system.

I agree strongly. It was the single showstopper reason to remain in the ECF in some membership debates I attended.

[Sean Hewitt]

1. About 50 or 60 games worth of data is going to give you a more accurate answer than the last 30 games (this applies to the more active players who play more than 30 games a year).

I am a statistician and can confidently say that this is not always the case.

However, before deciding on a methodology you have to agree what you want a grade to tell you - and I'm not sure that everyone agrees on what that is!

[Adam Raoof]

The Board would do well to remember that the ECF provides one and only one service that matters to the vast majority of players.

That service is the grading system.

I agree strongly. It was the single showstopper reason to remain in the ECF in some membership debates I attended.

Point taken - I agree. Nobody can say I haven't tried to move the grading service in a positive direction. If there is a problem with the methodology I am not the right person to address that. However I do not think arguments about the grading system can be resolved to everyone's satisfaction. I think what we have is very good, and I agree with Mike Gunn - if someone disagrees then the action they need to take is to write a paper and submit it to the grading team for consideration.

[Krishna Shiatis]

Point taken - I agree. Nobody can say I haven't tried to move the grading service in a positive direction. If there is a problem with the methodology I am not the right person to address that. However I do not think arguments about the grading system can be resolved to everyone's satisfaction. I think what we have is very good, and I agree with Mike Gunn - if someone disagrees then the action they need to take is to write a paper and submit it to the grading team for consideration.

I would like to say thank you to Adam and the grading team for the sweeping changes they have made and for all the extra information we now have at our fingertips. It is very useful and sometimes a little scary that so much information is out there, but it must have been a mammoth task and it is much appreciated by me.

Thank you also for listening to our suggestions (re FIDE age groups etc).

Kind regards,

Krishna