When is the new grading list coming out?

General discussions about ratings.
E Michael White
Posts: 1420
Joined: Fri Jun 01, 2007 6:31 pm

Re: When is the new grading list coming out?

Post by E Michael White » Wed Aug 19, 2009 10:11 am

Matthew Turner wrote:2. Which game did he draw? surely if he drew with the ungraded player this would make a difference (in simple terms the ungraded player would come out higher than normally expected because he had drawn with a 219!)
Looking at a new/junior player's grade in isolation it makes no difference where he gets the points, once you are at the point of checking a grade and all new grades are to hand. Before that point Matthew is right there is some aplifying feedback; a draw with the ungraded would have meant that both that opponent and other juniors would have had slightly higher grades.

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: When is the new grading list coming out?

Post by Roger de Coverly » Wed Aug 19, 2009 10:20 am

Mike Gunn wrote:I understand that some changes could be made to grades at the end of August revision, but there is a limit to what can be done because of the time factor.
It seems to many of us that changing the methodology of grading calculations by "throwing away juniors previous grade" is such a fundamental change to the grading system that it shouldn't have been introduced with no prior public discussion giving about one week's notice before the start of the season. At the very least we would expect the board to formally approve the changes on behalf of the membership, thus making them responsible.

Sean Hewitt

Re: When is the new grading list coming out?

Post by Sean Hewitt » Wed Aug 19, 2009 10:33 am

Roger de Coverly wrote: This goes back to one of last year's "debates". You may recall that the historic averages of published grades going back to 1994 were retrieved and didn't show much of a trend. Certainly not the 1 per year beloved of the deflation theorists.
Roger, without wishing to open old wounds that simply is not true at all. You may remember my post here http://www.ecforum.org.uk/viewtopic.php ... 1996#p4400 which in turn quotes from a post from the SCCU forum going back to 2006.

Examing only players who appeared both in the grading lists of 1996 and 2006 so as to remove the effects of leavers and joiners of the grading system (a method that the doubters at the time now seem to accept as being the correct one) we see that the grades of these 4815 players have changed thus

1996 grade Change in grade
200-217 4.9
190-199 1.77
180-189 -0.89
170-179 -1.8
160-169 -3.76
150-159 -2.46
140-149 -2.4
130-139 -3.74
120-129 -4.66
110-119 -5.29
100-109 -4.28
90-99 -8.43
80-89 -9.01
70-79 -12.19
60-69 -14.48
50-59 -14.87
0-49 -22.21

So every group of players has seen their grades go down in that 10 year period apart from the very highest graded players, and the lower the initial grade the greater the reduction. That certainly looks like a trend to me.

Sean Hewitt

Re: When is the new grading list coming out?

Post by Sean Hewitt » Wed Aug 19, 2009 10:41 am

Roger de Coverly wrote:
Mike Gunn wrote:I understand that some changes could be made to grades at the end of August revision, but there is a limit to what can be done because of the time factor.
It seems to many of us that changing the methodology of grading calculations by "throwing away juniors previous grade" is such a fundamental change to the grading system that it shouldn't have been introduced with no prior public discussion giving about one week's notice before the start of the season. At the very least we would expect the board to formally approve the changes on behalf of the membership, thus making them responsible.
Agreed. I'm surprised that council wasn't asked as it is, as you say, a fundamental change.

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: When is the new grading list coming out?

Post by Roger de Coverly » Wed Aug 19, 2009 10:43 am

Sean Hewitt wrote:So every group of players has seen their grades go down in that 10 year period apart from the very highest graded players, and the lower the initial grade the greater the reduction. That certainly looks like a trend to me.
Would you care to tell us the number of players in each band and the percentage they represented of the published players in each year? One should be perhaps be cautious about drawing global conclusions from data subsets, particularly where the "obvious" tests like the means don't show much.

You've got the "run the recursion and compare to published" stuff and the above but are there any other supposed pointers to the existence of deflation?

Neill Cooper
Posts: 1298
Joined: Sun Feb 24, 2008 4:43 pm
Location: Cumbria

Re: When is the new grading list coming out?

Post by Neill Cooper » Wed Aug 19, 2009 11:36 am

Thinking of all the effort and expertise participants here (in particular Roger and Michael) have put into analysing the grading list in the absence of full details, I wondered if they could help (or even join) the grading team?

E Michael White
Posts: 1420
Joined: Fri Jun 01, 2007 6:31 pm

Re: When is the new grading list coming out?

Post by E Michael White » Wed Aug 19, 2009 11:44 am

Matt Harrison wrote:My guess is that pass 1 just uses the games between 2 graded opponents to set their grades. Then pass 2 works out the grades for ungraded players based on their games against graded opponents. Then pass 3 works out the grades for ungraded versus ungraded. Then pass 4 redoes the entire grading using the values obtained from passes 1, 2 and 3. This is re-run until the grades converge. And I suspect the 40 point rule is included in each pass.
If in pass 4 you are using grades of graded players which remain unmodified, then the results are the same as ignoring the estimates in your passes 1-3 and starting pass 4 off using any estimates for ungraded players. These could all be set to 0 or 1000 or whatever you like, without changing the emerging grades for the ungraded; this is a property of markov processes.
Last edited by E Michael White on Wed Aug 19, 2009 8:57 pm, edited 1 time in total.

Ola Winfridsson
Posts: 324
Joined: Fri Aug 14, 2009 12:26 pm

Re: When is the new grading list coming out?

Post by Ola Winfridsson » Wed Aug 19, 2009 12:02 pm

Sean: Even if we do accept this as a trend, there at least a couple of issues that must be raised in conjuction with this:

1) The average age of those 4,800 players in each respective banding compared to the average age in these bands for the list as a whole. Elite players tend to be at the strongest between 25 and 40 years of age. However, for us amateurs that particular age span will in most cases coincide with (in some cases rapid) career advancement, getting married, starting a family etc. and somewhere between the ages of 45-50 our general chess strength definitely tends to slowly wane. So if the average age was let's say 30 in 1996, it's not unlikely that aged 40, the grade would have gone down, due to other commitments giving less time for chess

2) Even if we don't take the assumption that the general standard of chess is increasing into account, it's only fair to assume that juniors and young adult players - as well as those "extra strong joiners and leavers" - who've entered the list after 1996 are likely to progress quicker and perhaps to a slightly higher standard than the 4,800 stayers - quite simply because up to the age of 25 (or so) we tend to have more time for chess (or in the case of the stronger than average joiners and leavers, spending more time on chess). The nature of the ECF system (which to all intents and purposes is relative, not absolute) would then "punish" the stayers who for one reason or another are not able to put in the same amount of work on their chess.

In other words this trend is likely (in my view) to be the result of an analysis of established players (whose playing strength is more likely than not to have stabilized), but with probably less time and incentives to work on their chess, and a disregard of the rapidly improving players and other players who put in more work on their chess.

It would be very interesting to see a comparison of just those 4,800 players going back to 1986 (and perhaps also 1976) - I wouldn't be surprised if we would then see an increase in their grade.

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: When is the new grading list coming out?

Post by Roger de Coverly » Wed Aug 19, 2009 12:53 pm

Ola Winfridsson wrote:Even if we do accept this as a trend, there at least a couple of issues that must be raised in conjuction with this:
It's also really odd (to my competitive mind) that players are content to keep sub-50 grades over a long period of years and their years of experience don't seem to enable them to get a bit better. I did some lucky dip browsing in the download list and they do exist. It goes into that strange but true box. Whether it's because of some local cluster, that their search depth is no more than 3 ply or their positional sense limited to "is this move legal" is difficult to say. It's also true of course that the 40 point rule makes it more difficult to escape from the bottom depths of the list.

Alex Holowczak
Posts: 9085
Joined: Sat May 30, 2009 5:18 pm
Location: Oldbury, Worcestershire

Re: When is the new grading list coming out?

Post by Alex Holowczak » Wed Aug 19, 2009 1:50 pm

Roger de Coverly wrote:
Ola Winfridsson wrote:Even if we do accept this as a trend, there at least a couple of issues that must be raised in conjuction with this:
It's also really odd (to my competitive mind) that players are content to keep sub-50 grades over a long period of years and their years of experience don't seem to enable them to get a bit better. I did some lucky dip browsing in the download list and they do exist. It goes into that strange but true box. Whether it's because of some local cluster, that their search depth is no more than 3 ply or their positional sense limited to "is this move legal" is difficult to say. It's also true of course that the 40 point rule makes it more difficult to escape from the bottom depths of the list.
I suppose it may be the fact that people have to stumble across things, rather than be taught some simple principles. Things that seem "simple" to good players might not be so for others. Then they may have other interests, and only play chess for a bit of fun. So their mind isn't concentrated on it. They might fall into the Queen trap in the Petrov's, then forget about it through inactivity for 6 months and fall into it again.

I've been trying to learn how to play Go recently, and I played a friend online yesterday who was born in China (relevant, because it's big over there), but last played ten years ago as a 5-year old. He was still much better than me! He explained very simple concepts that made perfect sense, but just didn't occur to me until someone explained it. I suppose it's just a question of experience and coaching. Not every chess player is lucky enough to have someone of a high standard coaching them.

Edit: I just beat him! :D

Then again, maybe it just doesn't bother them. They maybe play social games against friends, and they're of a roughly equal standard, and don't really want to improve. They might go to a club for the social aspect, play a few games for them now and again to fill in for regulars or something to get a grade, and be content with that. Most people might play poker without any real attempt to do things that a professional would do. Very few amateur poker players start studying the mathematics of it, they just play now and again for something to do.

Howard Grist
Posts: 83
Joined: Tue Dec 12, 2006 1:14 pm
Location: Southend-on-Sea

Re: When is the new grading list coming out?

Post by Howard Grist » Thu Aug 20, 2009 2:08 am

My, this topic seems to have been quite active during my week's 'holiday'. As before, these comments are my own and not necessarily those of ECF. I will also, for the moment, avoid commenting on the intricacies of the grading methodology for the moment. This is not because it is an ECF secret but I will need some time to digest what has been written and is almost certainly only of minority interest.

I suppose the the most important point I have to make is that the method of calculating grades for both juniors and new players will be changing for the end of August list. The exact details of these changes are still being finalised.

Some comments/replies on specific points: I last posted on page 16, so you may have to page back quite a bit for what I'm referring to.
Ian Thompson wrote:
Howard Grist wrote:If both players score at least 3 points ...
Could you explain this please?
If one of the players fails to score 3 points in the 30 game match then they will have scored less than 10% and the 40 point rule will come into operation.
Sean Hewitt wrote:then why do you need the junior increment at all??
The only purpose of the junior increment is so that organisers don't have to advertise different grading bands for juniors. The 2008 list incorporated junior increments and these could hardly be withdrawn without giving notice to event organisers.
Roger de Coverly wrote:
Alex Holowczak wrote:Indeed, they said that instead of someone graded 190 having a 60-40 chance against someone graded 180, it was 57-43 instead. As a result, the system was changed. What exactly was wrong with it not being 60-40?
Various statements were made with little (in my opinion) convincing backup as to exactly how this was measured.
This is quite simple to measure: You look at the results of all games in which the results of all games in which the players are graded 10 points apart. In these games the stronger player only scores 57%. A full chart is shown below. The blue line denotes the results of games played between players with category A-C grade. This ought to follow the red/green lines, particularly between the fractional scores of .2 and .8. The fact that it doesn't would indicate that there is a problem.
Standard.gif
Paul McKeown wrote:...when it was pointed out in the early 2000's that FIDE ratings and BCF grades seemed out of kilter according to the formula FIDE = BCF * 8 + 600. Of course this was true, but the real problem lay with the FIDE ratings being inflated.
There is a problem with FIDE ratings which have inflated. If you do a plot of FIDE rating difference of the two players and game results you come up with a similar graph to the one above. This is hardly evidence that all is well with the ECF system!
Roger de Coverly wrote:I've never been convinced there was much of a problem in the first place - at least not in the top quartile.
If you plot the actual results against grading differences for players in the top quartile you will come up with a very similar chart to the one above. This would indicate that the grading system is not working correctly for the top quartile and no piecemeal solution could be employed.
the non-existent Jade Hoobe wrote:If all juniors this season are being treated as new starters, that means all their results last season were calculated without the 40 point rule being taken into consideration. Therefore, am I correct in thinking that if, last season, an former 80-graded junior player lost to a 140 graded adult player, the junior would still have received 90 points (ie 10 points MORE than their previous grade despite losing the game) ? If so, no wonder many juniors have seemingly inflated grades.
No, this doesn't happen. Results from previous seasons are calculated using last season's grades and the junior would get 70 points for a loss against an adult graded 140.
John Upham wrote:I believe both Howard Grist and Richard Haddrell are paid £3k per annum.
Your belief is incorrect. The only person on the current grading team who has ever received any money from the ECF for their grading work is Richard Haddrell. Additionally the amount that he receives is not 3K per annum.
Brian Valentine wrote:It is a complete fluke if the system comes out as zero sum. It depends on the relative grades of leavers and joiners. Here is an analysis of last year's movements.

The 2008 list I refer to is the 2008 grades posted in 2009 or, if missing, the red grade from the 2008 list

2008 list: 10257 entries at an average of 133.8
leavers: 1589 at an average of 126.8
in both lists: 8668 at 135.0 in 2008 increasing to 135.4
joiners: 1489 at 122.5.

Giving an overall 2009 average of 133.5

The joiners are only so high because a significant proportion of them are overseas with a high grade. My guess is these go in at an estimate of their elo conversion (something I don't think has been discussed here). You only need the numbers or the averages of these leavers and joiners to change slightly to start observing material inflation or deflation. The real "currency change" is how and why the survivors increase by 0.4. Note that this in in the opposite direction to the overall average.
If the people in the both lists includes a junior or two, and juniors are improving (or at least given points in a belief that they are) and the adults are staying the same (fairly central grading system principle), it is hardly surprising that for players in both lists the mean 2009 grade is higher than that in 2008.
Roger de Coverly wrote:
Neill Cooper wrote:I hear that the grading team met last weekend
Do we know who they actually are and who takes the decisions?
On the basis of recent meetings, the team would seem to comprise Howard Grist, Richard Haddrell, Carl Hibbard, Cyril Johnson, Chris Majer, Roger Marsh, Dave Thomas & David Welch. It doesn't have a formal role, decisions are the responsibility of Dave Thomas, the Manager of Grading & Rating but he tends to go by the general consensus of the grading team.
You do not have the required permissions to view the files attached to this post.
Former ECF Grading System Programmer

User avatar
Carl Hibbard
Posts: 6028
Joined: Fri Dec 08, 2006 8:05 pm
Location: Evesham

Re: When is the new grading list coming out?

Post by Carl Hibbard » Thu Aug 20, 2009 8:12 am

Howard Grist wrote:I suppose the the most important point I have to make is that the method of calculating grades for both juniors and new players will be changing for the end of August list. The exact details of these changes are still being finalised.
Thanks Howard but an "official" post on the ECF site would help now this information is out in the public domain :!:
Cheers
Carl Hibbard

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: When is the new grading list coming out?

Post by Roger de Coverly » Thu Aug 20, 2009 8:17 am

Howard Grist wrote:I suppose the the most important point I have to make is that the method of calculating grades for both juniors and new players will be changing for the end of August list. The exact details of these changes are still being finalised.
This "making it up as you go along" approach is immensely unsatisfactory particularly when you are modifying a system with a reasonable fifty year track record. Why not just leave the system alone for another year and stop meddling until you have a coherent approach?

Howard Grist wrote:This is quite simple to measure: You look at the results of all games in which the results of all games in which the players are graded 10 points apart. In these games the stronger player only scores 57%. A full chart is shown below. The blue line denotes the results of games played between players with category A-C grade. This ought to follow the red/green lines, particularly between the fractional scores of .2 and .8. The fact that it doesn't would indicate that there is a problem.
This could demonstrate a number of issues not least a non-linear relationship of results between players of different grades. You seem to be drawing conclusions about the entireity of the grading system from a subset of its data. To make such a comparison valid, surely you should recalculate the grades using A-C players only.

It also seems to me that when you compare actual results to predicted results, you are measuring a mixture of two things - the reliability of the grade and the consistency of the players. Particularly with juniors, consistency may not be high from one game to the next.

User avatar
Carl Hibbard
Posts: 6028
Joined: Fri Dec 08, 2006 8:05 pm
Location: Evesham

Re: When is the new grading list coming out?

Post by Carl Hibbard » Thu Aug 20, 2009 8:21 am

Roger de Coverly wrote:Why not just leave the system alone for another year and stop meddling until you have a coherent approach?
A gutsty call and one I would favour to be honest, there has to be complete confidence in new grades and now that is lacking I feel

But then I don't make the call :oops:
Cheers
Carl Hibbard

Brian Valentine
Posts: 574
Joined: Fri Apr 03, 2009 1:30 pm

Re: When is the new grading list coming out?

Post by Brian Valentine » Thu Aug 20, 2009 8:36 am

<The fact that it doesn't would indicate that there is a problem.>

Howard,
The graph does not necessarily indicate that there is a problem. It is inevitable that this situation will arise: Glickman discusses it with regard to the US grading list, the Scots have it and so does elo. I have posted the maths in an earlier thread. There is a valid question on whether it diverges too much.

<If the people in the both lists includes a junior or two, and juniors are improving (or at least given points in a belief that they are) and the adults are staying the same (fairly central grading system principle), it is hardly surprising that for players in both lists the mean 2009 grade is higher than that in 2008.>

Again the issue is more whether the increase is reasonable. There is wider circulation going on: Increasingly active player's grades rise while those becoming less active tend to fall. I was not suggesting that the increase is a problem, but somehow the graders have to form an opinion on whether the changes are "true" rather than plain inflationary.

It has been an eye opener for me, looking at the data; the situation is far more dynamic than I had realised. I do hope that you and the team can get the list into a better place.