ECF Grading Proposals

General discussions about ratings.
Roger de Coverly
Posts: 21291
Joined: Tue Apr 15, 2008 2:51 pm

Re: ECF Grading Proposals

Post by Roger de Coverly » Wed Oct 23, 2019 12:03 am

NickFaulks wrote:
Tue Oct 22, 2019 11:55 pm
Is there documentation of how the desired result was achieved, and why?
The archives of this site should cover it, in particular this very "Grading Debate" sub-forum.

It wasn't an issue that was ever put to the vote at an ECF Council meeting.

Dragoljub Sudar
Posts: 165
Joined: Wed Jun 16, 2010 10:34 pm

Re: ECF Grading Proposals

Post by Dragoljub Sudar » Wed Oct 23, 2019 12:04 am

Thanks Nick and Brian

Chris, I guess that if the ECF decide to convert the historical grades of ungraded players they may first convert them to 'new' ecf grades and then to the 4 digit.

I think the conversion was roughly old grade x. 81 + 43 or thereabouts. Before the conversion I was a 135 to 140 but then I became a 155 to 160.

MartinCarpenter
Posts: 3041
Joined: Tue May 24, 2011 10:58 am

Re: ECF Grading Proposals

Post by MartinCarpenter » Wed Oct 23, 2019 8:47 am

It was something like that yes. Basically trying to close a gap that had seemed to develop between weak(er) players & stronger ones. The immediate change obviously adjusted that.

It does have to be said that there were then a few years where the immediate effects of the change quite obviously worked through to affect the higher grading bands as well. I think that's broadly done now. Maybe the intended result was finally achieved, I don't know :)

NickFaulks
Posts: 8452
Joined: Sat Jan 02, 2010 1:28 pm

Re: ECF Grading Proposals

Post by NickFaulks » Wed Oct 23, 2019 9:11 am

MartinCarpenter wrote:
Wed Oct 23, 2019 8:47 am
Basically trying to close a gap that had seemed to develop between weak(er) players & stronger ones.
Surely the point of the system is that people playing graded games sorts that out? Anyway, it seems that I am coming to this issue ten years late.

Within the FIDE Rating System, this "stretching" has been seen to a marked degree, and is widely confused with "inflation". The differences in rating between the world #50 and the world #500, and between the #500 and the #5000, have steadily increased over the past twenty years. This is because in each case the former is now scoring more heavily against the latter than they used to. I believe there are identifiable reasons for this.
If you want a picture of the future, imagine a QR code stamped on a human face — forever.

MartinCarpenter
Posts: 3041
Joined: Tue May 24, 2011 10:58 am

Re: ECF Grading Proposals

Post by MartinCarpenter » Wed Oct 23, 2019 9:27 pm

It was meant to be something that wasn't sorting itself out 'naturally'. The threads I could see on here weren't super informative. Anyway. 2008 if you're really interested for some reason. Not really advised :)

Purely curiosity - especially with the big changes already done/coming! - but I'd actually be intrigued if anyone has checked if/how they worked out. As far as I can tell they seemingly meant 200 grades to stay stable - increases lower down - but they do seem to have picked up a bit.

I know I picked up a little bit more than my conversion was meant to give me.

Anyway :)

Brian Valentine
Posts: 574
Joined: Fri Apr 03, 2009 1:30 pm

Re: ECF Grading Proposals

Post by Brian Valentine » Thu Oct 24, 2019 7:40 am

MartinCarpenter wrote:
Wed Oct 23, 2019 9:27 pm

Purely curiosity - especially with the big changes already done/coming! - but I'd actually be intrigued if anyone has checked if/how they worked out. As far as I can tell they seemingly meant 200 grades to stay stable - increases lower down - but they do seem to have picked up a bit.

Anyway :)
It didn't work out well. Things have slowly been reversed, except the average grade was increased and is broadly unchanged from the new level. The dispersion of grades has steadily increased. For instance in 2009 3.9% were graded above 200 whereas 6.6% are now.

Roger de Coverly
Posts: 21291
Joined: Tue Apr 15, 2008 2:51 pm

Re: ECF Grading Proposals

Post by Roger de Coverly » Thu Oct 24, 2019 8:55 am

Brian Valentine wrote:
Thu Oct 24, 2019 7:40 am
For instance in 2009 3.9% were graded above 200 whereas 6.6% are now.
The effect of having more of the world's top players taking part in domestic graded events presumably influences this. Gibraltar, the London Classic and Isle of Man for example. The recent Grand Swiss brings even more of the top players into domestic grading or rating.

The point about the mean and median staying put makes an amount of sense. General reasoning suggests that many players should meet a field of approximately the same grade as themselves and score 50%. Presuming that any stretching is symmetric, what they gain through under or over performing in one direction, they lose in the other.

User avatar
Chris Goodall
Posts: 1057
Joined: Sun Oct 10, 2010 6:40 pm
Contact:

Re: ECF Grading Proposals

Post by Chris Goodall » Thu Oct 24, 2019 12:39 pm

Roger de Coverly wrote:
Thu Oct 24, 2019 8:55 am
Brian Valentine wrote:
Thu Oct 24, 2019 7:40 am
For instance in 2009 3.9% were graded above 200 whereas 6.6% are now.
The effect of having more of the world's top players taking part in domestic graded events presumably influences this. Gibraltar, the London Classic and Isle of Man for example. The recent Grand Swiss brings even more of the top players into domestic grading or rating.

The point about the mean and median staying put makes an amount of sense. General reasoning suggests that many players should meet a field of approximately the same grade as themselves and score 50%. Presuming that any stretching is symmetric, what they gain through under or over performing in one direction, they lose in the other.
Players near the median should meet a field of approximately the same grade as themselves and score 50%. But when I sampled 5 players graded exactly 200, the median of their opponents' grades over 140 games was 188. There aren't enough players of 200 strength for them to face.

The rule that every game from the most recent grading period is used in the calculation, could in theory cause "stagflation" of grades at the top. If the number of games played in ECF-graded tournaments drops off as you get higher into the 200s, because GMs would rather travel to a tournament in Eastern Europe than play in their club championship, then when you have someone who plays 30 games a year paired against someone who plays 60, the 60-game player is likely to be the weaker one. When the stronger player wins, they score a minimum of 1/3 of a Clarke point (10/30), but the weaker player loses only a minimum of 1/6 of a Clarke point (10/60), so at least a third of a Clarke point has appeared out of thin air.
Donate to Sabrina's fundraiser at https://gofund.me/aeae42c7 to support victims of sexual abuse in the chess world.

Northumberland webmaster, Jesmond CC something-or-other. Views mine. Definitely below the Goodall Line.

NickFaulks
Posts: 8452
Joined: Sat Jan 02, 2010 1:28 pm

Re: ECF Grading Proposals

Post by NickFaulks » Thu Oct 24, 2019 4:34 pm

Brian Valentine wrote:
Thu Oct 24, 2019 7:40 am
It didn't work out well. Things have slowly been reversed, except the average grade was increased and is broadly unchanged from the new level. The dispersion of grades has steadily increased.
To me, that is a good sign. The system is healing the damage done to it by the shock.
If you want a picture of the future, imagine a QR code stamped on a human face — forever.

Mike Gunn
Posts: 1014
Joined: Wed Apr 11, 2007 4:45 pm

Re: ECF Grading Proposals

Post by Mike Gunn » Thu Oct 24, 2019 5:23 pm

I never understood the argument for recalibration. It was Sean Hewitt's doing and he wrote a paper explaining why it was done and I suppose I should go back and read it, but my problem is this: the idea behind the grading system is that if 2 players with grades 25 apart play a match then one of them should win 3 games out of 4 (i.e. 75% = 50% +25%). But when Sean looked at all the results in the grading system the stronger player was scoring (say) 70%. (The argument is similar for other grading differences and the winning % was always less than it should be.) Now what I don't understand is why this happens - surely there is one dataset of game results and that is used to calculate the grades in the first place - so why aren't the calculated results consistent? I should apologise for bringing this up again but I will be eternally grateful if someone can explain this in a couple of sentences ...

E Michael White
Posts: 1420
Joined: Fri Jun 01, 2007 6:31 pm

Re: ECF Grading Proposals

Post by E Michael White » Thu Oct 24, 2019 7:09 pm

Mike Gunn wrote:
Thu Oct 24, 2019 5:23 pm
I never understood the argument for recalibration. It was Sean Hewitt's doing and he wrote a paper explaining why it was done and I suppose I should go back and read it, but my problem is this: the idea behind the grading system is that if 2 players with grades 25 apart play a match then one of them should win 3 games out of 4 (i.e. 75% = 50% +25%). But when Sean looked at all the results in the grading system the stronger player was scoring (say) 70%. (The argument is similar for other grading differences and the winning % was always less than it should be.) Now what I don't understand is why this happens - surely there is one dataset of game results and that is used to calculate the grades in the first place - so why aren't the calculated results consistent? I should apologise for bringing this up again but I will be eternally grateful if someone can explain this in a couple of sentences ...
Sean's Paper is here http://www.lrca.org.uk/lrca/Grading/Gra ... ction1.doc

It is unwise to assume that iterating until you reach a steady state will produce grades where most of the results align with the model probabilities. There is more than one set of probabilities to retain each steady state; changing the model probabilities each year might produce a closer alignment but is impractical.

Roger de Coverly
Posts: 21291
Joined: Tue Apr 15, 2008 2:51 pm

Re: ECF Grading Proposals

Post by Roger de Coverly » Thu Oct 24, 2019 8:06 pm

Mike Gunn wrote:
Thu Oct 24, 2019 5:23 pm
But when Sean looked at all the results in the grading system the stronger player was scoring (say) 70%. (The argument is similar for other grading differences and the winning % was always less than it should be.)
My observation on this was "so what". For those scoring around 50% and assuming they meet as many players graded among themselves as below, their under performance when higher graded would be offset by their out performance when lower graded. For the very top players, who only ever play lower graded opposition, their gap to the great masses would decrease if they failed to get their predicted 75%.

It's one of my criticisms of the ECF's structure that it was unable to strangle this wasteful project at birth. The resources devoted to it could arguably have delivered a monthly Elo based system ten years ago.

MartinCarpenter
Posts: 3041
Joined: Tue May 24, 2011 10:58 am

Re: ECF Grading Proposals

Post by MartinCarpenter » Thu Oct 24, 2019 10:44 pm

Thanks for that paper. Maybe it was a real problem. I do wonder how on earth they expected the change to stick. If someone is playing at a grade X vs a roughly stable field and you differentially uprate all their weaker opponents, they'll inevitably directly benefit from that.

That certainly seemed to happen from what I saw in Yorkshire (small sample mind!). Old style 170's got the inflation in the first few years peaking up to near 190 at times, then that wave of inflation moved up to the 190/200 grades & so on. Seemingly rather more stable - and slightly lower overall - now so whatever :)

User avatar
Chris Goodall
Posts: 1057
Joined: Sun Oct 10, 2010 6:40 pm
Contact:

Re: ECF Grading Proposals

Post by Chris Goodall » Thu Oct 24, 2019 11:04 pm

MartinCarpenter wrote:
Thu Oct 24, 2019 10:44 pm
Thanks for that paper. Maybe it was a real problem. I do wonder how on earth they expected the change to stick. If someone is playing at a grade X vs a roughly stable field and you differentially uprate all their weaker opponents, they'll inevitably directly benefit from that.
The same reason British Rail expected the Beeching Report to save them money. They got a set of recommendations: immediate drastic action followed by boring and difficult ongoing work. They assumed that the less drastic the action, the less important it was to the success of the whole exercise.
Donate to Sabrina's fundraiser at https://gofund.me/aeae42c7 to support victims of sexual abuse in the chess world.

Northumberland webmaster, Jesmond CC something-or-other. Views mine. Definitely below the Goodall Line.

User avatar
Chris Goodall
Posts: 1057
Joined: Sun Oct 10, 2010 6:40 pm
Contact:

Re: ECF Grading Proposals

Post by Chris Goodall » Thu Oct 24, 2019 11:29 pm

E Michael White wrote:
Thu Oct 24, 2019 7:09 pm
It is unwise to assume that iterating until you reach a steady state will produce grades where most of the results align with the model probabilities. There is more than one set of probabilities to retain each steady state; changing the model probabilities each year might produce a closer alignment but is impractical.
That chess results are logistic not linear is a fact independent of any other parameters of the grading system. You can no more force future results to conform to a linear expectation formula by carefully choosing the initial grades, than you can force a 2-dimensional map of the world to show you the true distance between every pair of cities by carefully drawing it.
Donate to Sabrina's fundraiser at https://gofund.me/aeae42c7 to support victims of sexual abuse in the chess world.

Northumberland webmaster, Jesmond CC something-or-other. Views mine. Definitely below the Goodall Line.

Post Reply