GRADING ANOMALIES

General discussions about ratings.
User avatar
Robert Jurjevic
Posts: 207
Joined: Wed May 16, 2007 1:31 pm
Location: Surrey

Re: GRADING ANOMALIES

Post by Robert Jurjevic » Mon Oct 24, 2011 1:22 pm

Players' list in grade improvement order for 2011 season since 2008 season...

Please find the players' list in grade improvement order for 2011 season since 2008 season (PDF document) at:
http://www.jurjevic.org.uk/chess/grade/11i08.pdf

column legend:

REF - ECF grading reference ID
NAME - full player's name
SEX - player's sex
FIDECODE - player's FIDE code
NATION - player's nationality
GAMES - number of standard graded games the player played in 2011 season
CATEGORY - 2011 season grading category based on GAMES
GS - player's GS grade for 2011 season
AGS3 - player's AGS3 grade for 2011 season
GSLAST09 - player's GS grade for 2008 season
AGS3LAST09 - player's AGS3 grade for 2008 season
GS-GSLAST09 - player's GS grade improvement in 2011 season since 2008 season
AGS3-AGS3LAST09 - player's AGS3 grade improvement in 2011 season since 2008 season

GS grade - official ECF current Grading System grade
AGS3 grade - ECF Amended Grading System three grade
(player's AGS3 grade for 2008 season was taken to be the same as player's GS grade for 2008 season)

Grading anomalies...

Example: Since 2008, according to GS, the grades of 273626D John Paul J Taylor and 121043K Steve Walls drifted apart for 119 grading points, but according to AGS3 they should have drifted apart for about 87.97 grading points. The question is is the 87.97 a better guess than the 119? (It is obvious that GS stretches the grades in respect to AGS3. If it has been found that the GS grades were stretched, if the stretching is inherent in the GS calculation method, the problem could be resolved by correcting the calculation method by say applying AGS3 or better ÉGS6.)

P.S. ...

what is PDF?
http://en.wikipedia.org/wiki/Portable_Document_Format

download PDF reader
http://get.adobe.com/uk/reader/enterprise/
Robert Jurjevic
Vafra

User avatar
Robert Jurjevic
Posts: 207
Joined: Wed May 16, 2007 1:31 pm
Location: Surrey

Re: GRADING ANOMALIES

Post by Robert Jurjevic » Wed Sep 26, 2012 8:04 pm

ECF grading anomalies and ÉGS6 as a possible solution...

Please allow me to informally appeal to Mr Chris Majer, who is Manager for Grading, for his consideration of my proposal to change the ECF grade calculation method.

What is meant by Élo...

May I say that when talking of Élo or any rating system one should take into account 1) rating scale, 2) relationship between expected performance 'p' and grade difference 'd', 'p = f(d)', where 'f' is some function (in Élo's time that was so called 'normal relationship', today that is so called 'logistic relationship') 3) and how often rating corrections are done (or if one rates on statistically significant data).

Élo FIDE or national rating scale is different from ECF grade scale. The relation is 'Élo = K + ECF*8' where 'K' is some constant. Consequently 'Élo2-Élo1 = (ECF2- ECF1)*8', i.e., say a difference of 10 ECF grading points is equivalent to a difference of 80 Élo rating points. An ECF grade (FIDE rating) difference of 10 grading (80 rating) points means that the expected performance of the stronger player is 50%+10=60% (this is valid for grading differences of approximately 30 grading points or less).

If actual performance (results from the games) is not as expected (according to the grades) the grades are corrected. One of the factors which affects the grade correction is the relationship between expected performance 'p' and grade difference 'd', 'p = f(d)'. Current ECF calculation method uses a fairly good linear approximation to 'logistic relationship' or 'normal relationship'. Nonlinear 'p = f(d)' only matters for grading games where grade difference is greater than about 30 grading points, otherwise the ECF linear approximation should be adequate.

How often the grades are corrected is very important. ECF luckily corrects grades taking into account all of the games from a grading period which is taken to be such that enough games have been played in it in order to regard the actual performance results statistically significant. In such cases the situation ought to be simple and the grades should be corrected so that the new grades match the actual performance in the grading period (for players who played less than 30 games in the grading period the grades from previous grading periods are taken into account). I have found that the current ECF grading calculation method changes the grades for too much, i.e., it grade-wise over-rewards improving and over-penalizes receding players which (in my opinion) results in grade stretching which means that in time (as the grading periods pass) a grade gap between two players may become greater than what would their actual performances suggest (say in five years the error may accumulate so that the gap between two players is 140 rather than 110, say a lower graded player would be 90 rather than 120 when compared to a 230 player, at one point such grade correction has been made when grades of weaker players had been increased in order to reduce the grade gap).

Élo FIDE or national rating system rates after periods in which a smaller number of games have been played (where the actual performance results are not statistically significant), say after every tournament or even after every game (live ratings). This complicates the situation as it may not be clear how much of the rating correction has to be applied for each individual game or tournament. Still the rating obtained by rating after every game or tournament should match (more or less) that which would be obtained if rating was done after every rating period with statistically significant actual performance. ECF assumes that if a player has played 30 games or more in a grading period the actual performance result is statistically significant (there is enough experimental data) which I believe is a fair assumption.

Improvements to the present grade calculation method....

I proposed three improvements to the present grade calculation method which I named AGS3, ÉGS5 and ÉGS6 grade calculation method (the numbers were used to distinguish between various calculation methods proposed as the idea was developing), each of which uses ECF (not FIDE) rating scale (a strong grandmaster is 270 not 2,800) and with the grades being calculated after grading periods where enough games have been played to regard the actual performances statistically significant.

AGS3 is the simplest and only corrects the error which is causing the grade stretching. It is basically ECF grading calculation method which applies less correction to grades in order not to stretch the grades. The same ECF linear approximation to relationship between expected performance 'p' and grade difference 'd', 'p = f(d)', is used.

ÉGS5 further improves over AGS3 so that it uses 'logistic relationship' between expected performance 'p' and grade difference 'd', 'p = f(d)'. This affects grade calculation in games where the grade difference is approximately larger than 30 grading points. ÉGS5 stands for Élo Grading System five and Élo is mentioned here for 'p = f(d)' and not for grade scale (i.e., grade scale of ÉGS5 is the same as that of AGS3 or current grading system, i.e., a strong grandmaster is 270 not 2,800) or how often the grades are calculated (i.e., the grades are calculated after grading periods where enough games have been played to regard the actual performances statistically significant).

ÉGS6 further improves over ÉGS5 so that it accounts for grade trust based on frequency of play. This is Professor Glickman’s idea to change less trusted grades more rapidly than more trusted grades. I estimated grade trust (rather primitively) using the number of games the players played in the last season. Grades of less active players would change more rapidly affecting the grades of more active players less. This would be good for two reasons 1) the grades of less active players would be corrected faster (they anyway cannot be trusted and may be wrong) and 2) the grades of more active players would not be affected as much with the games the more active players have played against the less active players.

ÉGS6 would be the best but AGS3 would be adequate and the only change would be to use the following grading rule:

Rule 2b: For a win you score average grade plus 25; for a draw, average grade; and for a loss, average grade minus 25. Average grade is half of the sum of your and your opponent's grade. Note that, if your opponent's grade differs from yours by more than 40 points, it is taken to be exactly 40 points above (or below) yours. At the end of the season an average of points-per-game is taken, and that is your new grade.

Detailed argument...

Detailed argument (in somewhat informal form) can be found at the link below...
http://www.jurjevic.org.uk/chess/grade/ ... malies.htm

Apologies...

Please accept my apologies for persisting in this matter.
Robert Jurjevic
Vafra

User avatar
Greg Breed
Posts: 723
Joined: Thu Apr 05, 2007 8:30 am
Location: Aylesbury, Bucks, UK
Contact:

Re: GRADING ANOMALIES

Post by Greg Breed » Thu Aug 08, 2013 4:59 pm

Hey Robert, have you heard back from anyone at the ECF about your proposals?

Has anyone thought about using 4-digit grades using the ECF system?
Convert ECF to 4-digit grade by multiplying by 8 and adding 650.
Then in the ECF grade calculating formula change the +/- 50 with +/- 200; and the 40 point rule would become the 160 point rule. At least this way we'd start to get closer looking to FIDE grades. I don't know what effect that would have, but there you go. Something to kick start this thread again.
Hatch End A Captain (Hillingdon League)
Controller (Hillingdon League)

User avatar
Robert Jurjevic
Posts: 207
Joined: Wed May 16, 2007 1:31 pm
Location: Surrey

Re: GRADING ANOMALIES

Post by Robert Jurjevic » Wed Sep 25, 2013 4:01 pm

Hello Greg,

Sorry for the delayed reply.
Greg Breed wrote:Robert, have you heard back from anyone at the ECF about your proposals?
Yes, there were some ECF officials keen to bring the "grading anomalies" (and my proposed solution, i.e., ÉGS6 ECF grade calculation method) to attention, but I haven't got any feedback yet.
Greg Breed wrote:Has anyone thought about using 4-digit grades using the ECF system? Convert ECF to 4-digit grade by multiplying by 8 and adding 650.
I think present ECF scale (a strong grandmaster being ECF 270 rather than FIDE 2800) makes more sense (than Élo scale used by FIDE) as ECF grading points correspond directly to the performance percents. You may wish to look for more details at the following link (in the "ECF grade vs FIDE rating scale..." section): http://www.jurjevic.org.uk/chess/grade/ ... malies.htm.
Robert Jurjevic
Vafra

David Gostelow
Posts: 40
Joined: Mon Apr 15, 2013 9:49 am

Re: GRADING ANOMALIES

Post by David Gostelow » Fri May 23, 2014 1:47 pm

The last 2 weekends in the 4ncl I have won one drawn one. The player being in the 1800-1900 bracket . Because each of the ones I won did not have a proper FIDE rating I ended up dropping rating points when my overall performance for the weekend was in ECF terms positive to equal. The 2 games won were against the slightly lower rated players (ECF) - To me this looks like a systematic anomaly . I play the lower rated players and win do not get graded and draw to the higher players and lose points. Any thoughts ??

NickFaulks
Posts: 8452
Joined: Sat Jan 02, 2010 1:28 pm

Re: GRADING ANOMALIES

Post by NickFaulks » Fri May 23, 2014 2:34 pm

Is this about your ECF grade or your FIDE rating?
If you want a picture of the future, imagine a QR code stamped on a human face — forever.

MartinCarpenter
Posts: 3041
Joined: Tue May 24, 2011 10:58 am

Re: GRADING ANOMALIES

Post by MartinCarpenter » Fri May 23, 2014 3:03 pm

It'll be about his FIDE grade and, no it isn't systemic because it could easily happen the other way round :)

What it is of course is another factor ensuring that FIDE grades at the lower levels of the 4NCL etc are randomised in amusing fashion. They're not nearly accurate enough to be worth worrying about.
(Its a good system of course but can't avoid GIGO.).

NickFaulks
Posts: 8452
Joined: Sat Jan 02, 2010 1:28 pm

Re: GRADING ANOMALIES

Post by NickFaulks » Fri May 23, 2014 3:40 pm

I'm not too worried about that. When a player gets an initial rating based on nine games, we have a good idea of how accurate that is likely to be. The fact that he may have played other games in similar conditions which had to be excluded is a pity, but not a problem. Of course, it could be systemic if he were tended to be overawed when facing an opponent with a FIDE rating, but only the player can solve that.

I don't even understand this particular complaint. In weekend 4 he beat a rated opponent and gained 16 rating points.
If you want a picture of the future, imagine a QR code stamped on a human face — forever.

Roger de Coverly
Posts: 21291
Joined: Tue Apr 15, 2008 2:51 pm

Re: GRADING ANOMALIES

Post by Roger de Coverly » Fri May 23, 2014 5:06 pm

NickFaulks wrote: Of course, it could be systemic if he were tended to be overawed when facing an opponent with a FIDE rating, but only the player can solve that.
Adult players without FIDE ratings may lack experience at playing at the typically slower pace of a 4NCL game. With juniors now being given ECF grades in anticipation of improvement and the lags in Elo rating created by rapidly improving players, the relationship between ECF grades and FIDE ratings is all over the place under around 2100.

NickFaulks
Posts: 8452
Joined: Sat Jan 02, 2010 1:28 pm

Re: GRADING ANOMALIES

Post by NickFaulks » Fri May 23, 2014 5:40 pm

Does that have any practical significance? Board order, perhaps?
If you want a picture of the future, imagine a QR code stamped on a human face — forever.

Roger de Coverly
Posts: 21291
Joined: Tue Apr 15, 2008 2:51 pm

Re: GRADING ANOMALIES

Post by Roger de Coverly » Fri May 23, 2014 5:46 pm

NickFaulks wrote:Does that have any practical significance? Board order, perhaps?

In the 4NCL, it's board orders, but teams are now allowed to choose which rating or converted grade they want to use at an individual player level.


In Swiss pairings on the other hand, I've had pairings done using the FIDE ratings where I was the seed, but on ECF grades, I would be the potential victim. That's not actually bad as it imports a degree of randomness to the otherwise boring predictability of seeded pairings.

Post Reply