GRADING ANOMALIES

[Roger de Coverly]

somebody could perform similar statistical analysis which has led to grade correction (resulting in recent introduction of new grades)

You are aware of what they did? It was a tabulation of result by grading difference. So they wrote down the score (in terms of the higher graded player) for all of the grading differences. So they got this result that players with a 10 point difference scored 57/100 instead of 60/100. Then they decided that this was wrong so they compressed the 10 point difference down to about 8 points to make it fit the theoretical linear curve better. In my mind there were a number of other possible explanations for the expansion effect which didn't require the destruction of the historic

It's possible to replicate their work (on a smaller scale) using tournament cross tables. It's best to group the grading bands a bit because of data shortage.

The point is that I think that it is more likely that GS assigns to ex-150 players who have improved to 162.5 level grades of 175, rather than AGS3 would assign to ex-150 players who have improved to 175 level grades of 162.5. Of course, one cannot prove which of the two statements is true without a priori assuming correctness of one of the systems

I don't accept this. If a player is playing to a 175 standard, his opponents will know this from the quality of play - basically 150 players will score 1 from 4 against 175 standard opposition and 175 standard will score 2 from 4. You can tell playing standard by little things like knowing openings that bit better, making critical decisions faster etc.

[Robert Jurjevic]

Hello Roger,

The point is that I think that it is more likely that GS assigns to ex-150 players who have improved to 162.5 level grades of 175, rather than AGS3 would assign to ex-150 players who have improved to 175 level grades of 162.5. Of course, one cannot prove which of the two statements is true without a priori assuming correctness of one of the systems

I don't accept this. If a player is playing to a 175 standard, his opponents will know this from the quality of play - basically 150 players will score 1 from 4 against 175 standard opposition and 175 standard will score 2 from 4. You can tell playing standard by little things like knowing openings that bit better, making critical decisions faster etc.

Let us assume that a 150 pool played a 175 pool scoring 50%. If one would wish to calculate new pool grades based on the games the pool players played then IMHO it would be plausible to assume that new pool grades should be equal (as each of the pools scored 50%).

IMHO there are three principal valid cases here: 1) new grade of both pools is 175, 2) new grades of both pools is 162.5 and 3) new grade of both pools is 150. (please note that cases in which new grades of both pools would be say 174, 173, 172,..., 162, 161,..., 153, 152, 151 are also valid)

Roger, I really do not understand how can you accept the case in which a grade of 150 is assigned to 175 pool and a grade of 175 to 150 pool, as by doing this you are obviously 'over-rewarding' the players from 150 pool and 'over-penalizing' the players from 175 pool, if the players from both pools were un-graded I hope that you would assign equal grade to both pools and that you wouldn't reward 25 extra points to one of the pools, though if you would I have no idea why and to which of the pools.

The same should apply if a 150 player played a 175 pool, a 150 pool played a 175 player or if a 150 player played a 175 player.

Kind regards,

[Roger de Coverly]

Let us assume that a 150 pool played a 175 pool scoring 50%.

I don't want to assume this. I want to consider a single 150 player playing in a 175 pool and scoring 50%. The real world example of this is a player stepping up from Majors to Opens. The 175's in the Open haven't got any worse but the ex-Major player is now equal to the players in the Open. I think he should get a 175 grade for his 50% performance. If you check out the movements in grade from one year to the next, you notice that a lot of players don't change much - thus implying that they scored in accordance with their pre-season grade. A minority of players will show changes and I see no particular reason to dilute their change provided they have played enough games at a higher level.

[Robert Jurjevic]

Hello Roger,

I do apologize for discussing this subject possibly too much and that I might be bothering you.

Let us assume that a 150 pool played a 175 pool scoring 50%.

I don't want to assume this. I want to consider a single 150 player playing in a 175 pool and scoring 50%. The real world example of this is a player stepping up from Majors to Opens. The 175's in the Open haven't got any worse but the ex-Major player is now equal to the players in the Open. I think he should get a 175 grade for his 50% performance. If you check out the movements in grade from one year to the next, you notice that a lot of players don't change much - thus implying that they scored in accordance with their pre-season grade. A minority of players will show changes and I see no particular reason to dilute their change provided they have played enough games at a higher level.

I think there should be no difference if a 150 player or 150 player pool plays a 175 player pool and that in both cases the same grading rule should be applied to allocate grading points in respect of each game for all players.

The 175 pool of players consists of individual players (with average grade of 175) and the method of calculating the grading points allocated for 150 player in respect of each game should not depend on the fact that each player the 150 player paled belongs to a pool of players named "players who played the 150 player". The rule for allocating grading points in respect of each game should depend on the two players in question in each individual game and the game result, and the fact that each of the players in each of the games can belong to a pool of players (a pool of players my opponent has played, a pool of players younger than 18, a pool of women with blond hair, a pool of male fairly handsome players, etc.) should have no bearing on the choice of the method for allocating grading points in respect of each game.

Let us assume that Robert 150 draws in a game against Roger 175 and that one faces a problem of allocating grading points in respect to that game to both Robert and Roger. Let us also assume that Robert played a pool of players with average grade of 175 (Roger is a member of that pool) and that Roger played a pool of players with average grade of 170 (Robert is a member of that pool). GS rule says "Robert for a draw against Roger scores 175 grading points" and to me this looks as if it is assumed that Robert played at the same level as Roger which is estimated to be 175, fair enough you can assume such a thing though I can't see on basis of what, possibly because it happened that the average grade of all Robert's opponents was 175, well what that has to do with this particular game. GS rule also says "Roger for a draw against Robert scores 150 grading points" and to me this looks as if it is assumed that Roger played at the same level as Robert which is estimated to be 150, fair enough you can assume such a thing though I can't see on basis of what, now you even can't use the fact that the average grade of all Roger's opponents was 170, anyway what would that have to do with this particular game. Wouldn't it be more plausible to assume that both Robert and Roger should be assigned equal grade and (because in principle one has no additional information to estimate at which level both of the players have played) to assume that they have played at the average level of 162.5 (penalizing Roger and rewarding Robert for the same number of grading points) and allocate to both players a grade of 162.5. It also looks to me that the game result would more suggest that one should allocate equal grade to both of the players (say 162.5) rather than different grades (say 150 to Roger and 175 to Robert). You now continue like this until you haven't allocated grading points for each individual game Robert played, then Robert's new grade is the average of the allocated grades for each game, and that is equivalent to calculating Robert's new grade taking into account all games he played against a pool of players which happens to have average grade of 175. You see that the pool as such (and its average grade) does not come into play, you always allocate grading points in respect to each individual game and the two players involved in it, you have no grading rule (or formulae) to allocate grading points to the pool of players at once not taking into account individual players and their games.

Kind regards,

[Roger de Coverly]

Let us assume that Robert 150 draws in a game against Roger 175 and that one faces a problem of allocating grading points in respect to that game to both Robert and Roger.

It's perfectly well defined - you assign Robert a score of 175 and Roger a score of 150. As part of a summation over 30 or more games, there's nothing wrong with this. Unlike an online server, you aren't intending to publish a grade immediately after the game.

Bottom line is that if a player meets a field of players with an average grade of 175 and scores 50% over at least 30 games, then the underlying premise of the ECF grading system is that he now should now rank equally with all other players of a 175 grade. Even he had a grade of 150 in the previous season, he is not and should not be eligible for under 170 competitions. If you are writing down a board order he should be below a player who was 180 this season and last and above a player who was 170 this season and last.

Again I think I am endlessly repeating the same point, which is that the practical effect of 25 points for a win is to reduce the pace at which the grading system reacts to changes in a player's standard. Let's try yet another example. We have a player who plays 30 games each season always scoring 50% but he moves up a class every year until he peaks at 200. So if he starts at 100, then his first grade is 100, second 125, third 150, fourth 175 and final 200. Under your system I think his grades are 100 , (100+125)/2 = 112.5, (112.5+150)/2 = 131.25, (131.25+175)/2 = 153.25, (153.25+200)/2 = 176.75.
Thus he's 23 points undergraded at the end of the sequence. In fact if he now stabilises at a 200 standard, I don't think he can ever actually reach 200 (199.99 perhaps). Next season is (176.25+200)/2 = 188.1, then (188.1+200)/2 = 194 etc. For what it's worth assume he never played the same person twice, so the effect on his opponents is minimal.

[Robert Jurjevic]

Hello Roger,

Sorry for continuing this discussion, but I have to say this...

Let us assume that Robert 150 draws in a game against Roger 175 and that one faces a problem of allocating grading points in respect to that game to both Robert and Roger.

It's perfectly well defined - you assign Robert a score of 175 and Roger a score of 150.

If Robert 150 drew against Roger 175, allocated grading points for that game for Robert and Roger would be as follows:

GS: 175.00 to Robert and 150.00 to Roger
AGS3: 162.50 to Robert and 162.50 to Roger
ÉGS5: 162.99 to Robert and 162.01 to Roger
ÉGS6: 162.99 to Robert and 162.01 to Roger (assuming that Robert and Roger played equal number of games in the previous season)

Statement 1: The point here is that I am arguing that allocating 175 to Robert and 150 to Roger for the game in which Robert drew against Roger is a big mistake, and that allocating 162.50 to Robert and 162.50 to Roger, or 162.99 to Robert and 162.01 to Roger, would be acceptable (even allocating 175 to Robert and 175 to Roger, or 150 to Robert and 150 to Roger, would be acceptable if there is a sound basis to assume so).

I really do not understand why you cannot accept the fact that at the end of the day the rules (GS 1a, AGS3 2b, or ÉGS5, ÉGS6 formulae) apply to grading of individual games (we want to allocate grading points to Roger and Robert based on the game in which Robert 150 drew against Roger 175)?

!
The fact that Roger and Robert may be members of various player pools should have no bearing in a discussion which tries to establish why one game grading rule is better or worse than the other (say why AGS3 rule 2b is better than GS rule 1a).

Say, in what way an average grade of Robert's opponents he played in a season may influence a decision on how much grading points one should allocate to Robert for his draw against Roger? The average grade of Robert's opponents could have been virtually any number, not necessarily 175, it could have been 120, 150, 200, etc. Say, saying that one has to allocate to Robert 175 grading points for his draw against Roger because the average grade of Robert's opponents was 131 is to me equally unsound as saying that one has to allocate to Robert 175 grading points for his draw against Roger because in this season Robert played a female player called Mary (Mary was a member of pool of players which were Robert's opponents).

So, if you would like to argue about my statement 1 above, may I kindly ask you please if you could not mention player pools, as they are irrelevant.

Thanks.


BTW it looks like you are persistent in penalizing yourself for that draw against me much more than what I would do (just joking, of course).

Kind regards,

[Roger de Coverly]

why one game grading rule

If you really want to talk about rules for grading just one game then IMHO Elo style systems are much better. So the grade after one game played is old grade plus factor * (excess over score implied by grade). So in the 150 to 175 example (treating it as 1800 to 2000 to get the dimensions right), the 25 point better player is expected to score 75% ( actually 74% in the marginally non-linear Elo model). Thus if you set factor to 15 then you get 1800 + 15 * 0.25 = 1803.75 ( 150.47) and 2000- 15 * 0.25 = 1996.25 (174.53) . So the "penalty" for drawing is 0.47. The ECF system is higher than this - for both players playing 30 games, the practical effect on the end season grade of a drawing against a player 25 points lower ( as opposed to one of the same grade) is 25/30 = 0.83 - an effect equivalent to a K of about 25.


The Elo calculation result or its equivalent is used for real time processing on the chess servers. In real time processing, then the players start at 150.47 v 174.53 if they play another game. Over the board has of practical necessity to use a "published" rating list, so a second game would also start them off at 150 and 175.

I thought I was trying to talk about rules for ranking players after a season of results and 25 points for a win introduces lag as demonstrated in the 100,125,150,175,200 example. As far as I understood it you are advocating that your 25 points for a win system is better for this purpose than the ECF system. I am saying that it isn't mostly because of the added lag.

The ECF system states that a player should expect to score 75% against opposition 25 points lower. The other side of the coin should be that a player scoring 75% should get a grade 25 points above his average opposition. In both cases for a sufficiently high number of games.

[Roger de Coverly]

(we want to allocate grading points to Roger and Robert based on the game in which Robert 150 drew against Roger 175)?

As another way of looking at this, remember this is just one game out of (assumed) 30 and look at the effect this one game makes to the end season computation ( I think it's called perturbation theory). So just by turning up and playing the lower rated player gains 25/30 ( 0.83). A win gains 50/30 (1.67) and a loss -1.67. So combining the results loss -0.83, draw 0.83, win 2.5. Your 25 point rule just halves this.


If either player plays more than 30 games, then the effect is diluted. If either player has less than 30 games over the measurement period ( given that the system looks backwards for games if there aren't enough in the previous season) then the effect is magnified.

[Paul McKeown]

Robert,

I really don't know what your disagreement is with Roger - it all seems very clear. Elo system is continuous, BCF system is periodic. Damping the BCF system merely introduces lag due to its periodic updates. Elo system deals with incremental changes, so a damping factor is necessary to avoid overshoot.

Regards,
Paul.

[Robert Jurjevic]

If you really want to talk about rules for grading just one game then IMHO Elo style systems are much better.

Not going into details of how FIDE calculates its ratings I can say that logistic dependence between expected performance and grade difference is assumed in ÉGS5 and ÉGS6 rules for allocating grading points in respect of each game, which becomes evident for rating difference larger than approximately 30 grading points. Say if Robert Jurjvic 121 draws against Michael Adams 267, allocated grading points for that game for Robert and Michael would be as follows:

GS: 161.00 to Robert and 227.00 to Michael
AGS3: 141.00 to Robert and 247.00 to Michael
ÉGS5: 145.94 to Robert and 242.06 to Michael
ÉGS6: 145.94 to Robert and 242.06 to Michael (assuming that Robert and Michael played equal number of games in the previous season)

II thought I was trying to talk about rules for ranking players after a season of results and 25 points for a win introduces lag as demonstrated in the 100,125,150,175,200 example. As far as I understood it you are advocating that your 25 points for a win system is better for this purpose than the ECF system. I am saying that it isn't mostly because of the added lag.

To calculate one's new ECF grade one would allocate grading points in respect of each game the player played in the season applying one of the rules for grading individual games (either GS, AGS3, ÉGS5, ÉGS6, etc.) and than take the average of all allocated grades to be the player's new ECF grade (if the player played less than 30 games in the season one would weight the average of all allocated grades with the player's grades from previous seasons to assure that enough games is taken into account).

Therefore, it is evident that the new ECF grade (which is in principle an average of allocated grades for every individual game) directly depends on the chosen rule for allocating grades for every individual game.

It looks to me like grade lag is more related to a minimum number of games required to calculate one's grade rather than to a rule chosen for allocating grading points in respect of each game. Say, requiring minimum of 60 games should increase grade lag in comparison to requiring 30 games or 1 game only (but it looks like 1 game only would not be nearly enough to get statistically significant data about one's performance), One way to approach this problem could be to grade after relatively small number of games (it can still be 30 but can be less) and apply ÉGS6 rule for allocating grading points in respect of each game, ÉGS6 would change the grades of players who played less games more rapidly, say in the example where Robert 150 drew against Roger 175, if Robert played 10 and Roger 60 games in the previous season, ÉGS6 would assign 172.26 to Robert and 171.29 to Roger, as Robert's grade would be less trusted than Roger's in that game, and it would be assumed that the game result is more caused by Robert playing above his level than Roger playing below his level.

The fact that the grades calculated using AGS3 rule change slower than the grades calculated using GS rule is directly a consequence of the differences in the rules for allocating grading points in respect of each game and IMHO a fault is in GS rule which is over-rewarding better performer and over-penalizing worse performer.

The ECF system states that a player should expect to score 75% against opposition 25 points lower. The other side of the coin should be that a player scoring 75% should get a grade 25 points above his average opposition. In both cases for a sufficiently high number of games.

I think that the player scoring 75% should get a grade of 12.5 above his average opposition.

That is because you cannot assume that all opposition players played on the level suggested by their grades, so you reward the player with 12.5 grading points on average in each game he played against the opposition and penalize the opposition players by 12.5 grading points on average in each game they played against the player, rather than leaving the opposition players' grades unchanged and reward the player with 25 grading points on average in each game he played against the opposition.

IMHO GS would make a mistake here by rewarding the player with 25 grading points on average in each game he played against the opposition and yet penalizing the opposition players by 25 grading points on average in each game they played against the player.

Please note that if a player scoring 75% will get grade of 25 or 12.5 above his average opposition depends on the rule which was used to allocate grading points in respect of each game.

Kind regards,

[Paul McKeown]

Robert,

It seems to me that you are now arguing as from an inward looking religious sect speaking with deaf ears to the unconverted. I don't know where Roger gets his reserves of patience, but I admire his tenacity.

P.

[Paul McKeown]

Robert,

Elo system updates game per game. Damping is needed.

BCF system takes an annual average. Damping would damage its accuracy.

P.

[Roger de Coverly]

That is because you cannot assume that all opposition players played on the level suggested by their grades

Why not? Surely the whole concept of being able to construct any sort of grading or rating system depends on being able to make an assumption about the utility and reliability of the brought forward grade. The ECF assumption was that it was correct and could be used to measure performance in the current season.

An example of sorts. 31 players of 175 standard play each other once in a season. In a totally boring manner they all score 50%. On all grading systems, their grade at the end of the season is 175. One of them drops out and is replaced by a 150 player who has improved between seasons and is now just as good as they are. Again they play and all score 50%. They won't all get the same grades though. The ex-150 player will get 175 on the ECF measuring system but all his opponents will get 174.17 ( 29* 175 + 150) /30. This is the deflation effect caused by improving players. You counter this on a broad basis by feeding extra points to players expected to improve such as juniors.

On your +25 system you would give the ex-150 player a new grade of 162.5 and each of the 30 opponents 174.58.

As Paul M observes, the practical effect is to increase the dampening effect. That's present in the ECF system as well but only for smaller game counts. So if we repeat the example with 16 players instead, then our ex-150 player is 162.5 because we add in 15 games from his previous 150 season. Our 175 players are (14*175 + 150 + 15*175 ) / 30 which is 174.17 again - so they benefit from being 175 the previous season.

Please note that I am building a scenario and asking how a grading system will cope with it. So over 15 games the ECF system gives our improved player a 162.5 grade which your +25 system does over 30. Over 30 games the ECF system totally ignores the previous grade (for the player himself) whereas the +25 system takes longer to converge (and only gets to 174.9999 because of the perpetual averaging.

[E Michael White]

The ex-150 player will get 175 on the ECF measuring system but all his opponents will get 174.17 ( 29* 175 + 150) /30. This is the deflation effect caused by improving players.

Oh no it isnt on your definition of deflation ! The mean stays the same if all players play the same number of games. To get deflation you need the improving player to play more games than at least some of his opponents.

[Roger de Coverly]

Oh no it isnt on your definition of deflation ! The mean stays the same if all players play the same number of games. To get deflation you need the improving player to play more games than at least some of his opponents.

Well I've always thought you needed a battery of tests.

In this particular case I was thinking of the subset of players in the 31 player competition which is 31 * 175 at the end of the previous season and 174.17 * 30 + 175 at the end of the following season. A real improvement to the grading system would be to find a way of adding back the 0.83 (when justified) in a less ad hoc manner than by bonus points for playing juniors or ignoring juniors' previous grades.


I think Clarke and the BCF first noticed that treatment of improving players was a design issue back in the 1960s - the number of elite players in the national list seemed to go down nearly every year without it being apparent that standards were falling or there were fewer players overall. This was the parallel behind my deflation comment.