Don't Understand Grading Calculation

 Posts: 1254
 Joined: Tue Nov 18, 2014 7:23 pm
Don't Understand Grading Calculation
Last season we had a foreign student join our club with a FIDE rating that equated to about 115. Playing quick games against him we decided that he was probably a bit stronger than that and placed him in our teams based on our estimates. At the end of the season his results were 8 wins against opponents with the following grades  132, 129, 121, 118, 125, 128, 144, 107. That looks to me to be an average grade of 125.5 which I would have thought would have resulted in a grade of 176. Instead his grade is 187. What aspect of the calculation have I failed to understand?
BTW, I did warn him that he would have an inflated grade with all wins but he went away over the summer rather than play congresses where he would have faced stronger opposition. Although we moved him up during the course of the season we were constrained by league grading difference limits and by the end of the season were perhaps exceeding them but not by enough for any other teams to lodge an official complaint.
BTW, I did warn him that he would have an inflated grade with all wins but he went away over the summer rather than play congresses where he would have faced stronger opposition. Although we moved him up during the course of the season we were constrained by league grading difference limits and by the end of the season were perhaps exceeding them but not by enough for any other teams to lodge an official complaint.
Ah, but I was so much older then. I'm younger than that now.
 Christopher Kreuzer
 Posts: 7499
 Joined: Fri Aug 06, 2010 2:34 am
 Location: London
Re: Don't Understand Grading Calculation
One of your player's games was against a junior (the one graded 118), that is what the asterisk (*) indicates. If you click through to that junior's grading page, their actual grade (calculated using the junior grading thingy method) is 124. I think this will account for some of the difference. OK, that is not enough to account for the difference. Maybe your player (as a new player) also has their grade calculated in a particular way (like for juniors).
I think what might be happening is that at least four of the players are "50 points below" the grade that the system is trying to assign to your player, so the performance there is recalculated. Take the initial estimate of 176 and redo the calculation with those four grades redone as "126" grades instead. And see what the result is.
132, 129, 121 [126], 118 [124 then 126], 125 [126], 128, 144, 107 [126].
New average: 180.
Then do this again:
132, 129 [130], 121 [126 then 130], 118 [124 then 126 then 130], 125 [126 then 130], 128 [130], 144, 107 [126 then 130].
New average: 182.
Then do this again:
132 [132], 129 [130 then 132], 121 [126 then 130 then 132], 118 [124 then 126 then 130 then 132], 125 [126 then 130 then 132], 128 [130 then 132], 144, 107 [126 then 130 then 132].
New average: 183.5 (round this to 184)
Basically, only the highest grade of 144 (bolded below) will not get bumped up this way.
The final iteration is:
132 [132 then 134], 129 [130 then 132 then 134], 121 [126 then 130 then 132 then 134], 118 [124 then 126 then 130 then 132 then 134], 125 [126 then 130 then 132 then 134], 128 [130 then 132 then 134], 144, 107 [126 then 130 then 132 then 134].
New average is 185.
Do it one more time (it wasn't quite the final iteration):
132 [132 then 134 then 135], 129 [130 then 132 then 134 then 135], 121 [126 then 130 then 132 then 134 then 135], 118 [124 then 126 then 130 then 132 then 134 then 135], 125 [126 then 130 then 132 then 134 then 135], 128 [130 then 132 then 134 then 135], 144, 107 [126 then 130 then 132 then 134 then 135].
New average is 186.
Maybe do it one more time?
New average is [(136*7 plus 144)/8]+50= 187
There we are!
I think what might be happening is that at least four of the players are "50 points below" the grade that the system is trying to assign to your player, so the performance there is recalculated. Take the initial estimate of 176 and redo the calculation with those four grades redone as "126" grades instead. And see what the result is.
132, 129, 121 [126], 118 [124 then 126], 125 [126], 128, 144, 107 [126].
New average: 180.
Then do this again:
132, 129 [130], 121 [126 then 130], 118 [124 then 126 then 130], 125 [126 then 130], 128 [130], 144, 107 [126 then 130].
New average: 182.
Then do this again:
132 [132], 129 [130 then 132], 121 [126 then 130 then 132], 118 [124 then 126 then 130 then 132], 125 [126 then 130 then 132], 128 [130 then 132], 144, 107 [126 then 130 then 132].
New average: 183.5 (round this to 184)
Basically, only the highest grade of 144 (bolded below) will not get bumped up this way.
The final iteration is:
132 [132 then 134], 129 [130 then 132 then 134], 121 [126 then 130 then 132 then 134], 118 [124 then 126 then 130 then 132 then 134], 125 [126 then 130 then 132 then 134], 128 [130 then 132 then 134], 144, 107 [126 then 130 then 132 then 134].
New average is 185.
Do it one more time (it wasn't quite the final iteration):
132 [132 then 134 then 135], 129 [130 then 132 then 134 then 135], 121 [126 then 130 then 132 then 134 then 135], 118 [124 then 126 then 130 then 132 then 134 then 135], 125 [126 then 130 then 132 then 134 then 135], 128 [130 then 132 then 134 then 135], 144, 107 [126 then 130 then 132 then 134 then 135].
New average is 186.
Maybe do it one more time?
New average is [(136*7 plus 144)/8]+50= 187
There we are!

 Posts: 19018
 Joined: Tue Apr 15, 2008 2:51 pm
Re: Don't Understand Grading Calculation
You always get something for winning, so the minimum grade is yours minus 40.Christopher Kreuzer wrote: ↑Mon Oct 07, 2019 1:00 pm
I think what might be happening is that at least four of the players are "50 points below" the grade that the system is trying to assign to your player, so the performance there is recalculated. Take the initial estimate of 176 and redo the calculation with those four grades redone as "126" grades instead. And see what the result is.
How strong is someone who chews up 120 players on a regular basis? The grading system struggles, but so do all rating systems?
 Christopher Kreuzer
 Posts: 7499
 Joined: Fri Aug 06, 2010 2:34 am
 Location: London
Re: Don't Understand Grading Calculation
From the help page:
"If an opponent (or the player himself) is ungraded, a "starting grade" is estimated, using all available information. See Estimating a starting grade for an ungraded player below."
"If an opponent (or the player himself) is ungraded, a "starting grade" is estimated, using all available information. See Estimating a starting grade for an ungraded player below."
Estimating a starting Grade for an ungraded player (or a junior)
A Rapid grade, where available, will be used in default of a Standard grade; and vice versa. If the player has no grade at all, a starting grade is calculated as follows, using all their games in the latest three years (for adults) or one year (for juniors), inclusive of the current year.
Stage 1 is to calculate a 'grade' for each ungraded player on all their games against graded opponents in the relevant period. The 40point rule is not used. If all their opponents are graded, it stops there and Stage 2 is omitted. The result will be used as their starting grade.
Stage 2 brings in games between the ungraded players. Once again the 40point rule is not used. The players are 'graded' on all their games, using as starting grades the figures obtained from Stage 1. The resulting 'grades' will not be very accurate. So they are fed in again as new starting grades, and Stage 2 is repeated. This continues, with increasing accuracy each time, until the figures (more or less) stop changing. The starting grades can then be considered accurate, and they are used in the grading proper.
The effect of Stage 2 is that the points scored for a game against an ungraded opponent are repeatedly adjusted till the starting grades are right. But it is important to note that the adjustment applies only to games from the most recent (so not previously calculated) grading period. Games from earlier periods are included every time round, but always with the original points value. A previousperiod game, once calculated, is never recalculated except for corrections.
 Christopher Kreuzer
 Posts: 7499
 Joined: Fri Aug 06, 2010 2:34 am
 Location: London
Re: Don't Understand Grading Calculation
Oops. I did my iterations doing the grades minus 50 not minus 40. Any iteration would have eventually converged on 187 I think. Is that what has happened here? The grading calculations guidance seems to indicate that this should NOT have happened here, as only one of his opponents was 'ungraded' or a 'junior'. Strange.Roger de Coverly wrote: ↑Mon Oct 07, 2019 1:16 pmYou always get something for winning, so the minimum grade is yours minus 40.Christopher Kreuzer wrote: ↑Mon Oct 07, 2019 1:00 pm
I think what might be happening is that at least four of the players are "50 points below" the grade that the system is trying to assign to your player, so the performance there is recalculated. Take the initial estimate of 176 and redo the calculation with those four grades redone as "126" grades instead. And see what the result is.
How strong is someone who chews up 120 players on a regular basis? The grading system struggles, but so do all rating systems?
[I think my iteration would have eventually got to 194, i.e. 144+50.]
Last edited by Christopher Kreuzer on Mon Oct 07, 2019 1:25 pm, edited 1 time in total.

 Posts: 19018
 Joined: Tue Apr 15, 2008 2:51 pm
Re: Don't Understand Grading Calculation
Now there's an anomaly. If the player in question hadn't played a junior, the iteration would have stopped at the lower value. That may help to explain why new players and juniors doing extremely well or extremely badly seem to get grades that a player with an established grade and the same performance wouldn't.Christopher Kreuzer (Grading Help) wrote: ↑Mon Oct 07, 2019 1:16 pm
Estimating a starting Grade for an ungraded player (or a junior)
Stage 1 is to calculate a 'grade' for each ungraded player on all their games against graded opponents in the relevant period. The 40point rule is not used. If all their opponents are graded, it stops there and Stage 2 is omitted. The result will be used as their starting grade.

 Posts: 19018
 Joined: Tue Apr 15, 2008 2:51 pm
Re: Don't Understand Grading Calculation
I have this idea that at least part of the iteration uses minus 50 rather than minus 40. It may have to, otherwise you get a convergence to highest player plus 50.Christopher Kreuzer wrote: ↑Mon Oct 07, 2019 1:20 pm
Oops. I did my iterations doing the grades minus 50 not minus 40. Any iteration would have eventually converged on 187 I think. Is that what has happened here? The grading calculations guidance seems to indicate that this should NOT have happened here, as only one of his opponents was 'ungraded' or a 'junior'.
From the grading Help.
Stage 2 brings in games between the ungraded players. Once again the 40point rule is not used.

 Posts: 3823
 Joined: Wed Apr 30, 2008 12:28 pm
Re: Don't Understand Grading Calculation
I think Chris was on the right track. I also think that all grading/rating systems struggle with 100 (or 0!) % scores. It's a "new" player and only 8 games as well. Many players on this forum would probably get 8/8 against that opposition but as we tend to have established grades the system can handle it without iterations of the results.

 Posts: 476
 Joined: Fri Apr 03, 2009 1:30 pm
Re: Don't Understand Grading Calculation
The calculation here is fairly straightforward, but difficult to justify.
The iteration method will get to the average of opponent's grades plus 50; which as already pointed out gives 176. In the final iteration the 40 point limit is introduced so 7 of those results become 186 with the other being 194. The new average is 187 and published as the grade.
The iteration method will get to the average of opponent's grades plus 50; which as already pointed out gives 176. In the final iteration the 40 point limit is introduced so 7 of those results become 186 with the other being 194. The new average is 187 and published as the grade.

 Posts: 1697
 Joined: Thu May 15, 2008 1:37 am
 Contact:
Re: Don't Understand Grading Calculation
It's perhaps worth pointing out that between five and eight results with at least one of those in the most recent 12month grading period gets a player an 'F' category grade. 'F' category grades are not used for future grading calculations (instead a new estimate is calculated) and are treated by some with caution (since the sample size used in the calculation is small and some of it may be dated).
 Chris Goodall
 Posts: 471
 Joined: Sun Oct 10, 2010 6:40 pm
 Contact:
Re: Don't Understand Grading Calculation
They don't have to. It's a German Tank Problem. If you capture a single tank, and its serial number is 60, estimate how many tanks have been manufactured? Since you only have one data point, the best assumption you can make is that half of all the tanks have a lower serial number, and half have a higher. So 120 tanks. You assume that the tank you observed is the median tank, the same assumption as in the Doomsday Argument.Kevin Thurlow wrote: ↑Mon Oct 07, 2019 5:49 pmI also think that all grading/rating systems struggle with 100 (or 0!) % scores.
So if you have a single tank, which is that your new player lost to a player graded 200, what do you do? If you think you can't wring much information out of a sample size of one, consider a sample size of zero. If I meet a chap who tells me he's a competitive chess player, and I know none of his past results, my best estimate is that he's a perfectly average player graded 128. (Let's assume I'm new to the area and don't recognise all the top players already.)
If I then see him lose to a player graded 200, what happens to my estimate? It seems obvious to me that it should go down. I've learnt that he's not one of the small number of players that a 200 can't beat. So now when I assume he's an average player, he is the average of a set that's been chopped off at the top end, therefore its midpoint is slightly lower.
If I were the ECF, my estimate of his grade would jump up to 150! Not only that, but if he then played nine more games against 200 players and lost them all, the ECF would become more confident that my initial grade was an underestimate, because if he wasn't graded within 50 points of 200, why is he playing 200s all the time? We're literally deriving information about the player's strength from the arbitrary decisions of competition organisers.
My Normalised Open Source Online Clarke System is going to estimate new players' grades as if they were German tanks. (In fact, new players with grading codes starting 31 have a median starting grade of 103, unless their "club" is either a country or "4NCL [Club]", in which case it's 189. So that's a further refinement right there.)
Chris Goodall, formerly known as Chris Wardle. Northumbria League handcranker; ECF Grader for Bernicia and the NCCU.
Newcastle is not in Scotland!
Newcastle is not in Scotland!

 Posts: 2656
 Joined: Tue May 24, 2011 10:58 am
Re: Don't Understand Grading Calculation
Hummm, I'm not sure you know.
I think team captains tend to have a decent idea how strong players are, and that determines the strength of the field they play to a fair extent. Also people entering the 'right' part of Open tournaments.
Seems like information to me. Weak, but information. Not got anything else to use, and not making any claims about the actual usability of the grades resulting, so why not?
I think team captains tend to have a decent idea how strong players are, and that determines the strength of the field they play to a fair extent. Also people entering the 'right' part of Open tournaments.
Seems like information to me. Weak, but information. Not got anything else to use, and not making any claims about the actual usability of the grades resulting, so why not?
 Chris Goodall
 Posts: 471
 Joined: Sun Oct 10, 2010 6:40 pm
 Contact:
Re: Don't Understand Grading Calculation
We do have something else to use. Guess 127 (not 128; miscalculated), you'll be righter than if you guess any other number. You can appreciate that guessing either 3.1415 or 900 is going to be very wrong, so somewhere in between those extremes must be an optimum value, right? Or if you know the player is new to the system and doesn't have a FIDE code from another country, revise your guess to 103. That's instantly more information than you can get out of one defeat to a 200, because almost everyone loses to 200s.MartinCarpenter wrote: ↑Sun Oct 20, 2019 9:30 pmHummm, I'm not sure you know.
I think team captains tend to have a decent idea how strong players are, and that determines the strength of the field they play to a fair extent. Also people entering the 'right' part of Open tournaments.
Seems like information to me. Weak, but information. Not got anything else to use, and not making any claims about the actual usability of the grades resulting, so why not?
Look at it this way: if you find the set of people who'd probably score 0/1 against a 200, and assume that your player is bang in the middle of it, their grade drops from 127 to 123. 123 puts them at the 50th percentile within the set. If you instead assume that your player is graded 150, then they are at the 73rd percentile within the set. Of all the people who would probably lose to this 200, this particular one is better than 73% of the others, based on nothing other than the fact that they lost the game
Chris Goodall, formerly known as Chris Wardle. Northumbria League handcranker; ECF Grader for Bernicia and the NCCU.
Newcastle is not in Scotland!
Newcastle is not in Scotland!

 Posts: 96
 Joined: Thu Jun 06, 2019 4:18 pm
Re: Don't Understand Grading Calculation
I don't agree; I think the flaw in the logic is that it does not regard as significant the fact that this unknown X has played a game against a player graded 200 (which is *already* information about their strength, even prior to the result).
i.e. the average grade among players who have played a graded game against someone graded 200 is significantly higher than the overall average of the chess population.
To take an even more extreme example: suppose one had to estimate the strength of player Y, given only that Y is not Norwegian and has played a classical game against Carlsen and lost. Although almost everyone would lose to Carlsen, the reasonable estimated strength of player Y should still be >2300.
i.e. the average grade among players who have played a graded game against someone graded 200 is significantly higher than the overall average of the chess population.
To take an even more extreme example: suppose one had to estimate the strength of player Y, given only that Y is not Norwegian and has played a classical game against Carlsen and lost. Although almost everyone would lose to Carlsen, the reasonable estimated strength of player Y should still be >2300.
 Chris Goodall
 Posts: 471
 Joined: Sun Oct 10, 2010 6:40 pm
 Contact:
Re: Don't Understand Grading Calculation
Sure, but the median grade of people who play against 200s is about 188, and the median grade of the players beaten by 200s is about 176, so why 150 and not 188 or 176?Joseph Conlon wrote: ↑Mon Oct 21, 2019 9:14 amI don't agree; I think the flaw in the logic is that it does not regard as significant the fact that this unknown X has played a game against a player graded 200 (which is *already* information about their strength, even prior to the result).
i.e. the average grade among players who have played a graded game against someone graded 200 is significantly higher than the overall average of the chess population.
Let's apply your logic from the other direction. A player walks into your club and claims to be from a country you were sure was still part of Russia. You put him on board 5 in Division 3, and he easily beats a couple of children graded 38 and 49. Do you therefore let him enter the Under 100 section of your congress, because the fact that he played against those children makes him a weaker player than if he hadn't?
Chris Goodall, formerly known as Chris Wardle. Northumbria League handcranker; ECF Grader for Bernicia and the NCCU.
Newcastle is not in Scotland!
Newcastle is not in Scotland!