Don't Understand Grading Calculation

General discussions about grading.
Kevin Thurlow
Posts: 3173
Joined: Wed Apr 30, 2008 12:28 pm

Re: Don't Understand Grading Calculation

Post by Kevin Thurlow » Mon Oct 21, 2019 9:37 am

I would ask the 200 how well his opponent played and provide an informal estimate of grade.

User avatar
Chris Goodall
Posts: 392
Joined: Sun Oct 10, 2010 6:40 pm
Contact:

Re: Don't Understand Grading Calculation

Post by Chris Goodall » Mon Oct 21, 2019 10:02 am

Kevin Thurlow wrote:
Mon Oct 21, 2019 9:37 am
I would ask the 200 how well his opponent played and provide an informal estimate of grade.
Then you have to know whether the 200 is the kind who thinks every game they win a Herculean effort worthy of a candidates' tournament, or the kind who assigns everyone below 195 to the same "idiot" category.
Chris Goodall, formerly known as Chris Wardle. Northumbria League hand-cranker; ECF Grader for Bernicia and the NCCU.
Newcastle is not in Scotland!

Kevin Thurlow
Posts: 3173
Joined: Wed Apr 30, 2008 12:28 pm

Re: Don't Understand Grading Calculation

Post by Kevin Thurlow » Mon Oct 21, 2019 1:36 pm

"Then you have to know whether the 200 is the kind who thinks every game they win a Herculean effort worthy of a candidates' tournament, or the kind who assigns everyone below 195 to the same "idiot" category."

Luckily, most are more sensible than that, and I would know what they were like before asking. After all, it's only one bit of info, but better than zero info.

MartinCarpenter
Posts: 2456
Joined: Tue May 24, 2011 10:58 am

Re: Don't Understand Grading Calculation

Post by MartinCarpenter » Mon Oct 21, 2019 5:48 pm

Chris Goodall wrote:
Mon Oct 21, 2019 9:23 am
Let's apply your logic from the other direction. A player walks into your club and claims to be from a country you were sure was still part of Russia. You put him on board 5 in Division 3, and he easily beats a couple of children graded 38 and 49. Do you therefore let him enter the Under 100 section of your congress, because the fact that he played against those children makes him a weaker player than if he hadn't?
Well what happens in league chess is that they play a few stronger people, you get a decent estimate and they're stuck on a realistic (if often slightly conservative) board. The strength of their teammates on either side is arguably more useful data than their opposition but using that would be slightly odd I guess.

User avatar
Chris Goodall
Posts: 392
Joined: Sun Oct 10, 2010 6:40 pm
Contact:

Re: Don't Understand Grading Calculation

Post by Chris Goodall » Tue Oct 22, 2019 11:29 am

MartinCarpenter wrote:
Mon Oct 21, 2019 5:48 pm
Well what happens in league chess is that they play a few stronger people, you get a decent estimate and they're stuck on a realistic (if often slightly conservative) board. The strength of their teammates on either side is arguably more useful data than their opposition but using that would be slightly odd I guess.
Why is that odder than using their opponent's strength?

I don't doubt that the opinions of a few players of known strength are better than an ECF grade that's based on a 100% or 0% score. Possibly better than an ECF grade full stop.

I do doubt that, if you know on what board they happened to play and therefore which players were sitting beside and opposite them, that data is such a reliable proxy for the opinions of a few players of known strength that it justifies revising your estimate of a player's strength upward from the 50th percentile to the 73rd after they've lost a game.
Chris Goodall, formerly known as Chris Wardle. Northumbria League hand-cranker; ECF Grader for Bernicia and the NCCU.
Newcastle is not in Scotland!

Brian Valentine
Posts: 450
Joined: Fri Apr 03, 2009 1:30 pm

Re: Don't Understand Grading Calculation

Post by Brian Valentine » Tue Oct 22, 2019 2:55 pm

Isn't Martin suggesting that the player moves up tp say 90th percentile when they start the game with the locational extra information and they drop to 73 when the addional information of the result becomes available?

Incidently, the proposed monthly grading calculation will assume that a player with no other information will be assumed to have drawn with a "German Tank" as their first result.

MartinCarpenter
Posts: 2456
Joined: Tue May 24, 2011 10:58 am

Re: Don't Understand Grading Calculation

Post by MartinCarpenter » Tue Oct 22, 2019 10:26 pm

Something like that, yes :)

Obviously this is fundamentally a daft argument given we all agree that you're essentially guessing, but a priori I do think that opposition strength should be marginally useful information. I suppose possible to check in theory.

E Michael White
Posts: 1345
Joined: Fri Jun 01, 2007 6:31 pm

Re: Don't Understand Grading Calculation

Post by E Michael White » Wed Oct 23, 2019 1:16 pm

Chris Goodall wrote:
Mon Oct 21, 2019 9:23 am
A player walks into your club and claims to be from a country you were sure was still part of Russia. You put him on board 5 in Division 3, and he easily beats a couple of children graded 38 and 49. Do you therefore let him enter the Under 100 section of your congress, because the fact that he played against those children makes him a weaker player than if he hadn't?
If those were his only results and the ECF included estimates on 2 games at the next grading date he would be (98+109)*.5 = 104, provided the junior grades stayed at 38 and 49, so you probably wouldn't let him play in an U100 event.
in connection with another point Chris Goodall wrote:
Sun Oct 20, 2019 8:35 pm
If I were the ECF, my estimate of his grade would jump up to 150!
140 actually, at the next grading date.

Oddly at the other extreme things seem to work a little better. If an ungraded player plays 15 players graded 140 and scores 100% he would be graded as 200. Doesn't seem that bad as an estimate.

User avatar
Chris Goodall
Posts: 392
Joined: Sun Oct 10, 2010 6:40 pm
Contact:

Re: Don't Understand Grading Calculation

Post by Chris Goodall » Wed Oct 23, 2019 2:47 pm

Brian Valentine wrote:
Tue Oct 22, 2019 2:55 pm
Isn't Martin suggesting that the player moves up to say 90th percentile when they start the game with the locational extra information and they drop to 73 when the additional information of the result becomes available?
So the fact that they found themself sitting opposite a certain person is 40 percentage points' worth of information, and the fact that they lost is only 17?
Chris Goodall, formerly known as Chris Wardle. Northumbria League hand-cranker; ECF Grader for Bernicia and the NCCU.
Newcastle is not in Scotland!

Angus French
Posts: 1627
Joined: Thu May 15, 2008 1:37 am
Contact:

Re: Don't Understand Grading Calculation

Post by Angus French » Wed Oct 23, 2019 3:39 pm

Why not ask leagues and congresses, through their results officers, to provide estimates which could then be used as seed values for the central grading team's formula? To encourage accuracy and consistency the results officers might be asked how an estimate was derived: from an outdated grade, from a FIDE rating, from a (non-ECF) national federation rating, from an online rating, from the observations of experienced players, from a combination of the above... The central grading team could even provide a registration facility for entering this information and it could even derive an estimate from the information provided.
Last edited by Angus French on Wed Oct 23, 2019 3:44 pm, edited 1 time in total.

User avatar
Chris Goodall
Posts: 392
Joined: Sun Oct 10, 2010 6:40 pm
Contact:

Re: Don't Understand Grading Calculation

Post by Chris Goodall » Wed Oct 23, 2019 3:41 pm

E Michael White wrote:
Wed Oct 23, 2019 1:16 pm
Chris Goodall wrote:
Mon Oct 21, 2019 9:23 am
A player walks into your club and claims to be from a country you were sure was still part of Russia. You put him on board 5 in Division 3, and he easily beats a couple of children graded 38 and 49. Do you therefore let him enter the Under 100 section of your congress, because the fact that he played against those children makes him a weaker player than if he hadn't?
If those were his only results and the ECF included estimates on 2 games at the next grading date he would be (98+109)*.5 = 104, provided the junior grades stayed at 38 and 49, so you probably wouldn't let him play in an U100 event.
in connection with another point Chris Goodall wrote:
Sun Oct 20, 2019 8:35 pm
If I were the ECF, my estimate of his grade would jump up to 150!
140 actually, at the next grading date.

Oddly at the other extreme things seem to work a little better. If an ungraded player plays 15 players graded 140 and scores 100% he would be graded as 200. Doesn't seem that bad as an estimate.
Clarke has always been OpptAverage - 50 + %pt score, has that 50 changed to 40 without me noticing?

I agree that 200 is not an overestimate for a player that scores 15/15 vs. 140s. I'd say it's quite an underestimate. Using a logistic curve, 14½/15 would be your expected score against opponents graded 584 Elo points below you, which is 73 Clarke points, which would put you at 213. 14¾/15, which is the lowest score that would round to 15/15 instead of 14½/15, would put you at 229. The German Tank of the 51 players in England graded at least 229 is number 26, Mark Hebden (238). So my educated guess is that your player is Mark Hebden in disguise 8)
Chris Goodall, formerly known as Chris Wardle. Northumbria League hand-cranker; ECF Grader for Bernicia and the NCCU.
Newcastle is not in Scotland!

Ian Thompson
Posts: 2200
Joined: Wed Jul 02, 2008 4:31 pm
Location: Awbridge, Hampshire

Re: Don't Understand Grading Calculation

Post by Ian Thompson » Wed Oct 23, 2019 4:07 pm

Chris Goodall wrote:
Wed Oct 23, 2019 3:41 pm
E Michael White wrote:
Wed Oct 23, 2019 1:16 pm
[
in connection with another point Chris Goodall wrote:
Sun Oct 20, 2019 8:35 pm
If I were the ECF, my estimate of his grade would jump up to 150!
140 actually, at the next grading date.
Clarke has always been OpptAverage - 50 + %pt score, has that 50 changed to 40 without me noticing?
Both 140 and 150 are right in their own way.

If a player without any other results loses to a player graded 200 then the ECF estimates his grade at 150 (opponent's grade - 50) for the grading calculation. The game is graded as a 150 player losing to a 200 player, so the player scores 140 points for the game.

Mike Gunn
Posts: 694
Joined: Wed Apr 11, 2007 4:45 pm

Re: Don't Understand Grading Calculation

Post by Mike Gunn » Wed Oct 23, 2019 4:12 pm

Angus French wrote:
Wed Oct 23, 2019 3:39 pm
Why not ask leagues and congresses, through their results officers, to provide estimates which could then be used as seed values ...
Isn't this what used to happen before the central computer program took over the job of working out what a new player's grade should be? (Except there was then no iteration because there was no program.) Local graders' estimates were usually accurate to within 10 points or so, I recall.

Roger de Coverly
Posts: 18321
Joined: Tue Apr 15, 2008 2:51 pm

Re: Don't Understand Grading Calculation

Post by Roger de Coverly » Wed Oct 23, 2019 4:55 pm

Mike Gunn wrote:
Wed Oct 23, 2019 4:12 pm

Isn't this what used to happen before the central computer program took over the job of working out what a new player's grade should be? (Except there was then no iteration because there was no program.) Local graders' estimates were usually accurate to within 10 points or so, I recall.

They could "cheat" of course, by waiting until they had a decent set of results before making an estimate. The other convention was to assume players had a grade of 100 in the absence of any other information.

It was only after the advent of automatic estimates that the problem of negative grades started to appear.

User avatar
Chris Goodall
Posts: 392
Joined: Sun Oct 10, 2010 6:40 pm
Contact:

Re: Don't Understand Grading Calculation

Post by Chris Goodall » Thu Oct 24, 2019 12:13 pm

Roger de Coverly wrote:
Wed Oct 23, 2019 4:55 pm
Mike Gunn wrote:
Wed Oct 23, 2019 4:12 pm

Isn't this what used to happen before the central computer program took over the job of working out what a new player's grade should be? (Except there was then no iteration because there was no program.) Local graders' estimates were usually accurate to within 10 points or so, I recall.

They could "cheat" of course, by waiting until they had a decent set of results before making an estimate. The other convention was to assume players had a grade of 100 in the absence of any other information.

It was only after the advent of automatic estimates that the problem of negative grades started to appear.
Local graders' estimates are subject to confirmation bias - they'll remember the ones they got right. I'd be surprised if they're not biased towards boring percentage chess too. A player loses to me with the Latvian Gambit, I'll estimate his grade at 120. Then he beats me with the Blackmar-Diemer, and I'll revise my estimate down to 90 :oops:

It would be good to get data on how much auto-estimates change in the following grading period, controlling for activity.

Is a grade of 0 supposed to represent the weakest person capable of playing a game under FIDE Laws, or a random move machine?
Chris Goodall, formerly known as Chris Wardle. Northumbria League hand-cranker; ECF Grader for Bernicia and the NCCU.
Newcastle is not in Scotland!

Post Reply