The 40-point rule

General discussions about ratings.
Andrew Farthing
Posts: 614
Joined: Sun Dec 28, 2008 11:39 pm

The 40-point rule

Post by Andrew Farthing » Fri Aug 21, 2009 8:25 pm

This may be a stupid question, but it'll be a break from some of the mind-boggling stuff that goes on elsewhere under the Grading Debate heading:

I can understand why the "maximum 40-point difference" rule applies to wins by the higher-rated player, i.e. there has to be a means to stop players losing grading points by winning a game and gaining points by losing. What I don't completely understand is why the same rule applies to draws and losses by the higher-rated player. In these cases, the grading outcome for the game seems legitimate to me.

Is there a specific rationale for this, or is it just that it's a necessary compromise to keep things (relatively) simple?

Andrew Farthing

Sean Hewitt

Re: The 40-point rule

Post by Sean Hewitt » Fri Aug 21, 2009 8:37 pm

I don't understand it either!

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: The 40-point rule

Post by Roger de Coverly » Fri Aug 21, 2009 8:43 pm

Andrew Farthing wrote:Is there a specific rationale for this, or is it just that it's a necessary compromise to keep things (relatively) simple?
It's parallel to the Elo 350 or 400 point rule and it's an incentive for the higher rated player to play.

At 145 would you want to play a league or tournament game against a player graded 60 with the available points 155 for a win, 60 for a draw and 10 for a loss?

I do hope this isn't the latest mad idea by the grading team.

Andrew Farthing
Posts: 614
Joined: Sun Dec 28, 2008 11:39 pm

Re: The 40-point rule

Post by Andrew Farthing » Fri Aug 21, 2009 9:00 pm

Roger de Coverly wrote:I do hope this isn't the latest mad idea by the grading team.
No - it was just something that I was curious about.

Andrew

Andrew Farthing
Posts: 614
Joined: Sun Dec 28, 2008 11:39 pm

Re: The 40-point rule

Post by Andrew Farthing » Fri Aug 21, 2009 9:09 pm

Roger de Coverly wrote:It's parallel to the Elo 350 or 400 point rule and it's an incentive for the higher rated player to play.

At 145 would you want to play a league or tournament game against a player graded 60 with the available points 155 for a win, 60 for a draw and 10 for a loss?
Well, I won't deny that I'd feel some extra pressure, but to be honest I get that even with the prospect of "only" suffering with 105 for the draw and 55 for a loss. What I find in practice is that this tends to be an incentive for me to play in stronger sections in congresses whenever I'd be close to the highest rated player in my "natural" section and would face the prospect of games against players who are much lower rated.

Anyway, as I said in my previous post, there was nothing behind the query except curiosity, so thanks for the explanation!

Andrew

David Williams
Posts: 336
Joined: Sun Mar 29, 2009 8:37 pm

Re: The 40-point rule

Post by David Williams » Fri Aug 21, 2009 11:57 pm

I thought the idea of a grading system was to give some idea of players' strength, not to incentivise them to play. If a 145 player needs to score 90% against a 105 to maintain his grade, and a 105 needs to score 90% against a 65 to do likewise, shouldn't the 145 need to do rather better against the 65?

Richard Bates
Posts: 3338
Joined: Fri Nov 14, 2008 8:27 pm

Re: The 40-point rule

Post by Richard Bates » Sat Aug 22, 2009 7:27 am

I don't really think much of the "incentivisation" argument. However i can think of three others.

Firstly, and perhaps most dubiously, i can imagine that to disregard the 40pt rule for draws and losses would make the administration of the system slightly more complex, although this is perhaps based on an assumption about how the system is administered in practice.

More relevantly perhaps, is an argument of consistency. The vast majority of rankings systems in chess, be they grading/rating systems or tournaments (where the aim is after all to find the best player), contain an assumption that two draws are equivalent to a win and a loss. Removing the 40pt rule as suggested would break that assumption, and quite illogically if adopted in both directions. ie. the stronger player would perform at 60 with 2 draws and 80 with a win and a loss, whereas the weaker would perform at 140 with two draws and 120 with a win and a loss. Leaving to one side the assumption of equivalence between two draws and win/loss, surely if one is considered better than the other it should be consistent for both stronger and weaker player? (all this of course leaves to one side the traditional objection to the ECF system that two players will swap grades with a 50 percent performance rather than end up at the same level).

Thirdly, and perhaps the best justification, is to consider the 40pt rule as the grading system's protection against statistical outliers. Essentially within any sample of games used to approximate an individual's strength there is the possibility of "freak" results (caused by any number of reasons, most obviously the inaccuracy of the opponent's grade itself) rendering that approximation inaccurate. By defining that as any result giving a notional performance of +/- 90pts such freaks can be automatically disregarded. The fact that someone might have claimed to have seen the game with their own eyes is irrelevant - statistically it cannot (should not?) have happened so is ignored.

Andrew Farthing
Posts: 614
Joined: Sun Dec 28, 2008 11:39 pm

Re: The 40-point rule

Post by Andrew Farthing » Sat Aug 22, 2009 7:48 am

Richard Bates wrote:Thirdly, and perhaps the best justification, is to consider the 40pt rule as the grading system's protection against statistical outliers. Essentially within any sample of games used to approximate an individual's strength there is the possibility of "freak" results (caused by any number of reasons, most obviously the inaccuracy of the opponent's grade itself) rendering that approximation inaccurate. By defining that as any result giving a notional performance of +/- 90pts such freaks can be automatically disregarded. The fact that someone might have claimed to have seen the game with their own eyes is irrelevant - statistically it cannot (should not?) have happened so is ignored.
I'm not a statistician, so my comment may be naive:

I can think of one game last season when I drew with a player graded 45 (old basis). My then grade was 122. My opponent wasn't a junior, and he hasn't improved his grade in the past year. His grade is probably a reasonably fair reflection of his average standard of play, because he tends to be very careless. However, he can play solidly and knows the endgame pretty well, so if he gets there in decent shape he can do well. As well as achieving a non-fluky draw against me - if anything I was lucky not to lose - he has beaten a 110 and drawn with a couple of players over 100 in the last two years.

In this case, I wouldn't argue that his grade is particularly inaccurate (I'd previously played lots of friendly games against him and had never failed to win). It's probably a little low, because when he does get a good result, the benefit is reduced by the 40-point rule. I don't therefore understand why a result such as the draw against me should be considered a "freak"; it's the fair reflection of a very inconsistent player, whose grading arguably ought to be allowed to be a straightforward average of the full range of those inconsistent results. It feels like a (minor) injustice to me.

Andrew

Richard Bates
Posts: 3338
Joined: Fri Nov 14, 2008 8:27 pm

Re: The 40-point rule

Post by Richard Bates » Sat Aug 22, 2009 9:30 am

Andrew Farthing wrote:
Richard Bates wrote:Thirdly, and perhaps the best justification, is to consider the 40pt rule as the grading system's protection against statistical outliers. Essentially within any sample of games used to approximate an individual's strength there is the possibility of "freak" results (caused by any number of reasons, most obviously the inaccuracy of the opponent's grade itself) rendering that approximation inaccurate. By defining that as any result giving a notional performance of +/- 90pts such freaks can be automatically disregarded. The fact that someone might have claimed to have seen the game with their own eyes is irrelevant - statistically it cannot (should not?) have happened so is ignored.
I'm not a statistician, so my comment may be naive:

I can think of one game last season when I drew with a player graded 45 (old basis). My then grade was 122. My opponent wasn't a junior, and he hasn't improved his grade in the past year. His grade is probably a reasonably fair reflection of his average standard of play, because he tends to be very careless. However, he can play solidly and knows the endgame pretty well, so if he gets there in decent shape he can do well. As well as achieving a non-fluky draw against me - if anything I was lucky not to lose - he has beaten a 110 and drawn with a couple of players over 100 in the last two years.

In this case, I wouldn't argue that his grade is particularly inaccurate (I'd previously played lots of friendly games against him and had never failed to win). It's probably a little low, because when he does get a good result, the benefit is reduced by the 40-point rule. I don't therefore understand why a result such as the draw against me should be considered a "freak"; it's the fair reflection of a very inconsistent player, whose grading arguably ought to be allowed to be a straightforward average of the full range of those inconsistent results. It feels like a (minor) injustice to me.

Andrew
Maybe it's your grade that is too high... 8)

On a serious note, you are looking at his occasional good result and suggesting that he is not getting the full benefit of those results. However you are ignoring the possible benefit he is getting on his losses. If he is genuinely so inconsistent that he can fluctuate so wildly in strength around a 45 average, then he is potentially gaining massive protection on his losses. For every loss he is being given his opponent's grade -50 (or 35 for players 85+). For him a more appropriate formula might be "opponent's grade-80"! It works both ways.

In fact, thinking about it some more, it is the application of +/- 50 that is probably key, with the 40pt rule following directly from that. If the range was set at +/- 30 then we would have a 25pt rule perhaps, if +/- 70 a 60pt rule. There is always the possibility of players with variations in strength to render +/- 50 too narrow a range. Similarly for others it may be that the range is too high.
Last edited by Richard Bates on Sat Aug 22, 2009 9:41 am, edited 2 times in total.

Neill Cooper
Posts: 1299
Joined: Sun Feb 24, 2008 4:43 pm
Location: Cumbria

Re: The 40-point rule

Post by Neill Cooper » Sat Aug 22, 2009 9:37 am

Howard's graph on page 31 of the ratings debate (http://www.ecforum.org.uk/viewtopic.php ... &start=450) shows that with an 80 point grading difference the weaker player typically gets 5% (1 draw in 10) for category A to C players and 10% (1 win in 10) for all category players. Even with 100 point difference the values are not much lower. The graph also shows that with a 40 point difference the lower graded player scores 20%, not the 10% the grading system assumes. (I think these differences are much more significant than the 10 point difference being 57% not 60%.)

So with the new grades a 40 point difference is what would have been a 50 point difference in the past, there should be many fewer games where the 40 point rule is invoked. [Unfortunately due to Junior grades being reset each year I know of many cases where it has been used where players (or their opponents) have had reasonably good or bad years.]

Richard Bates
Posts: 3338
Joined: Fri Nov 14, 2008 8:27 pm

Re: The 40-point rule

Post by Richard Bates » Sat Aug 22, 2009 9:46 am

Neill Cooper wrote:Howard's graph on page 31 of the ratings debate (http://www.ecforum.org.uk/viewtopic.php ... &start=450) shows that with an 80 point grading difference the weaker player typically gets 5% (1 draw in 10) for category A to C players and 10% (1 win in 10) for all category players. Even with 100 point difference the values are not much lower. The graph also shows that with a 40 point difference the lower graded player scores 20%, not the 10% the grading system assumes. (I think these differences are much more significant than the 10 point difference being 57% not 60%.)

So with the new grades a 40 point difference is what would have been a 50 point difference in the past, there should be many fewer games where the 40 point rule is invoked. [Unfortunately due to Junior grades being reset each year I know of many cases where it has been used where players (or their opponents) have had reasonably good or bad years.]
It may just be that the grading system is sound, but the game of chess mucks it up! ;)

Neill Cooper
Posts: 1299
Joined: Sun Feb 24, 2008 4:43 pm
Location: Cumbria

Re: The 40-point rule

Post by Neill Cooper » Sat Aug 22, 2009 10:30 am

Richard Bates wrote:It may just be that the grading system is sound, but the game of chess mucks it up! ;)
I'm sure it does: Consider 3 players graded A=170, B = 150, C = 130

Just because A scores 70% against B, and B scores 70% against C, why should A score 90% against C?

I notice that ELO uses a curve not a straight line, and the results are also curved. But I'm not expecting a perfect system, just one that is reasonable. I was reasonably happy with the old grading system, and the new one until Junior's were taken as new players every year.

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: The 40-point rule

Post by Roger de Coverly » Sat Aug 22, 2009 10:46 am

Richard Bates wrote:
It may just be that the grading system is sound, but the game of chess mucks it up! ;)
The Howard Grist graph is attempting to measure the reliability of the grading system as a results predictor but it's also showing the variability of results.

As a for example if he had a player whose personal results distribution was that they drew against everyone within 25 points, beat everyone 25 points below them and lost to everyone 25 points above them, then if they played an equal number of players in both directions their grade would be correctly right in the middle. They wouldn't improve the shape of Howard's graph very much.

The grading system assumes a linear relationship between grade difference and result. So in matches between 185s and 175s, it predicts a narrow win for the former, similarly between 175s and 165s. In terms of numbers, it requires 9-1 between 185s and 145s but the actual distribution of results may well be different.

We should have been shown the Howard Grist graph three years ago when the debate about reworking grades first went public. At least we would know where the grading team were coming from. I wouldn't myself infer that the graph shows the grades needed changing without more tests to see how much of the curve was just down to player variability or non-linear relationships across the grading differences. I'd also want to know whether the curve included junior enhancements and what it looked like if you removed them. It did however seem to show that players of the same grade scored 50% against one another - which is a measure of correctness.

E Michael White
Posts: 1420
Joined: Fri Jun 01, 2007 6:31 pm

Re: The 40-point rule

Post by E Michael White » Sat Aug 22, 2009 11:11 am

I have a sneaking suspicion that both the ECF and FIDE use the wrong type of metrics. I havent tried showing this mathematically yet but I think the FIDE way with k factors would be better with a linear metric, as Jeff Sonas has recently said, but I also think that the ECF with its averaging approach needs something a bit more curvy particularly at the ends. The straight bit in the middle is approximately ok but its a pain at the 40 point difference mark where there is a sharp corner.