Minimum Grades

[Paul Dupré]

If we're going to switch to Elo, which I don't have a major problem with, then we should make sure that the details are sorted in such a way that what we move to is an improvement on what we have at the moment. Some work would need to be done (simulations etc.) before coming to any conclusions, and deciding on whether or not it's worth it.

Off course its worth it, otherwise we can waste the next 20 years trying to fit extremely weak players into a system that was never designed for such.

I've been a grader for 6 years and been graded for nearly forty.

WTF. Simulations have already been done for the Arpad Elo system for some 73 years. It works just fine.

[Paul Dupré]

It is already a fact that grading players out of the grading band 50-250 will create problems in the long term. Now that we are grading players less than 50, we should either add say 100 or possible 200. So, therefore make the grading band 50 to 450, increasing all current grades by 200

I don't think you understand how the system works.

My Grading.png

I think you'll find I do know how the grading system works - I've been calculating my own grading for over 20 years on spreadsheets. This summary shows how many times the BCF/ECF got it right (figures in blue in the far right column). Far more often recently since graders stopped using pen and paper. In 1985 they were way out by 10 points.

[Sean Hewitt]

As is perhaps known, I didn't and still don't particularly believe in the hypothesis of stretching

I seem to recall that you claimed vociferously that there was no stretching and that all the regrading exercise would do would be to give free grading points to GMs and other stronger players, whose grades would rocket upwards as a result.

We're 3 years on from the implementation of the grading fix. When might we expect to see these rocketing grades?

[Sean Hewitt]

The rest of us face the opposite dilemma, of having no evidence for stretching. Any chance of the statistical analysis being published?

Paul - I did quite a bit of work on this at the time. In a nutshell, the problem was that if two groups of players were graded 25 points apart, the grading system required them to score 75% v 25% to maintain their grades. Yet, the data showed that they actually scored 68% v 32%. In other words, it was more difficult to maintain a grade playing lower graded players than it should have been.

I don't have the analysis any longer, but I did write a summary of my findings (in laymen's terms) which you can find here. At the time I wrote this in 2006, the grading team simply refused to believe the evidence. I remember Chris Majer telling me that he "simply didn't believe" that there could be anything fundamentally wrong with the ECF grading system. 3 years later, not only did they believe it but they implemented most of my suggestions (notably the treatment of junior players). In fact, they actually went further than I suggested by recalculating everyone's grade from scratch rather than 'fixing' their old grade.

Sceptics like Roger stated that we would see grades of GMs rocket as they picked up loads of free grading points as a result of the fix. Those of us who understood the problem said that wouldn't happen. I'll leave you to judge which is right.

I hope that the grading team re-perform the check each year as I recommended. Although the changes I suggested will reduce the stretching going forward, they will not eliminate it as the Clarke system as used is inherently deflationary. By keeping an eye on it, it may be possible to introduce the occasional 'leap' grading point to keep things accurate.

[Sean Hewitt]

The gradng system is currently unreliable in the range 200-230 (compare ECFs and Elos),

That was also the case well before the grading fix (see the chart in my 2006 paper) so doesn't really tell us anything.

[Roger de Coverly]

Paul - I did quite a bit of work on this at the time. In a nutshell, the problem was that if two groups of players were graded 25 points apart, the grading system required them to score 75% v 25% to maintain their grades. Yet, the data showed that they actually scored 68% v 32%. In other words, it was more difficult to maintain a grade playing lower graded players than it should have been.

You never did address the "so what" point. If you score 68% against players 25 points below you, then you score 32% against players 25 points above you. So for a player playing an average field of their own grade and scoring 50%, their grade remains stable.

It was eventually revealed that the hypothesis was one of spread rather than deflation. By spread, I mean that the difference in grading points between the very top and bottom players and the average was widening. The evidence of this at the tail of the distribution is the appearance of negative grades. On the hand, if you use the term deflation to mean a reduction in values for the whole grading range, the series of averages didn't support this.

But let's think about the effect of 68% v 75% on players who are the top of domestic chess. They don't have players above them they can score 32% v 25%, so you might expect so see a drift of the grades downwards towards the middle. Similarly those at the bottom have no-one to beat, so scoring 32% against 25% should pull them back towards the middle. So the effect of 68% v 75% should be to compress the system rather than to spread it.

The effect of the changes has been to give "average" players grades about 20 points too high, so the mean and median are around 20 points higher than they've been historically. You also have to ask whether a grading system that allows players in their sixties a lifetime best grade can really be considered not to have been inflated.

[Sean Hewitt]

You never did address the "so what" point.

I shall be sure to use the "so what" point every time you bang on about how unfair the membership scheme is to players who play 1 game a season.

[Alex Holowczak]

I think you'll find I do know how the grading system works

So, why are you suggesting that adding 100, 200, or any other arbitrary number to everyone's grade will do anything, without changing the underlying algorithm?

[Paul Dupré]

I think you'll find I do know how the grading system works

So, why are you suggesting that adding 100, 200, or any other arbitrary number to everyone's grade will do anything, without changing the underlying algorithm?

Why are you struggling to understand that adding 200 will get rid of all the zero grades.

[Alex Holowczak]

I think you'll find I do know how the grading system works

So, why are you suggesting that adding 100, 200, or any other arbitrary number to everyone's grade will do anything, without changing the underlying algorithm?

Why are you struggling to understand that adding 200 will get rid of all the zero grades.

I'm not. I get that completely, but that isn't the problem.

The problem with the grading system isn't the zero grades. The problem is that grading assumes your published grading is approximately your strength. For juniors, or new players to the game, the published grade is less likely to be approximately your strength, for at least two reasons: (1) The variance in your ability from one game to the next is higher - particularly for juniors, and (2) The player is improving - quite rapidly in the case of juniors.

Adding 200 to everyone's grade, so that additional juniors can get grades without them all being 0, would actually make the grading system less accurate than it is now, because you increase the inaccuracy of the published grades.

We could increase the number of graded players overnight if we reduced the number of graded games required to get an initial grade from 9 to 8 (or even fewer), but this would reduce the mathematical integrity of the system, so no one has. It has been reduced from 10 to 9 already.

There has to be a balance between getting people grades en masse, and not buggering up the system. Without having done the statistics to show as much, adding an arbitrary constant to everyone's grade, to allow more junior events to tag on at the bottom of the grading list, is changing that equilibrium in favour of buggering up the system. For what benefit? We get children with a relatively low ability onto the bottom of the grading list, who will invariably stop playing chess if their grade isn't high enough for them to be able to make the transition from junior chess into adult chess while in their teens - which is the vast majority of 10-12 year olds. So why bother getting them on the list in the first place?

[Paul Dupré]

So why bother getting them on the list in the first place?

Why are you asking me, ask CCF. They get paid for getting Juniors onto the grading list.

[Paul Dupré]

By the way who's stupid idea was it to use iteration.

[Alex Holowczak]

So why bother getting them on the list in the first place?

Why are you asking me, ask CCF. They get paid for getting Juniors onto the grading list.

The question was rhetorical in nature, intended to provoke thought. It seems to have provoked your usual anti-CCF views, alas.

By the way who's stupid idea was it to use iteration.

I don't know which individual amongst the grading team was - it was before my time - but between them, they came up with it.

If you want continuity between grades from one year to the next, then it was a stupid idea. If you want an accurate grading system, it was an improvement.

It was thought, rightly or wrongly, that the improvement to the accuracy was necessary in order for the system's credibility to be retained.

I was winning a number of U100 games in the last season with the old grades. Any system that means I win things clearly lacks credibility.

It's my suspicion - again, I haven't done the statistics - that iterating everybody's grade would lead to more a more accurate grading system. It only applies to juniors because they're the players whose grades are most likely to not reflect their actual playing strength, and iteration smooths the bumps, if you want to think of it that way. For adult players, the fundamental assumption I mentioned above holds.

[Paul Dupré]

If you want continuity between grades from one year to the next, then it was a stupid idea. If you want an accurate grading system, it was an improvement.

It was thought, rightly or wrongly, that the improvement to the accuracy was necessary in order for the system's credibility to be retained.

Well, obviously they didn't use good test data. Otherwise, they would understood why iteration doesn't work on big percentages (ie. Callum Kilpatrick 8½/9), OR for people who score zero.

It reminds me of the two lighthouse men who played each other 30 games during one chess season. One graded 180, and the other 140. During their matches there was a lot of wins and draws, but at the end the score was 15-15. Guess what happened next, when the new grades came out?

[Alex Holowczak]

Well, obviously they didn't use good test data. Otherwise, they would understood why iteration doesn't work on big percentages (ie. Callum Kilpatrick 8½/9), OR for people who score zero.

You do them a disservice. Very few rating systems work if someone scores 9/9 or 0/9, or a score very near to it. If you take Elo, and I play an average rated field of 2000. If I score 0 out of 9, I'd have an initial rating of 1200, because they subtract 800 as a substitute for infinity. This is why you need at least 1 out of 9 for an initial rating. If I score 9 out of 9, I'd have an initial rating of 2135. If you score more than 50%, this is a fudge to lower your rating, expecting you to then bring it up to par when your k is still 30.

If win all my games apart from one, and then draw it, what grade should I be assigned?

It reminds me of the two lighthouse men who played each other 30 games during one chess season. One graded 180, and the other 140. During their matches there was a lot of wins and draws, but at the end the score was 15-15. Guess what happened next, when the new grades came out?

The 180 would come out as 140, and the 140 as 180.

Exactly the same would happen under Elo. The same thirty games, between a 1400 and an 1800, if you lost 15 and won 15, would be that the 1800 becomes 1611, and the 1400 becomes 1589, for k = 15. For k = 30, on the other hand, the 1800 would become 1422, and the 1400 would become 1778.

One issue here is the size of the k-factor. The ECF's version of k is adding 50 for winning and subtracting 50 for losing. The system would be smoother if a lower number than 50 was used.

The other issue is how often you keep count. If the FIDE-ratings updated after every game, then the two ratings would indeed converge to 1600 if they played enough games. With ECF it wouldn't, but the ECF system is based on the lighthouse men issue not arising in practice. I think that, as grading assumptions go, this is a pretty safe one. Most players, over the course of their 9-30 games, will play a variety of different opponents. So the system doesn't need to cater for this. This notwithstanding, if ECF grades were on a rolling, live basis, and incorporated previous grading history as it went, then in the lighthouse men problem, the grades would, too, converge. The second issue is how often you produce a new grading list. Do it live, and it converges. Do it after a year, and you could overshoot.

The second issue will always be present in any rating system. This is why FIDE, presumably, are trying to bring out rating lists with ever-increasing frequency.

Having 50 for a win and 50 for a loss was reasonable over a 12-month period of grading. Now that we have 6-monthly grading, perhaps we need to reduce 50 to 25? People commented elsewhere that their grades seemed more up and down than before. Clearly this was only anecdotal, but reducing 50 to 25 would help to reduce their up-and-down-ness.

Why do we use 50? That's what Clarke did, presumably, and we've copied him ever since. He was working with Elo, so Elo presumably agreed with the choice of 50.