New grades (split from Editorial thread)

General discussions about ratings.
Matthew Turner
Posts: 3600
Joined: Fri May 16, 2008 11:54 am

Re: New grades (split from Editorial thread)

Post by Matthew Turner » Thu Jun 05, 2008 1:44 am

Is it then true that (forgetting the forty point rule) if I play an ECF graded player then the difference between a win and a loss is 100 points. If I play a non-ECF graded player in an ECF graded event then the difference between a win and a loss is 100 - 100/n where n is the number of games that that player plays against ECF graded opponents. Doesn't that mean for players who play in the 4NCL there is 'compression' in the system, because the 'benefit' of winning is lessened. This would mean that there would be deflation for the top players. However, they would soon be able to get these points back on the weekend circuit and this deflation would work it's way down the ranks. Does this hypothesis fit the facts?

Howard Grist
Posts: 83
Joined: Tue Dec 12, 2006 1:14 pm
Location: Southend-on-Sea
Contact:

Re: New grades (split from Editorial thread)

Post by Howard Grist » Thu Jun 05, 2008 2:37 am

Matthew Turner wrote:Is it then true that (forgetting the forty point rule) if I play an ECF graded player then the difference between a win and a loss is 100 points. If I play a non-ECF graded player in an ECF graded event then the difference between a win and a loss is 100 - 100/n where n is the number of games that that player plays against ECF graded opponents.
Yes, except that n is the number of games that the player plays in ECF graded events - all games in graded events count towards a player's grade - even those played between two ungraded players.
Matthew Turner wrote:Doesn't that mean for players who play in the 4NCL there is 'compression' in the system, because the 'benefit' of winning is lessened. This would mean that there would be deflation for the top players. However, they would soon be able to get these points back on the weekend circuit and this deflation would work it's way down the ranks. Does this hypothesis fit the facts?
I don't see what's so different about you playing an ungraded GM in the 4NCL to a somone playing a newcomer at the bottom end of a congress - except that the latter happens more frequently. I think you're looking in the wrong place, the top end of the scale has the more stable set of players playing a larger number of games. The most obvious place to see that things aren't as they should be is at the lower end of the spectrum, where most people posting here have long since left behind. Historically, a complete beginner would be given a grade of 50-60 by the event organiser and you very rarely saw published grades below this figure. However my club currently has 7 adult players graded below 60, they are all relative newcomers to the game and so you would expect them to be improving. However, they all went down when the grades were published last July.
Former ECF Grading System Programmer

Sean Hewitt

Re: New grades (split from Editorial thread)

Post by Sean Hewitt » Thu Jun 05, 2008 6:48 am

Matthew - I apologise as I had forgotten that the conversion is used for ECF members playing foreigners in foreign tournaments if they have pre-registered to do so!! However, I suspect that a very very small percentage of the graded games each year come from this source.

I was thinking only of English events where the conversion is not used, even if the player is a foreigner with a rating.

In such an event the player is treated as any other ungraded player.

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: New grades (split from Editorial thread)

Post by Roger de Coverly » Thu Jun 05, 2008 9:50 am

I think you're looking in the wrong place, the top end of the scale has the more stable set of players playing a larger number of games.
That of course is where the non-believers in the theory of historic deflation are coming from.
The most obvious place to see that things aren't as they should be is at the lower end of the spectrum, where most people posting here have long since left behind. Historically, a complete beginner would be given a grade of 50-60 by the event organiser and you very rarely saw published grades below this figure. However my club currently has 7 adult players graded below 60, they are all relative newcomers to the game and so you would expect them to be improving. However, they all went down when the grades were published last July.
Do we have any real clue as to why this is and why it won't repeat in future years? At that level maybe, you are trying to distinguish between players who can deliver mate with K+R v K against those that need K+Q v K. Because it should be quite possible at that level to improve by well over 50 points in the course of a season, the difference between the begin year strength and end year strength could be huge.

The introduction of the software for estimating a new player's grade from his results may have had an effect on newcomers of low strength. In the "old" days wasn't the initial estimate done by "graders'/organisers'" guess? I'm thinking here of a player who starts the season with a strength of 20 but finishes it with a strength of 100. An end of season league grader's estimate would probably have been towards the high end of the 20-100 range. Would the estimation software come out with something like 60 by virtue of averaging results over the year? Does the estimation software give more weight to end of season results?

Some adult players round the 100-120 mark have a feeling of their grades being pulled downwards, those in the 150 plus range just see their peer group at the around the same grades for many years - as verified by the history on the ECF website. Is there also a hidden inflation, protecting the higher rated? 40 point rule perhaps? The higher the grade, the more often the 40 point rule downwards comes into play.

Looked at objectively, the grade calculation gives a performance measure over a time period. If you have a rapidly improving player, the calculation will give you a mid period measure. The value of a mid-period measure as a predictor of the following periods results is not high.

Matthew Turner
Posts: 3600
Joined: Fri May 16, 2008 11:54 am

Re: New grades (split from Editorial thread)

Post by Matthew Turner » Thu Jun 05, 2008 10:15 am

I am still struggling a bit with this, but I am sure that there is something going on with this grading performance calculation software. Let's say that I am new to chess and only manage to play one graded game in my first year (this is perfectly plausible if my school gets knocked out in the first round of the National Schools'). If I lose to a player graded at 100, then it appears to me that my grading permance is 40, since my 'estimated' grade is 50 and I have lost one game to a 100 for which I must lose at least 10 points hence my next season's grade will be 40. I may have misunderstood again, but this looks to have some seriously deflationary possibilities to me.

Matt Harrison
Posts: 129
Joined: Fri Apr 06, 2007 4:51 pm

Re: New grades (split from Editorial thread)

Post by Matt Harrison » Thu Jun 05, 2008 11:38 am

It's a tricky business to be sure.

Something seems to be awry with the system, but it doesn't appear to me to be uniform. If you look at established older players in the 150-200 range their grades seem to be fairly consistent over time. This is purely anecdotal, based on looking at the grading history of my son's adult opponents for the past year or two on the grading database. The argument for uplifting the grades of these players (such as Roger and Tim) seems poor.

But if you look back at junior tournaments from 10 years ago, grades for the stronger players (not the very top ones, but the average in a decent tournament) seem to be a lot higher than they are now. I've looked at the West of England congress and the London Junior congress websites.

My son (grade 119 this year), will have a grade of about 140 next year, so he's improving - which is clear from his results over the year. But his grading improvement is all at the expense of adults. This year he's played about 44 graded games, about 50% against fellow juniors, 50% against adults. Performance versus juniors is 121, versus adults 140. This is obviously only a single example, but conversations with his peers and their parents would tend to confirm that a junior of grade 130 is substantially stronger than an adult of the same grade.

My suspiscion (and I have a maths degree as well so can understand Sean's analysis) is that there is something in the system at the lower end that gives juniors an artifically low starting grade, and the 40 point rule then slows their increase to their 'correct' grade. As new juniors tend to only play in junior tournaments this is perpetuated. The stronger juniors who keep playing past age 11 or 12 then tend to enter the adult chess playing pool via congresses and then leagues. They are good players, and substantially undergraded. The junior bonus goes some way to correct this, but nowhere near enough, plus it tails off as the juniors get older, which may be wrong as well.

Another (unrelated) point that might be worth considering is whether the relationship between performance and grading differential is linear, especially for lower graded players. Even the weakest beginners have access to computers and online play. This doesn't help in all areas of the game evenly, but it seems to me that it certainly widens opening knowledge. I played as a reasonable strength junior in the 70s and early 80s. But my knowledge of opening theory was limited to 2-3 openings per colour, based on what books I could borrow from the local library. When I started playing again a few years ago, Fritz (and my son) helped widen my knowledge very quickly. Against a strong player I will lose, but against someone only 20-30 points higher I'm less likely to be wiped out in the opening than I would have been 20 years ago. How this could be tested I have no idea.

Sean Hewitt

Re: New grades (split from Editorial thread)

Post by Sean Hewitt » Thu Jun 05, 2008 2:58 pm

Matt - You're pretty much on the money.

The issue causing the defaltion (or, at least, the major cause) is rapidply improving players. This is usually juniors, although an adult who rapidply improves is also a problem. I would classify rapidply improving as gaining more than 10 points.

The problem with these players (and there are a significant number of them) is that when a stable player plays them, their grading "reward" - ie the number of points they receive from the game - is lower than the level of performance dictates. Let me give you an example. 3 players play a series of games - lets say 20 each against the other 2 players (so 40 games each). Player A and B are both stable 150 players. Player C is a junior currently graded 125 but actually playing at 150 strength. So, all three are in fact of the same strength, and we would expect each match to finish 10:10 so lets assume that they do.

Player C will be rewarded with a grade of 150, properly reflecting his current playing strength because he has only played stable players.

Player A and Player B are rewarded with grades of 138 - significantly below their actual playing strength because they have played lots of games against rapidly improving player(s).

The example is of course exagerated. But given that 32% of the current ECF grading base are juniors (Under 18) this is I believe the major contributing factor - and we are all affected to differing extents.

Matthew Turner
Posts: 3600
Joined: Fri May 16, 2008 11:54 am

Re: New grades (split from Editorial thread)

Post by Matthew Turner » Thu Jun 05, 2008 3:09 pm

Sean, I am sure what you say is true, but it has always been true. It does not explain why deflation has suddenly become a feature of the system. There must have been a new external force that has created deflationary pressures. My example suggests that grading performance software (that estimates the grades of ungraded players) is quite a likely culprit. It would be interesting to know when this was introduced to see if it ties in with the first signs of deflationary pressures.

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: New grades (split from Editorial thread)

Post by Roger de Coverly » Thu Jun 05, 2008 5:26 pm

There must have been a new external force that has created deflationary pressures. My example suggests that grading performance software (that estimates the grades of ungraded players) is quite a likely culprit. It would be interesting to know when this was introduced to see if it ties in with the first signs of deflationary pressures.
5 years ago - here's the contemporary report

http://www.sccu.ndo.co.uk/0203/grad.htm

In particular Richard H's comment
There's one in particular that is causing concern on the Grading team. Negative grades. When the scale was fixed, I don't know how many years ago, it wouldn't have occurred to anyone to grade U8 tournaments, or even U80, and the idea of a negative grade would have seemed laughable. It isn't now. Don't look for them in the printed list. If you work out at minus 70, and people do, your grade is printed as 1. But it will count as minus 70 in calculations, as the program stands currently. Is this honest? What's the solution? Print it as minus 70? Print it as 1 and count it as 1? Add 200 to everyone's grade? Switch to Elo? (But be careful, you don't have to go much below minus 70 before you're negative in Elo as well.) The problem has been around for years, but it has come to a head and we're past the point where we can ignore it and hope it will go away. It will only get worse, if the increase in junior grading continues. The new calculation routine will even accelerate the plunge, because it's more honest than the old one about very low grades. My solution? Add 200 points, or 300, or 500, to everyone's grade. And do it in the Grading List 2004. It will be an upheaval, but postponing it another year won't help. Opinions invited.

Matthew Turner
Posts: 3600
Joined: Fri May 16, 2008 11:54 am

Re: New grades (split from Editorial thread)

Post by Matthew Turner » Thu Jun 05, 2008 7:04 pm

It's interesting, but I think you have to be carerful about causality here - surely deflation causes negative grades, rather than visa-versa. Really the most interesting comment from is "The new calculation routine will even accelerate the plunge, because it's more honest than the old one about very low grades". Wouldn't this imply that rather than adding 200, 300 or 1752 points to each person's grade the answer would be to make the computer program less honest. Just say no-one can have a grade of below 25 or 40 or whatever you like really.

Howard Grist
Posts: 83
Joined: Tue Dec 12, 2006 1:14 pm
Location: Southend-on-Sea
Contact:

Re: New grades (split from Editorial thread)

Post by Howard Grist » Thu Jun 05, 2008 7:39 pm

Some ECF grading system history:-

1987: National grading sytem introduced. This works on the basis of graders calculating points and games totals for each player in an event.
1999: New system introduced, working on the basis of reporting results, with these being converted to points and games centrally, although graders could still send in results in the 1987 format
2001: Problems with the 1999 system result in reverting to the 1987 system, although the 1999 system is still used to convert results sent in the 1999 format to the 1987 format.
2003: The current system is introduced.

The 2003 results exhibit a very similar mismatch of expected to actual results as those in 2007.

It could be that the problems were introduced in the transition period from 1999-2002, or they could be have been there for much longer but it's impossible to tell from the points/games totals of the time.

As to problems with the calculation of grades for new players, there were definitely problems in this area in 2001-2. One lucky 11 year old girl came out with a grade of 148 based on scoring 4½/6 in an U80 event, however the general lack of user-friendliness of the 1999 stystem didn't encourage investiagation into whether the grades it calculated were actually correct.
Matthew Turner wrote:If I lose to a player graded at 100, then it appears to me that my grading permance is 40, since my 'estimated' grade is 50 and I have lost one game to a 100 for which I must lose at least 10 points hence my next season's grade will be 40. I may have misunderstood again, but this looks to have some seriously deflationary possibilities to me.
Your performance for this game will indeed be 40. This wouldn't however be a published grade. If you subsequently play 9 games against someone graded 100, and score 50%, your published grade would be 94, and your grade for each of your opponents would also be deemed to be 94.
Former ECF Grading System Programmer

Sean Hewitt

Re: New grades (split from Editorial thread)

Post by Sean Hewitt » Fri Jun 06, 2008 7:36 am

Matthew Turner wrote:It's interesting, but I think you have to be carerful about causality here - surely deflation causes negative grades, rather than visa-versa. Really the most interesting comment from is "The new calculation routine will even accelerate the plunge, because it's more honest than the old one about very low grades". Wouldn't this imply that rather than adding 200, 300 or 1752 points to each person's grade the answer would be to make the computer program less honest. Just say no-one can have a grade of below 25 or 40 or whatever you like really.
Matthew - Absolutely correct. Deflation has caused negative grades.

How log has deflation been around? As far as I could see (and this is a bit of a guess as its based on extrapolation - the exact data doesnt exist) the problem has been evident for about 30-35 years (see my letter dated 14/12/06 http://www.sccu.ndo.co.uk/letgrad.htm) - I suggested 1970-75. This is pure conjecture, but could this co-incide with the explosion of junior chess after the 1972 Spassky - Fischer match?

But I have to disagree about making the program less honest!! Just as you rightly say that negative grades are caused by deflation rather than the other way around, the same is true of low grades. They are the result of deflation. Fix the problem by correcting the grades, treat juniors in the way that I have suggested - and the problem is gone.

I'm sure you realise that the uplift will not be a flat rate for all players. Essentially, the lower graded you are, the more your grade will increase - squeezing the difference between any two players together.

E Michael White
Posts: 1420
Joined: Fri Jun 01, 2007 6:31 pm

Re: New grades (split from Editorial thread)

Post by E Michael White » Fri Jun 06, 2008 10:06 am

My view is that deflation became more evident when different player activity rates became more pronounced. This will have happened when weekend tournaments increased in number around 1970 onwards. The ECF grading formula does not cope well with varying activity rates. Prior to 1970 from what I remember most players played in League chess and only occasional congresses and most players seemed to play about 10 games. Under those conditions ECF grades behave very similarly to FIDE grades. Richard Clarkes papers do not mention the effect of wide variations in player activity so he probably did not then consider it a problem. This effect could be reduced by increasing the 30 game rule to 50 or 70 for A grades but this is probably not what players want.

I don’t believe including juniors after 1972 in increased numbers had as much effect as Sean seems to think because in those pre electronic calculator days new players were sometimes provisionally estimated as 100 for the purpose of calculating results of graded players. Ungraded players only got points against graded opponents. This approach would have had a stabilising effect on the grades of the graded majority.

To me the indiscriminate use of iterative programs which produce results a year out of phase as well as exaggerate the effect of deflation in the iteration has made matters worse. The results of iteration should never be accepted without first establishing whether the process produces consistent and unbiased estimators.

The new starter process does not seem to have as an objective to maintain the level of those with published grades. New players should be brought in in a way that does not affect the level of others grades. Various tests can be done from one year to the next to adjust the overall level of new starters.

Matthew Turner
Posts: 3600
Joined: Fri May 16, 2008 11:54 am

Re: New grades (split from Editorial thread)

Post by Matthew Turner » Fri Jun 06, 2008 10:18 am

Fairly obviously, Mean grade = Total number of grading points / number of players. So, how are points lost to the system? Well primaily, it comes from juniors entering with a grade of 40, improving until they are 11, getting to 100 grade and giving up when they join senior school. Hence if juniors entered with a grade of 80 the problem would be vastly reduced, remembering of that the inverse process occurs at the other end of the age spectrum.
The point is that artificially inflating current grades will only stave off further problem. A slight alteration to the grading program could substantially 'fix' deflation. How about everyone U18 will join the grading calculations with an estimated grade of 80. It might be less accurate in a mathematical sense, but it would seem to me to produce a far better grading system.

Matt Harrison
Posts: 129
Joined: Fri Apr 06, 2007 4:51 pm

Re: New grades (split from Editorial thread)

Post by Matt Harrison » Fri Jun 06, 2008 10:35 am

Have to say this seems to me to be a sensible pragmatic approach. Another one would be to use Rapidplay grades as the starting standard grade for juniors. At U11 level, the length of games isn't radically different for most juniors regardless of time control. It looks to me like the iteration for ungraded produces lower grades than are desired, and then the 40 point rule slows down the ascent of improving players to their 'true' grade, plus pull down the grades of those they play.

Like you I was wondering about the grading points 'lost' when improving juniors leave the game. They are replaced by younger players taking up the game with much lower grades. This would seem to me to point to lead to a significant drop in average grade, but the effect is mainly in rapidplay as there aren't that many u11 juniors playing standard play tournaments.

Post Reply