Inflation in the international system

General discussions about ratings.
Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Inflation in the international system

Post by Roger de Coverly » Mon Jul 27, 2009 12:47 pm

An interesting article by Jeff Sonas in which he speculates about inflation in the international Elo system There's a particular reference to the 2200 and 2000 cut offs which applied for a number of years in the 1980s and 1990s.

http://www.chessbase.com/newsdetail.asp?newsid=5608

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: Inflation in the international system

Post by Roger de Coverly » Tue Jul 28, 2009 11:58 am

More generally, here is a piece by Nick Faulks on the philosophy behind rating systems.

http://www.fide.com/component/content/a ... ing-system
Last edited by Roger de Coverly on Wed Jul 29, 2009 11:33 pm, edited 1 time in total.

Tim Spanton
Posts: 1205
Joined: Thu May 01, 2008 11:35 am

Re: Inflation in the international system

Post by Tim Spanton » Tue Jul 28, 2009 12:16 pm

Both very interesting, especially Jeff Sonas's piece

Alex Holowczak
Posts: 9085
Joined: Sat May 30, 2009 5:18 pm
Location: Oldbury, Worcestershire

Re: Inflation in the international system

Post by Alex Holowczak » Tue Jul 28, 2009 12:30 pm

Nick Faulks wrote:The view was expressed in Athens that the Elo system, in conjunction with the current K-factor of 10, does not reflect recent performance closely enough to be the appropriate tool to decide selection into the 2010 candidates matches.
I agree with him completely on that point. Extending it, I think that using it as a minimum threshold for eligibility into tournaments as a net for those who haven't qualified through other means (such as the British Championship) is a bad idea. For both the World Championship, and the British Championship, I would be in favour of scrapping rating as a means of qualification, and find another goal-based way of letting players of that rating into it. Some examples:
  • Let all FIDE titled players qualify. This lets everyone in who has an active rating over 2350 based on this year's entry, plus some others. The advantage of this would be that the qualification is norm-based, rather than rating-based.
  • Encouraging more national Open events to have the status of a British Championship qualifier.
  • The highest placed non-qualifier from local League Individual Tournaments qualify.
Another issue with rating inflation will mean there is an increase in FIDE titled players. The fact that one of the criteria that makes you eligible for a title-norm is that you must have rating x is a bad thing if the ratings are inflating. It means more people will have a rating > x, and hence more titled players if they can get the norms.

E Michael White
Posts: 1420
Joined: Fri Jun 01, 2007 6:31 pm

Re: Inflation in the international system

Post by E Michael White » Tue Jul 28, 2009 11:23 pm

If we have inflation in FIDE ratings and deflation in the ECF ratings, Jeff Sonas should recalulate FIDE ratings on the ECF basis, take the average and hope the two effects cancel out.

Tim Spanton
Posts: 1205
Joined: Thu May 01, 2008 11:35 am

Re: Inflation in the international system

Post by Tim Spanton » Wed Jul 29, 2009 10:30 am

We don't have deflation in the ECF system - we allegedly have "stretching"

E Michael White
Posts: 1420
Joined: Fri Jun 01, 2007 6:31 pm

Re: Inflation in the international system

Post by E Michael White » Wed Jul 29, 2009 11:45 am

Thank you Roger for pointing out these two links.

Nick Faulkes has the right idea to challenge the underlying assumption that if the results of X v Y and Y v Z are known then the result of XvZ can be calculated. Some of his thought experiment is very similar to mine, which I posted on the ECF system eg considering the effect of large RR events, which show that rating systems cannot achieve what many of us expect them to do.

ELO did his best to reduce deflationary influences by using the K factor method which means, ignoring new starters, the total of all the A-Es in an event is 0, even when two players play in a different number of tournaments during the rating period. The ECF system does not achieve this.

Neither system however effectively controls spread ie the increasing difference between the lowest and highest players. The result is that FIDE appears to have spread with fewer deflationary effects and the combined effect appears as top level inflation while the lower players dropped out of the lists with the previous rating floors.

The ECF which doesnt control deflation as effectively, seems to have spread plus deflation which appears to be stability at the top level and deflation at the lower level.

Paul McKeown
Posts: 3732
Joined: Thu Apr 12, 2007 3:01 pm
Location: Hayes (Middx)

Re: Inflation in the international system

Post by Paul McKeown » Wed Jul 29, 2009 12:03 pm

I'm not convinced of the so-called "deflation" in the ECF grading system. Something of a mythical beast, that one, much spoken of, but rarely sighted. And as Tim said, it's polite to refer to it as "stretching".

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: Inflation in the international system

Post by Roger de Coverly » Wed Jul 29, 2009 12:19 pm

E Michael White wrote:Neither system however effectively controls spread ie the increasing difference between the lowest and highest players.
If you accept that the highest players get better and the the standard of the lowest (newcomers to competitive chess) is constant, then a widening spread between world championship contenders and raw beginners is both inevitable and desirable.
E Michael White wrote:The ECF which doesnt control deflation as effectively, seems to have spread plus deflation which appears to be stability at the top level and deflation at the lower level.
I continue to believe

(a) that the ECF were very wrong to wreck the historic continuity of the ECF grades
and
(b) that they still have yet to resolve the problems in the range (beginner to average).

My pet argument is that as the perceived problems (negative and very low grades) have only really emerged over the last decade is that we look at what has changed in the last decade. There've been two changes which may well be connected
(i) extension of the grading system into the territory of low standards - mostly young juniors
(ii) the abolition of the "grader's estimate" for new players in favour of a backwards estimate from actual results.

I believe the grader's estimate was acting as a counter-deflation device because it could help inject points into the bottom end of the system by acting as a "minimum grade for new players". Many on-line systems do it this way for new players - every new player gets an initial provisional rating which may be well above their actual strength.

I have an idea for Fide as well. Perhaps they could bolt on an English style performance measurement grade to get up to date rankings. So use and calculate the Elo ratings as normal as the "estimate" of player strength. Work out the performance grade ( average Elo of opposition plus/minus 400* (wins - losses). Publish it say every 6 months. Minimum standard for inclusion is (say) 15 games. The performance grade is only used for publication and ranking, the next period uses the usual Elo formula. If someone like Kramnik doesn't play 15 games in the measuring period, they just drop out of the list.

E Michael White
Posts: 1420
Joined: Fri Jun 01, 2007 6:31 pm

Re: Inflation in the international system

Post by E Michael White » Wed Jul 29, 2009 1:23 pm

Roger de Coverly wrote:I have an idea for Fide as well. Perhaps they could bolt on an English style performance measurement grade to get up to date rankings. So use and calculate the Elo ratings as normal as the "estimate" of player strength. Work out the performance grade ( average Elo of opposition plus/minus 400* (wins - losses). Publish it say every 6 months. Minimum standard for inclusion is (say) 15 games. The performance grade is only used for publication and ranking, the next period uses the usual Elo formula. If someone like Kramnik doesn't play 15 games in the measuring period, they just drop out of the list.
Maybe you dont remember but in the original ELO system if a player played 80 games in the rating period their rating was replaced by a current performance on those 80 games.

I am reminded during the prolonged debate on gradings on the ECForum of the Simpsons astronaut episode where a group was selected comprising a mathematician a different sort of mathematician and a statistician. The three main debating protagonists were probably Sean Hewitt who is a statistician, yourself and myself both mathematicians of sorts. Your approach seems to be to put forward theories and conjectures and then to collect evidence which you believe supports those views whereas my approach is to look at underlying mathematical structures and find systematic deflationary or inflationary features. I dont think we are going to agree on all points but as I have said before treating ungraded as 80 or 100 for the calculation of others grades is antideflationary. In addition it can be used as a control tap by varying the 80 points from one year to the next.

A feature of injected deflation not mentioned so far is that backward grading of ungraded players with a Markov process does not necessarily produce the correct figures. What happened around 1980/1990 was that this process was introduced alongside a drive by the ECF to grade many more juniors and junior rapidplay games were graded as longplay with this in mind. The effect was that many more juniors became graded at much younger ages and lower grades giving many years ahead of being rapidly improving active players who are a big cause of deflation.

I have a 1962 grading list which shows Keene as ungraded in 1961 and graded 5A (about 180 ECF) in 1962 presumably his first grade and likewise Hartston was first graded 5A (180) in 1963. They were probably around 15 when first graded and very few juniors U16 were graded. My first grade as a much older teenager was 5B or 169 so there were fewer years to cause junior based deflation. If you examine the grading download file now there are many juniors aged 8-15 with lower grades who can go on to contribute to deflation for many years.

Dont get sidetracked by deflation in your games this pm Roger.
Last edited by E Michael White on Wed Dec 16, 2009 9:56 am, edited 1 time in total.

E Michael White
Posts: 1420
Joined: Fri Jun 01, 2007 6:31 pm

Re: Inflation in the international system

Post by E Michael White » Thu Jul 30, 2009 11:06 pm

Roger

There is a danger of this Torquay event swamping the important things in chess life which are of course discussions on deflation in the ECF system; so I offer a few other thoughts for your and others consideration before play starts.
Roger de Coverly wrote:
E Michael White wrote:Neither system however effectively controls spread ie the increasing difference between the lowest and highest players.
If you accept that the highest players get better and the standard of the lowest (newcomers to competitive chess) is constant, then a widening spread between world championship contenders and raw beginners is both inevitable and desirable.
Well I might not accept all of this. The spread should be slowing as I expect beginners now enter the grading system at a stronger level than say 10 years ago as many seem to practise on the Internet before entering graded competitions or graded league club play. Predictions of spread returning made by Brian Valentine et al should take this into account. If it really slows up it might make Howard Grists changes look completely effective when we will be stuck with red grades for a few years. I would agree stronger players must be getting better. How could they get worse or standstill ? If minded we could calibrate the upper ECF levels by selecting 20 or so opening, middle and end game positions as a regression test bed and persuade a sample of players at various graded levels to play a Fritz specified version and level running on a specified GHz PC with specified OS version and repeat the exercise 3 years later with different players. But it might take the fun out of guessing instead.
Roger de Coverly wrote:I continue to believe

(a) that the ECF were very wrong to wreck the historic continuity of the ECF grades
and
(b) that they still have yet to resolve the problems in the range (beginner to average).
I would agree (a) but for different reasons; they should have made gradual corrections for spread and deflation over the years but abandoned the notion that grades could predict results accurately. But having allowed grades to get where they are is this a necessary step in the right direction ? Not in my view I would have tried to measure deflation/spread and discover all the influences before moving on it.

With (b) you have a problem, as does Brian Valentine’s analysis, in that you are measuring the statistical term “average” on a corrupted scale. A fundamental assumption of parametric statistics is that errors can be predicted and measured against a homogenous or exact scale for example metre lengths or events such as heads or tails. If this is not the case techniques such as analysis of variance and the central limit theorem simply do not work accurately. What you are actually measuring is the average of corrupted or stretched/deflated estimates of players strengths and comparing this with a subjective scale which no one can quantify. Thus all the statistical grade surveys you have read on this and other chess BBs are not as useful as they may appear. Having said that I would agree with most of (b) !

I also agree with a previous comment you made which was that more research and explanation for the changes were needed.

In the link you posted it shows that Mr Sonas has partially switched his efforts to a non parametric approach by considering the rankings or order of players eg the 100th best player. I expect this is a result of consideration of the scale which he believes is corrupted possibly non-uniformly by inflation. I don’t think FIDE have yet accepted the spread effect which is going to be difficult as they have been discarding data at the lower levels for many years. Mr Sonas recognises the increase in percentage of players graded as an issue but doesn’t as yet acknowledge the other corruptors such as player persistency ie players actively playing longer than previously or other changes to playing patterns.

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: Inflation in the international system

Post by Roger de Coverly » Fri Jul 31, 2009 11:20 am

E Michael White wrote:With (b) you have a problem, as does Brian Valentine’s analysis, in that you are measuring the statistical term “average” on a corrupted scale.
I'm just using the term "average" in a very general sense to just mean that grading range where there are as many published players above as below. As the distribution curve of graded players is approximately normal, this also corresponds to the greatest frequency. On the August 2008 actuals this was at 110. On the March 2009 red grades (now known to be incorrect) this was about 133. As the likes of Keith Arkell have lost a handful of points, the effect of the red grades is to move the "Arkell" level about 25-30 points closer to the "average".

Detail on the exact process used is a well kept secret, but the implication of some of the public statememts is that the red grades could be whatever they wanted them to be, since they started with the recursive process. It would have seemed ridiculous to take 25 points off the GMs to bring them down to 205-215 but that is the equivalent of what's been proposed.

At present, there's a gap of around 120 points between the weekend congress GM at around 230-235 and the "average player" at 110. The ECF grading team propose to reduce this to around 95 points. Do they have good reasons to do this?

To my mind, the big influx of young players was the expansion (of which I was part) in the late sixties and early seventies. Junior players actually entering the mainstream have been a trickle in recent years by comparison. It was observed of one congress that whilst players over 50 were plentiful and there were a few under 18s, that there was only one player in his twenties.

Brian Valentine
Posts: 574
Joined: Fri Apr 03, 2009 1:30 pm

Re: Inflation in the international system

Post by Brian Valentine » Mon Aug 03, 2009 5:49 pm

I would just like to point out that my analysis was based on the premis that the system was functioning correctly. Even in that situation stretch exists. I am more aware of the impurities in the system, because of the debates that are taking place here. These impurities will all contribute to stretch. I did use heroic assumptions on how big stretch would be in a pure system. I don't think the prediction of spread returning is weakened by identifying new impurities - quite the opposite. Also I don't think the concept of "average" is central to that analysis.

I was hoping a new rating list would be out by now so I could look at more of the various ideas in a robust way. I am very interested in the influence of leavers and joiners on the system, but only have the 2008 list in an electronic form. Obviously this list excludes leavers from the 2007 list.

I agree wholeheartedly with Roger on the system being debased by losing of historic continuity and don't think we should even consider gradual changes for perceived inflation/deflation until we are sure the system is suffering from such condition rather than a symptom of some other disease (as I believe - but cannot prove - is behind perceived stretch).