Minimum Grades

General discussions about ratings.
Post Reply
Roger de Coverly
Posts: 21291
Joined: Tue Apr 15, 2008 2:51 pm

Minimum Grades

Post by Roger de Coverly » Mon Oct 29, 2012 11:55 am

The thread on membership outcomes has turned to the subject of Junior grades. As it stands at present, there is a floor of zero. The practical effect of this is to support grading levels in the 0 to 50 range. But suppose you created a standalone system exclusively for junior only events. To differentiate this from the adult system, add, say 500, so that zero becomes 500. You then have the potential to have grades of 450 being the players that lose to the 500s. This would give rankings to those currently "not good enough", but is it desirable? The problem coming being provided the players graded 450 improved, they could push the players they beat down to 400.

Frequent recalculation by treating most or all players as newcomers can presumably reduce the potential downward spiral, but also reduces the credibility that there's continuity from one year to the next.

It might be better to stay clear of Elo ranges as well, and add five thousand. The number after the 5 then becomes the potential adult equivalent, so a Junior grade of 5099 would be eligible for an under 100 tournament.

User avatar
John Upham
Posts: 7162
Joined: Wed Apr 04, 2007 10:29 am
Location: Cove, Hampshire, England.
Contact:

Re: Minimum Grades

Post by John Upham » Mon Oct 29, 2012 1:09 pm

The Berkshire Junior Chess Association seems to be able to operate their own grading system.

See http://www.bjca.org.uk/

Are you happy with it?
British Chess News : britishchessnews.com
Twitter: @BritishChess
Facebook: facebook.com/groups/britishchess :D

Alex Holowczak
Posts: 9085
Joined: Sat May 30, 2009 5:18 pm
Location: Oldbury, Worcestershire
Contact:

Re: Minimum Grades

Post by Alex Holowczak » Mon Oct 29, 2012 1:21 pm

Roger de Coverly wrote:Frequent recalculation by treating most or all players as newcomers can presumably reduce the potential downward spiral, but also reduces the credibility that there's continuity from one year to the next.
My understanding, allowing for the one or two misexplanations that may arise:

At the moment, juniors are recalculated every year, and hence treated as ungraded. This was done in the name of the grades being more accurate this way.

The issue is that the grades of adult players stay relatively constant throughout their life. So a published grade of 150, for an adult, means that their strength is probably about 150. The variance will tend to be quite low.

For juniors, this isn't true. If a junior is graded 50(ish), then they're improving more quickly than their grade can keep up with them. By using their published grade in the calculation, you'd be using a number for your grade calculation that's less than their actual ability. This was apparently the cause for the stretching; the juniors are always undergraded, and the adults are always playing juniors whose grades are too low. (The add-10 fudge for juniors was supposed to help this, but didn't fully solve the problem.)

Hence, iteration for junior players. Iteration gives a grading estimation before your grade is calculated, which is deemed to be more accurate than the published grade.

One of the issues, particularly for juniors, is that they tend to clump together towards the bottom of the list. If you look at the grades of players, there's a disproportionately high number of juniors at the bottom of the list, compared to the top. (This probably isn't groundbreaking news!) So if they only play junior-only chess, rather than adult, their grade may lag behind their standard of play, so when they do break into adult chess, the problem of lag re-arises if you don't use iteration.

There are a few questions:
(1) Have 6-monthly lists slowed that problem?
(2) What about new players; rather than just junior players? Should new players of any age be iterated? If yes, for how long?
(3) Has the iteration helped the accuracy?

These are questions the grading team will be working on, before too long.

Anyway, lunchtime. :)

Roger de Coverly
Posts: 21291
Joined: Tue Apr 15, 2008 2:51 pm

Re: Minimum Grades

Post by Roger de Coverly » Mon Oct 29, 2012 1:26 pm

John Upham wrote:The Berkshire Junior Chess Association seems to be able to operate their own grading system.

See http://www.bjca.org.uk/

Are you happy with it?
The parallel evidence of the top Under 18s are that the ratings are deflated relative to both ECF and FIDE systems. But that might be down to a paucity of results at the top and the memory effect of Elo style systems.

As to why they set up an independent system, that's better discussed under "Outcome of membership schemes".

Roger de Coverly
Posts: 21291
Joined: Tue Apr 15, 2008 2:51 pm

Re: Minimum Grades

Post by Roger de Coverly » Mon Oct 29, 2012 1:42 pm

Alex Holowczak wrote: This was apparently the cause for the stretching; the juniors are always undergraded, and the adults are always playing juniors whose grades are too low. (The add-10 fudge for juniors was supposed to help this, but didn't fully solve the problem.)
If you talk to the President, you will find that he did some research into this, when the BCF first went national with the grading. This was before the days when low standard junior chess was routinely graded, but when there were a lot more juniors taking part in adult events. His approach was to look at the distribution of grades by Union. What he uncovered was that the mean and median of SCCU grades was about 10 points higher than the rest of the country and also that they were increasing year on year. There were inconclusive debates about whether London or Southern players were better or just had inflated grades for some reason. This was in the days when new players were given a local estimate rather than using iteration.

Previously there had been a unilateral 10 point addition for all calculations involving juniors, so if you beat a junior, you counted their grade plus 60. This was replaced by a sliding scale based on age and for ease of calculation, this was added into the grades as published.

The negative and zero grade issue started to appear once graders' estimates were replaced by the iterative process. Graders' estimates would have provided a floor value even if inconsistent from one area or event to the next.

Alex Holowczak
Posts: 9085
Joined: Sat May 30, 2009 5:18 pm
Location: Oldbury, Worcestershire
Contact:

Re: Minimum Grades

Post by Alex Holowczak » Mon Oct 29, 2012 4:58 pm

Roger de Coverly wrote:The negative and zero grade issue started to appear once graders' estimates were replaced by the iterative process.
On what evidence are you suggesting this is true?

If national grading started before graders' estimates were succeeded by iteration, then it may be that this problem was always there, but undiscovered. Then, people only noticed it as being an issue at about the same time the iteration came in. Another possible explanation is that the grading of events with players down to 0 had been reached at about the same time as the iteration came in.

The point is, I guess, it won't be as simple a cause-effect relationship as you say it is. Well, I suppose it might be, but you've presented no evidence.

So this is all just conjecture, really.

Roger de Coverly
Posts: 21291
Joined: Tue Apr 15, 2008 2:51 pm

Re: Minimum Grades

Post by Roger de Coverly » Mon Oct 29, 2012 8:04 pm

Alex Holowczak wrote: So this is all just conjecture, really.

Not conjecture, memory. The batch system was being phased out around the period 2000 to 2002. The exact date can be detected from when detail results are available in the grading archive. It is around this time that the problem of negative grades starts to be mentioned and for that matter when the CCF started rating massive junior events.

From http://www.sccu.ndo.co.uk/0203/grad.htm, some comments by Richard H
You'll notice that most of the Rapid influx seems to have been Junior, and that its mean grade has gone down nearly as much as its activity has gone up. There will be a connection there, but the new system may also have something to do with it. It is said to lack the old one's tendency to overgrade new players.
Issues for the future
There's one in particular that is causing concern on the Grading team. Negative grades. When the scale was fixed, I don't know how many years ago, it wouldn't have occurred to anyone to grade U8 tournaments, or even U80, and the idea of a negative grade would have seemed laughable. It isn't now.
The problem has been around for years, but it has come to a head and we're past the point where we can ignore it and hope it will go away. It will only get worse, if the increase in junior grading continues. The new calculation routine will even accelerate the plunge, because it's more honest than the old one about very low grades.

Alex Holowczak
Posts: 9085
Joined: Sat May 30, 2009 5:18 pm
Location: Oldbury, Worcestershire
Contact:

Re: Minimum Grades

Post by Alex Holowczak » Mon Oct 29, 2012 8:36 pm

Roger de Coverly wrote:
Alex Holowczak wrote: So this is all just conjecture, really.

Not conjecture, memory. The batch system was being phased out around the period 2000 to 2002. The exact date can be detected from when detail results are available in the grading archive. It is around this time that the problem of negative grades starts to be mentioned and for that matter when the CCF started rating massive junior events.

From http://www.sccu.ndo.co.uk/0203/grad.htm, some comments by Richard H
You'll notice that most of the Rapid influx seems to have been Junior, and that its mean grade has gone down nearly as much as its activity has gone up. There will be a connection there, but the new system may also have something to do with it. It is said to lack the old one's tendency to overgrade new players.
Issues for the future
There's one in particular that is causing concern on the Grading team. Negative grades. When the scale was fixed, I don't know how many years ago, it wouldn't have occurred to anyone to grade U8 tournaments, or even U80, and the idea of a negative grade would have seemed laughable. It isn't now.
The problem has been around for years, but it has come to a head and we're past the point where we can ignore it and hope it will go away. It will only get worse, if the increase in junior grading continues. The new calculation routine will even accelerate the plunge, because it's more honest than the old one about very low grades.

Richard's comments were conjecture. In 2002/3 because he had no idea what the resultant overgrading would be like. He commented that it didn't overgrade juniors. If we accept that as true, then there's no evidence presented to suggest that this hastened the stretching. Indeed, for all we know, it might have accelerated or decelerated the stretching.

Richard offers a reason for what happened has having caused by the increasing tendency to grade junior-only tournaments, which is something I mentioned in the other thread. Intuitively, I can see the logic in the argument, but I don't know. If it was responsible for the stretching, then the measures taken in 2008 to recalibrate, and calculate differently, may have fixed the problem. It may not. We don't know until we look into it.

The way adults are graded relies on the assumption that a published grade is approximately their strength. I'm told that if an adult plays 30 games, there's a 95% confidence that the grade reflects their strength. (Using confidence intervals - I've not done the Maths myself, but enough have for me to assume that this is correct.) For a junior, this confidence is reduced, and so the initial assumption - that a published grade is approximately their strength - is a flawed one. Therefore, they are being treated differently in order for the system to work, and their grade is calculated by iteration in order to rise the confidence that the grade reflects their strength.

Roger de Coverly
Posts: 21291
Joined: Tue Apr 15, 2008 2:51 pm

Re: Minimum Grades

Post by Roger de Coverly » Mon Oct 29, 2012 8:51 pm

Alex Holowczak wrote: If we accept that as true, then there's no evidence presented to suggest that this hastened the stretching. Indeed, for all we know, it might have accelerated or decelerated the stretching.
As is perhaps known, I didn't and still don't particularly believe in the hypothesis of stretching, not least because the word inflation was always used. I particularly disagreed with the opinion expressed by one rash commentator that there had been 1 point per year deflation since the 1970s. Had that been the case, a 170s standard then, would have become a 130s standard in 2008. For that matter, the advocates of revaluation refused to look back into the Edwards investigations of the late 80s which had the mean and median of the grades in the high 120s.

It seems very obvious to me, that if you set a minimum grade, that the resulting system evolves differently to one without a minimum, or with a minimum only applied to new players.

Neill Cooper
Posts: 1298
Joined: Sun Feb 24, 2008 4:43 pm
Location: Cumbria
Contact:

Re: Minimum Grades

Post by Neill Cooper » Mon Oct 29, 2012 9:19 pm

I remember the problems in the early 2000s with junior rapidplay grades. My children had just started playing graded chess at CCF and there were some unrealistically high rapidplay grades. The problem in those days was that 0 was the minimum grade. The situations was quickly rectified with 0 being the minimum published grade. Most players with a published grade of 0 actually have a negative grade, which can be -20 or even -90. The system stores the more detailed information but the grade is published as 0.

Alex Holowczak
Posts: 9085
Joined: Sat May 30, 2009 5:18 pm
Location: Oldbury, Worcestershire
Contact:

Re: Minimum Grades

Post by Alex Holowczak » Mon Oct 29, 2012 9:38 pm

Roger de Coverly wrote:As is perhaps known, I didn't and still don't particularly believe in the hypothesis of stretching
Well, I have no information to go on, because I wasn't around at the time. So I have a choice of which camp to believe:
(1) People who have done lots of statistics to show that it is happening, or
(2) Someone who hasn't done any of those statistics.

Faced with that choice, I don't think it's irrational of me to side with (1) in this particular argument, until more evidence is available.
Roger de Coverly wrote:I particularly disagreed with the opinion expressed by one rash commentator that there had been 1 point per year deflation since the 1970s. Had that been the case, a 170s standard then, would have become a 130s standard in 2008.
If this is what I think it is, I've seen this particular work. When I was at secondary school, I would regularly play (and lose...) to someone whose father played chess in the 1970s and 80s, but he gave up for various non-chess related reasons. He was about 170 in that era, and kept up with the game during that time, without actually playing it. As his son was getting into it, he would say to me that exactly that had happened; someone who was 170 then was probably only about 130 now. His argument was different from yours. Someone of 170 standard then, and 130 standard now, was probably just as good on a relative basis as he was then. In terms of raw skill, a modern 170 was far better than a 170 in those days. This is purely anecdotal, of course. I wonder if others who have taken breaks in their play have noticed this?

Anyway, this would be what looks like deflation. x has a grade of 170 in 1970, and 130 in 2010. He's just as good as he ever was, skillwise. While this can't happen to everyone in the population, if all the weaker players dropped out of the system during that time - or at least, disproportionately more of them - then you could end up with the phenomenon you describe.

Someone saying such a phenomenon is rash doesn't actually make it any less likely to be true. If it were simple, we wouldn't be having this discussion. :)

Paul Dupré
Posts: 331
Joined: Fri Jul 11, 2008 11:04 pm
Location: Sutton, Surrey

Re: Minimum Grades

Post by Paul Dupré » Mon Oct 29, 2012 9:58 pm

Alex Holowczak wrote:If it were simple, we wouldn't be having this discussion. :)
No, most problems are simple.

It is already a fact that grading players out of the grading band 50-250 will create problems in the long term. Now that we are grading players less than 50, we should either add say 100 or possible 200. So, therefore make the grading band 50 to 450, increasing all current grades by 200, OR move to another grading system that works for all players including the very weak players that CCF grade so that they can increase their income.

My suggestion would be a total transfer to an ELO type system, like Scotland and Wales. Makes sense does it not.
Any postings on here represent the truth, and nothing but the truth, so help me God,
...and by the way the world is flat.

Alex Holowczak
Posts: 9085
Joined: Sat May 30, 2009 5:18 pm
Location: Oldbury, Worcestershire
Contact:

Re: Minimum Grades

Post by Alex Holowczak » Mon Oct 29, 2012 10:05 pm

Paul Cooksey wrote:
Alex Holowczak wrote:
Roger de Coverly wrote:As is perhaps known, I didn't and still don't particularly believe in the hypothesis of stretching
Well, I have no information to go on, because I wasn't around at the time. So I have a choice of which camp to believe:
(1) People who have done lots of statistics to show that it is happening, or
(2) Someone who hasn't done any of those statistics.

Faced with that choice, I don't think it's irrational of me to side with (1) in this particular argument, until more evidence is available.
The rest of us face the opposite dilemma, of having no evidence for stretching. Any chance of the statistical analysis being published?
We have the same dilemma; we have no information. I just chose to go with (1) when it came to forming an opinion.

I doubt we'll see the paper. Most of it was probably done by e-mail, phone, and unminuted face-to-face conversation, with no one really writing a formal paper. I only have so much time available, and there are other higher-priority things going on behind the scenes that I would rather spend that time doing. Getting people to search through five-year old e-mails, and putting them into a form ready for public digestion, isn't a sensible way of using the time I'm dedicating to Home Chess, or indeed the time of the people who did the work originally.

Alex Holowczak
Posts: 9085
Joined: Sat May 30, 2009 5:18 pm
Location: Oldbury, Worcestershire
Contact:

Re: Minimum Grades

Post by Alex Holowczak » Mon Oct 29, 2012 10:15 pm

Paul Dupré wrote:It is already a fact that grading players out of the grading band 50-250 will create problems in the long term. Now that we are grading players less than 50, we should either add say 100 or possible 200. So, therefore make the grading band 50 to 450, increasing all current grades by 200
I don't think you understand how the system works. Adding an arbitrary 200 points to everyone's grade will make no difference whatsoever. The problem for players graded below 50 is not the fact that the numbers are below 50 - it's that their behaviour/improvement is different from people who are about 150. An adult 150 is a pretty constant 150, whereas a player graded below 50 is likely to be improving rapidly; especially if they're a junior.
Paul Dupré wrote:move to another grading system that works for all players including the very weak players that CCF grade so that they can increase their income.
Leaving aside the unnecessary CCF-jibe, if you know of a grading system that works for all players, get in touch with the Manager for Grading. If it works when tried and tested, we'll use it.
Paul Dupré wrote:My suggestion would be a total transfer to an ELO type system, like Scotland and Wales. Makes sense does it not.
An ELO system? That's some Blue Sky thinking there! :D

The Elo rating system is not necessarily the answer to everything. The FIDE version of Elo suffers in England from having vastly underrated juniors. Domestically, it could be tidied up a little with various k factors. I think of the Clarke grading system as being a linear approximation of the Elo system, but with a slightly different method of calculation. They're based on the same fundamental principles. Therefore, they have similar flaws in their methodology.

If we're going to switch to Elo, which I don't have a major problem with, then we should make sure that the details are sorted in such a way that what we move to is an improvement on what we have at the moment. Some work would need to be done (simulations etc.) before coming to any conclusions, and deciding on whether or not it's worth it.

Roger de Coverly
Posts: 21291
Joined: Tue Apr 15, 2008 2:51 pm

Re: Minimum Grades

Post by Roger de Coverly » Mon Oct 29, 2012 10:40 pm

Paul Cooksey wrote:The rest of us face the opposite dilemma, of having no evidence for stretching. Any chance of the statistical analysis being published?
There never really was anything published that got to the bottom of matters and addressed sceptics. The grading team and those who supported the revaluation got it into their heads that grades needed to be recalculated and no amount of asking them why and what was the point would stop them. It didn't seem possible for the voting members on the ECF Council to stop them either. At the top end, GMs were 240 ish, IMs 220 ish and FMs 200 ish and it hadn't changed much over the years. So if you could score 1/4 against FMs and 3/4 against 150s that would maintain a 175 grade. So were GMs/IMs/FMs not getting credit for an improved standard of play?

Post Reply