ECF to scrap its grading system

Debate directly related to English Chess Federation matters.
J T Melsom
Posts: 1294
Joined: Wed Aug 12, 2009 11:12 pm

Re: ECF to scrap its grading system

Post by J T Melsom » Mon Dec 10, 2018 10:01 am

I note my use of the word 'robustness' was picked up earlier in the thread. I am not a statistician so if it annoys those who are I apologise. I'm using it as a contrast to 'volatility' or similar.

David Williams
Posts: 336
Joined: Sun Mar 29, 2009 8:37 pm

Re: ECF to scrap its grading system

Post by David Williams » Mon Dec 10, 2018 10:26 am

I may be being a bit dense here.

Working out the grade of someone who plays more than thirty games a year on December 31, using the current method. Whether you collect the data annually, half-yearly or hourly, the calculation is based on all the games played in the previous twelve months. The more frequently the data is collected and grades calculated, the more up to date the opponents' grades used in the calculation.

I just can't see how doing the calculations more frequently makes the method unsound. If anything it removes one of its flaws.

MartinCarpenter
Posts: 3044
Joined: Tue May 24, 2011 10:58 am

Re: ECF to scrap its grading system

Post by MartinCarpenter » Mon Dec 10, 2018 11:06 am

For people playing 30+ games a year, its absolutely fine :) If they only play say 10-15? Much trickier. You either allow games 2-3 years old, which is somewhat against the 'idea' of such frequent grading lists or accept very significant randomness.

The Yorkshire grades for really active (50+ year) players were essentially absolutely static.

The 'proper' answer to kids asking for a grade based on a few games is of course that the grade isn't worth the paper its written on without a decent number of games behind it. Especially not for someone who is inevitably initially improving at huge speed. Rather tough love of course!

Kevin Thurlow
Posts: 5821
Joined: Wed Apr 30, 2008 12:28 pm

Re: ECF to scrap its grading system

Post by Kevin Thurlow » Mon Dec 10, 2018 11:20 am

Part of the problem is that a lot of people put too much store in grades. When Richard Clarke started the system, he told people not to take it too seriously - it was a measure of past performance, not an accurate measure of ability. This is why he used bands of grades, e.g. 1a, 1b, 2a, 2b etc., warning that it didn't mean that a 1a player was better than a 1b player. I'm sure Elo thought the same way.

Alison Bexfield
Posts: 23
Joined: Mon Nov 02, 2015 12:30 pm

Re: ECF to scrap its grading system

Post by Alison Bexfield » Mon Dec 10, 2018 11:25 am

Martin Carpenter: I suspect we have different views of what a grade should be. I think the current grade should be a result of the most recent information about a player's current playing strength. I would not expect it to be 'certificate of strength', statistically based, unless the person has played a significant number of graded games. But that does not make an estimated grade based on a handful of games worthless. Indeed players emerge with a tournament performance grade from playing a single six round tournament. This is useful information on their current strength.

An estimated grade can be given for someone as soon as they have played a handful of games. If they play more this becomes refined and less volatile, but it is still helpful information. It enables the player to pick appropriate opponents and target improvement. We tend to talk about juniors but it also applies to adults.

NickFaulks
Posts: 8453
Joined: Sat Jan 02, 2010 1:28 pm

Re: ECF to scrap its grading system

Post by NickFaulks » Mon Dec 10, 2018 11:48 am

Alison Bexfield wrote:
Mon Dec 10, 2018 9:32 am
I understand that I have up days and down days with my play. I don't mind my grade reflecting that.
Alison,

Whether you mind is not the only point, or even perhaps the most important one. Your opponents are entitled to expect that your published grade is a fair estimate of the strength of play they might expect from you, since it affects their own grade.

This is not a comment on which approach best achieves that.
If you want a picture of the future, imagine a QR code stamped on a human face — forever.

J T Melsom
Posts: 1294
Joined: Wed Aug 12, 2009 11:12 pm

Re: ECF to scrap its grading system

Post by J T Melsom » Mon Dec 10, 2018 12:14 pm

Strikes me that there are many things in life that get tested and feed-back comes in a variety of ways and at a variety of speeds. Exam results and university places are confirmed much more slowly. Good feed-back on chess performance isn't just a number at all, and if the number is at all useful in an informal sense then let people produce it themselves - does it actually need a formal stamp of legitimacy? I sense these numbers are more attractive to the tiger parent than the child. It all seems a bit Gradgrind to me. That said if the tests in my earlier post can be satisfied then no problem - just be clear why we want this change.

Alison Bexfield
Posts: 23
Joined: Mon Nov 02, 2015 12:30 pm

Re: ECF to scrap its grading system

Post by Alison Bexfield » Mon Dec 10, 2018 12:25 pm

Nick,

You say: "Your opponents are entitled to expect that your published grade is a fair estimate of the strength of play they might expect from you, since it affects their own grade."

I agree.
Far better to have more current information influenced by recent play available to see, rather than an out of date version. Some information, even if based on a small number of recent games, is better than no information because no lists are out yet or deemed insufficient games. In Go tournaments, a new player (eg an adult who has only played online and not over the board games) tries their first tournament. They enter at an estimated level based on a chat with the organiser as to what level they may be playing at. They play three games and the European grading programme (akin to the FIDE one) calculates them a grade which is up on the site in the next week. It is plain for all to see that it is based on only three games but it gives an indication of the level of play of that person. As they play more the grade adjusts. Why wait six months to give this information out?

Nick Grey
Posts: 1838
Joined: Thu Dec 01, 2011 12:16 am

Re: ECF to scrap its grading system

Post by Nick Grey » Mon Dec 10, 2018 2:05 pm

Nick if we have current info for all moving to monthly will have an effect on all.
It may be useful for opponents to know x is on a run of 17/20 against an average opponent of 130 or 160 or 200. But the cost of collection & support to
do this does not come cheap in terms of systems, monies & volunteer hours/weeks so why bother. It is meaningless & not worth the cost of production.
Clarke - measure of past performance & not to be taken seriously. I'm not expecting any child to have a grip of statistics. My first secondary school lesson - There are lies, damned lies, and statistics!

I have many interesting discussions with the Education & Skills Funding Agency on formula funding, funding in schools being a post code lottery, not fair funding, nor being based on the characteristics of pupils. Or based on outcome measures. As we point out to them there are no outcome measures -low prior low attainment & a government constraint - no school can lose more than 1.5 per pupil funding each year - meaning it will take about 35 years to hit their new formula. That's without any cost inflation over those years. Pointless but a significant issue for how schools are funded - many juniors in our schools.

NickFaulks
Posts: 8453
Joined: Sat Jan 02, 2010 1:28 pm

Re: ECF to scrap its grading system

Post by NickFaulks » Mon Dec 10, 2018 4:45 pm

Nick Grey wrote:
Mon Dec 10, 2018 2:05 pm
Clarke - measure of past performance & not to be taken seriously.
But what else can you measure?
If you want a picture of the future, imagine a QR code stamped on a human face — forever.

MartinCarpenter
Posts: 3044
Joined: Tue May 24, 2011 10:58 am

Re: ECF to scrap its grading system

Post by MartinCarpenter » Mon Dec 10, 2018 5:45 pm

Alison Bexfield wrote:
Mon Dec 10, 2018 12:25 pm
Far better to have more current information influenced by recent play available to see, rather than an out of date version.
The idea of out of date information is very dodgy for chess players. What loads of chess players do - and this is fairly trivial to verify - is improve for maybe 5(?) years then hit a level and stick at that basically for the rest of their careers (40+ years worth of games.).

All the local variations within that are then driven by a combination of real life & luck, and those vary so fast that they don't really predict near term expected future performance.

Honestly 3 years/30 games could be made quite a lot more conservative for the bulk of club level chess players, but you do need to compromise for juniors/new players etc.
Alison Bexfield wrote:
Mon Dec 10, 2018 12:25 pm
Some information, even if based on a small number of recent games, is better than no information because no lists are out yet or deemed insufficient games.
Until people have played a few games there simply isn't any information to be had. There's a near random number. If you think it'll encourage people then you can stick it up publicly somewhere, but you have to accept what it is :)

You very definitely can't then use those grades in any official capacity, or for calculating other peoples grades.

In general I really do think its better to only stick up grades once they're good enough to be vaguely trustworthy. We're arguably a bit slow to do that at the moment.

Alison Bexfield
Posts: 23
Joined: Mon Nov 02, 2015 12:30 pm

Re: ECF to scrap its grading system

Post by Alison Bexfield » Mon Dec 10, 2018 7:10 pm

Martin
I don't buy this concept of perfect grades because individual performance does vary over time for all sorts of reasons. There will never be perfect games where people play every time exactly to predicted grades. They are distracted by all sorts of life events and changes in motivation and from their opponents also, as you rightly say. But there are probabilities of this happening which become higher when grades are based on more information. I thought the ECF system already took this into account by adding a letter to the grade which gave an indication of its likely robustness.

You say "You very definitely can't then use those grades in any official capacity, or for calculating other peoples grades." And it is this I disagree with. When one of my club members plays in a 6 round tournament one can calculate a performance grade for that tournament. This can be used in a semi official way to determine which section is most appropriate in the next event where sections are grade determined. I can also use it as a base for calculating approximate grades for other as yet ungraded members in my club by comparing where I rank them against that player. Regarding their effect on others in the grading system, this problem is simply fixed by giving much less weighting to games against people with less 'robust' grades.
As soon as someone has played a handful of games there is information on playing level, especially if the handful of games included some wins and losses.

I have made far too many posts on this topic and will bow out now.

MartinCarpenter
Posts: 3044
Joined: Tue May 24, 2011 10:58 am

Re: ECF to scrap its grading system

Post by MartinCarpenter » Mon Dec 10, 2018 7:58 pm

Very sensible - I'll add one thing here. It definitely isn't me after perfect grades. I'd be very happy with the original letter style vague things.

Trying to micro track local fluctuations worries me a bit more (a lot more conceptually except I don't care much!) than whatever we do to give new starters numbers.

User avatar
JustinHorton
Posts: 10364
Joined: Mon Aug 04, 2008 10:06 am
Location: Somewhere you're not

Re: ECF to scrap its grading system

Post by JustinHorton » Mon Dec 10, 2018 9:00 pm

Alison Bexfield wrote:
Mon Dec 10, 2018 7:10 pm

I have made far too many posts on this topic and will bow out now.
Your total number of posts: 19

Mine: 5692 before this one.

Reckon you can do a couple more if you want.
"Do you play chess?"
"Yes, but I prefer a game with a better chance of cheating."

lostontime.blogspot.com

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: ECF to scrap its grading system

Post by Roger de Coverly » Mon Dec 10, 2018 9:17 pm

David Sedgwick wrote:
Fri Dec 07, 2018 8:40 am
The 100th Counties Championships Finals on Saturday 13th July 2019 may be the last event which is Clarke graded.

On past performance, the grades will come out as usual in the last week of July 2019. That leaves a week for the new monthly super system to produce its first results.

It's more than likely nothing will be produced, or if it is, no-one will have much confidence in the results. Any new system can be safely ignored, by Leagues anyway, for at least another six months.

If the real interest was in generating grade estimates for new players and juniors, rather than random noise for established ones, I would have thought the existing new player and junior estimation routines could be spun monthly or even weekly. What they produced would be only for information as in use in tournament eligibility, Swiss pairings and board orders, rather than having any impact on grades of established players. These changes could be held back until the six monthly cut off. If leagues and clubs were able to report more frequently than six-monthly, these results would be incorporated as well.