ECF to scrap its grading system

Debate directly related to English Chess Federation matters.
NickFaulks
Posts: 8453
Joined: Sat Jan 02, 2010 1:28 pm

Re: ECF to scrap its grading system

Post by NickFaulks » Fri Feb 08, 2019 7:19 pm

It sounds as though this newsletter contains interesting stuff. Where can it be found?
If you want a picture of the future, imagine a QR code stamped on a human face — forever.

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: ECF to scrap its grading system

Post by Roger de Coverly » Fri Feb 08, 2019 8:22 pm

NickFaulks wrote:
Fri Feb 08, 2019 7:19 pm
It sounds as though this newsletter contains interesting stuff. Where can it be found?
It is emailed.

Ian Thompson
Posts: 3551
Joined: Wed Jul 02, 2008 4:31 pm
Location: Awbridge, Hampshire

Re: ECF to scrap its grading system

Post by Ian Thompson » Sat Feb 09, 2019 7:05 pm

Roger de Coverly wrote:
Fri Feb 08, 2019 6:22 pm
The latest ECF newsletter announces a proposed implementation of January 2020. It means that those leagues, most of them, who ignore January grades, can at least complete their season.
What caught my eye was the question on the survey that said "Do you agree with the Board's decision to move to monthly grading?" So it's already been decided that it's going to happen, unlike the calculation method, where the question is "Do you agree with the Board's proposal to use a four-figure Elo style system?"

As a grader, I've received nothing asking if I'm both willing and able to provide monthly updates, which would seem to be essential to monthly grading being successful.

Angus French
Posts: 2151
Joined: Thu May 15, 2008 1:37 am

Re: ECF to scrap its grading system

Post by Angus French » Sat Feb 09, 2019 7:38 pm

Ian Thompson wrote:
Sat Feb 09, 2019 7:05 pm
Roger de Coverly wrote:
Fri Feb 08, 2019 6:22 pm
The latest ECF newsletter announces a proposed implementation of January 2020. It means that those leagues, most of them, who ignore January grades, can at least complete their season.
What caught my eye was the question on the survey that said "Do you agree with the Board's decision to move to monthly grading?" So it's already been decided that it's going to happen, unlike the calculation method, where the question is "Do you agree with the Board's proposal to use a four-figure Elo style system?"

As a grader, I've received nothing asking if I'm both willing and able to provide monthly updates, which would seem to be essential to monthly grading being successful.
Quite. It's the local results officers who do the majority of the work to produce grades. They ought to be consulted. When he was Director of Home Chess and responsible for grading, Alex Holowczak assured ECF Council this would happen:
Minutes of the April 2016 Finance Council Meeting wrote:Council was also reassured that we would not be making any changes to the current grading system itself until after leagues and congresses had been consulted with.

Nigel White
Posts: 126
Joined: Wed Oct 16, 2013 4:22 pm

Re: ECF to scrap its grading system

Post by Nigel White » Sun Feb 10, 2019 3:16 pm

While I normally try to refrain from being critical of volunteers who do their best in sometimes challenging circumstances, I really do think the consultation on grading changes circulated with the latest ECF newsletter is wholly inadequate. I've had experience of a number of consultation exercises across a range of sectors, and this one falls far short of the minimum standards that are required.

For a consultation exercise to be helpful, it should present the rationale for any proposed changes, together with a proper proposal on what is being considered, and seek views on the specific issues where there may be genuine options on how to proceed. As an ECF member (and, perhaps more importantly, as grader for the North Circular League), I have seen nothing about why the change is being considered and how it would be implemented. Without this context, asking whether I agree with the 'decision' to move to monthly grading is, in my opinion, meaningless. (In principle, I'm all in favour of more frequent grading, but whether it is sensible will depend on the details of how it will be implemented, about which I don't have a clue.) The question about the format of new grades is in my view almost cosmetic, compared with some of the details which are not addressed.

I will be responding to the consultation making these points.

Brian Valentine
Posts: 574
Joined: Fri Apr 03, 2009 1:30 pm

Re: ECF to scrap its grading system

Post by Brian Valentine » Sun Feb 10, 2019 5:16 pm

Nigel White wrote:
Sun Feb 10, 2019 3:16 pm
While I normally try to refrain from being critical of volunteers who do their best in sometimes challenging circumstances, I really do think the consultation on grading changes circulated with the latest ECF newsletter is wholly inadequate. I've had experience of a number of consultation exercises across a range of sectors, and this one falls far short of the minimum standards that are required.

For a consultation exercise to be helpful, it should present the rationale for any proposed changes, together with a proper proposal on what is being considered, and seek views on the specific issues where there may be genuine options on how to proceed. As an ECF member (and, perhaps more importantly, as grader for the North Circular League), I have seen nothing about why the change is being considered and how it would be implemented. Without this context, asking whether I agree with the 'decision' to move to monthly grading is, in my opinion, meaningless. (In principle, I'm all in favour of more frequent grading, but whether it is sensible will depend on the details of how it will be implemented, about which I don't have a clue.) The question about the format of new grades is in my view almost cosmetic, compared with some of the details which are not addressed.

I will be responding to the consultation making these points.
Now there are a few concrete markers about this change, I plan to start publishing some details in the near future.

Brian Valentine
Manager of ECF grading

Matt Chapman
Posts: 6
Joined: Mon Aug 10, 2009 2:57 pm

ECF Monthly Grading Consultation

Post by Matt Chapman » Mon Feb 11, 2019 1:51 pm

This could be a general discussion thread for the proposals to move to monthly gradings and 'a four-figure Elo style system', though I also have specific queries. If there is a document somewhere that answers my questions, please let me know.

First on the proposal to move to monthly grading lists. This makes sense as more up-to-date grades will always be better. But I wonder how this will be implemented in practice, e.g. for 'A' category grades, will games only be included from 12 months before each list (so grades will become Jan-Jan, then Feb-Feb, Mar-Mar etc.) or will some other method be used? Will all the categories retain the same requirements or would changes have to be made?

My bigger question/concern is on the move to a 'four-figure Elo style system'. It is unclear to me from the description in the ecf newsletter whether this is meant to mean retaining the ecf grading system, but multiplying everything by 10, or a change to a system like FIDE currently uses (and most national federations).

Personally I think the ECF system has a great advantage over FIDE and other elo-systems in that grades are much more responsive to recent performances. How many players are there that have FIDE ratings many years out of date, and no matter how many events they enter their rating seems to crawl at a snails pace, with the end result that the ratings seem rather bad at actually predicting results (though this may be perception more than reality). In the ECF system, at worst the games used are 36 months old, and in many cases grades are based entirely/mostly on the last 12 months, which is a much better indication of how strong a player is right now.

I would hope that, if we do end up moving to a different system, we can at least use some of the innovations that have come about in rating systems since Elo, which has become antiquated. For example, online websites like chess.com and lichess.org use Glicko-2 which has the useful feature of an uncertainty value on ratings. Such a value can be time-dependent, so that someone who has not played in many years has a high uncertainty, and their rating adjusts much more quickly as a result.

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: ECF Monthly Grading Consultation

Post by Roger de Coverly » Mon Feb 11, 2019 2:38 pm

As to what the ECF are proposing to do, I don't think they've decided.

If they keep the Clarke method, I'd think a better way of doing monthly grades is to work out what the grade would be if you played no more games until the next six-monthly list. That doesn't appear to be on the table as a potential solution, although it would appear to have only minimal software development costs.

Matt Chapman wrote:
Mon Feb 11, 2019 1:51 pm
Such a value can be time-dependent, so that someone who has not played in many years has a high uncertainty, and their rating adjusts much more quickly as a result.

Regarding Gliko, if the Australian experience of a few years ago is anything to go by, that's an undesirable feature in an over the board rating system. What can happen is that someone goes abroad say for long time and then gets an insanely high k factor when they return.

You also need confidence in the system. In other words a statement of which results have been included and enough information to enable an improbable grade or rating to be checked.

Angus French
Posts: 2151
Joined: Thu May 15, 2008 1:37 am

Re: ECF Monthly Grading Consultation

Post by Angus French » Mon Feb 11, 2019 3:11 pm

Can I suggest to the Mods that this topic is combined with this one: viewtopic.php?f=25&t=9988?

Brian Valentine
Posts: 574
Joined: Fri Apr 03, 2009 1:30 pm

Re: ECF to scrap its grading system

Post by Brian Valentine » Tue Feb 12, 2019 9:15 am

I have placed a proposal for monthly grading on the main ECF website under grading matters. I intend to begin an FAQ document today.

NickFaulks
Posts: 8453
Joined: Sat Jan 02, 2010 1:28 pm

Re: ECF to scrap its grading system

Post by NickFaulks » Tue Feb 12, 2019 10:11 am

Brian Valentine wrote:
Tue Feb 12, 2019 9:15 am
I have placed a proposal for monthly grading on the main ECF website under grading matters. I intend to begin an FAQ document today.
A very interesting document. Brian mentions, and it should be stressed, that this proposal is a reaction to problems that are peculiar to England and should be read in this light.

My eye fell on

"10. The k factor will be 20 except that in a month where a junior player has outperformed expectation, then the k factor will be 40".

If a historical analysis has shown that this would have worked well in English conditions then I suppose it is reasonable to give it a try, but prima facie it looks awfully like a systematic bias.
If you want a picture of the future, imagine a QR code stamped on a human face — forever.

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: ECF to scrap its grading system

Post by Roger de Coverly » Tue Feb 12, 2019 10:41 am

NickFaulks wrote:
Tue Feb 12, 2019 10:11 am
but prima facie it looks awfully like a systematic bias.
That's the point isn't it? National Elo systems have needed to apply various hacks to the original theory to deal with players who fail Elo's base assumption that "chess strength" was a stable if unknown attribute. The FIDE version has demonstrated that K=40 can rapidly get a junior player up to their perceived strength, but suffers from the problem that a bad result or two crashes the rating rapidly. So in terms of rankings against adult players, it's unstable.

NickFaulks
Posts: 8453
Joined: Sat Jan 02, 2010 1:28 pm

Re: ECF to scrap its grading system

Post by NickFaulks » Tue Feb 12, 2019 11:06 am

Roger de Coverly wrote:
Tue Feb 12, 2019 10:41 am
That's the point isn't it?
Probably, but the underlying assumptions are worthy of discussion.

In an environment where most rateable games go unreported, it may be necessary for the results of improving juniors which are reported to be turbocharged. If there is such a creature as a junior who is not improving, their rating will also be given an upward kick. Reasonable tradeoff?
If you want a picture of the future, imagine a QR code stamped on a human face — forever.

David Sedgwick
Posts: 5249
Joined: Mon Apr 09, 2007 5:56 pm
Location: Croydon

Re: ECF to scrap its grading system

Post by David Sedgwick » Tue Feb 12, 2019 11:07 am

NickFaulks wrote:
Tue Feb 12, 2019 10:11 am
A very interesting document. Brian mentions, and it should be stressed, that this proposal is a reaction to problems that are peculiar to England and should be read in this light.

My eye fell on

"10. The k factor will be 20 except that in a month where a junior player has outperformed expectation, then the k factor will be 40".

If a historical analysis has shown that this would have worked well in English conditions then I suppose it is reasonable to give it a try, but prima facie it looks awfully like a systematic bias.
Am I right in thinking that FIDE do the exact opposite?

I recall a case a year or so ago of an English junior who thought that he had qualified for the FM title, but had not done so as, under the FIDE Regulations, his k factor had been reduced from 40 to 26 in the month when he would otherwise have reached 2300.

NickFaulks
Posts: 8453
Joined: Sat Jan 02, 2010 1:28 pm

Re: ECF to scrap its grading system

Post by NickFaulks » Tue Feb 12, 2019 11:16 am

David Sedgwick wrote:
Tue Feb 12, 2019 11:07 am
Am I right in thinking that FIDE do the exact opposite?
Hardly the exact opposite. I have not gone back and checked this case but If I remember it correctly, his k factor was reduced not because his results were good but because he had so many games reported in that period that k=40 whould have caused overshooting. This could have been up or down.
If you want a picture of the future, imagine a QR code stamped on a human face — forever.