ECF Grading Proposals

General discussions about ratings.
Kevin Thurlow
Posts: 5821
Joined: Wed Apr 30, 2008 12:28 pm

Re: ECF Grading Proposals

Post by Kevin Thurlow » Sat Aug 31, 2019 3:26 pm

Good summary, John.

Good assessment of Brian, too.

User avatar
Paolo Casaschi
Posts: 1186
Joined: Thu Jan 08, 2009 6:46 am

Re: ECF Grading Proposals

Post by Paolo Casaschi » Sat Aug 31, 2019 6:46 pm

John Swain wrote:
Sat Aug 31, 2019 1:06 pm
juniors especially, but really all players, should be encouraged to enjoy chess for its own sake and not become obsessed by grades.
This should really go both ways: I completely agree we should not be obsessed by grades, therefore we should let the ECF thinker with the systems as they see fit, without worrying too much that under certain circumstances the system might possibly risk to turn worse than it is today...

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: ECF Grading Proposals

Post by Roger de Coverly » Sun Sep 08, 2019 7:34 pm

According to the ECF's latest newsletter, they are joining forces with the Welsh Chess Union.
Monthly grading update
ECF Chief Executive Mike Truran writes - We have had some very good discussions recently with the Welsh Chess Union, the outcome of which is that the two federations hope to be able to work together on implementing the planned new monthly grading system. The project is being led on the ECF side by Director of Membership Dave Thomas and Manager of Grading Brian Valentine, and we hope to have the specification and outline costings completed in the next few weeks.

User avatar
Chris Goodall
Posts: 1057
Joined: Sun Oct 10, 2010 6:40 pm
Contact:

Re: ECF Grading Proposals

Post by Chris Goodall » Mon Sep 16, 2019 9:20 pm

John Swain wrote:
Sat Aug 31, 2019 1:06 pm
Advantages or alleged advantages of monthly grading:

- online results are instant, after each game, so it makes sense for ECF grades to move in this direction (but why not go the whole hog and have the Yorkshire 'Chessnuts' daily experience?)
I've been wondering that. The great thing about monthly grading periods, is that it's now almost impossible to play more than 30 games in a single grading period. So for the sake of simplicity you can do away with the "weights" from Chessnuts and normalise everything to 30 games. And then every game is a zero-sum transaction of Clarke points (grade * 30) from one player to the other. So any games you played in the past become irrelevant to the calculation. If you know every player's current balance of Clarke points, you have everything you need to calculate their updated balance after a game is played, and hence their updated grade. It becomes an online algorithm, or a Markov process if you're mathy.

(I wonder whether there's a correction that could make it order-invariant - your grade is the same regardless of which way round you beat two players with different grades. A naive version would reward players who minimise their sum-of-progressive points balance, by beating the worse player first. As would the current system if you played only one game every 6 months. I suspect the correction involves taking the 30th root of a 30th power.)

Anyway, since you don't need to store any history in order to calculate the effect of a transaction, you could append that transaction to a ledger, record the updated grades of both players, and then forget about it. Bitcoin for chess players! Once a transaction (game) is in the ledger, it can't be removed (like a block in the blockchain). If it turns out you duplicated a game, you just append an equal and opposite transaction that gives the Clarke points back to their former owner. Or if you got the right game with the wrong result, say a game that you thought was won by White was actually won by Black, you subtract 100 points from White and give 100 to Black.

The ledger could just be plain text and kept in a public place like a Wiki server where people can write comments into it. Since all the calculations are public and you have version control through the software, you have complete transparency. Then you could build whatever competition pages or player pages you wanted by referencing rows in the ledger. You could anonymise the player tokens so a player who knows their own token can identify their own ledger entries, but not necessarily anyone else's.

There are details to work out, but a chess game doesn't require anything like the same security as a bitcoin transaction, right? And there have been maybe 10 million chess games in Britain, compared to 455 million bitcoin transactions? So provided ECF's move to monthly grading periods is a real thing, we could make a complete open-source fork of ECF grading with far less effort than a blockchain startup. Those monthly grading periods are the crucial factor - otherwise you can't normalise to 30 games and it all falls apart.
Donate to Sabrina's fundraiser at https://gofund.me/aeae42c7 to support victims of sexual abuse in the chess world.

Northumberland webmaster, Jesmond CC something-or-other. Views mine. Definitely below the Goodall Line.

David Robertson

Re: ECF Grading Proposals

Post by David Robertson » Mon Sep 16, 2019 11:01 pm

Suggest you draft this as a motion, and get your delegate to move it at Council. Could be fun

MartinCarpenter
Posts: 3044
Joined: Tue May 24, 2011 10:58 am

Re: ECF Grading Proposals

Post by MartinCarpenter » Tue Sep 17, 2019 8:50 am

Good luck coping with reassinging the date, geographic area etc of an incorrectly entered tournament once its stuck on an immutable chain :)

The weighting thing isn't easy. Cutting it to 30 games is slightly silly when some people are playing 50+ games a year - they're all data. The time based cut off that Chessnuts did didn't work that well for 10-15 a season though, as it meant that grades were incredibly volatile early season.

Kevin Thurlow
Posts: 5821
Joined: Wed Apr 30, 2008 12:28 pm

Re: ECF Grading Proposals

Post by Kevin Thurlow » Tue Sep 17, 2019 9:39 am

"Suggest you draft this as a motion, and get your delegate to move it at Council. Could be fun"

Brilliant! Except, in the old days it would be put late in the agenda to stop it being discussed...

User avatar
Chris Goodall
Posts: 1057
Joined: Sun Oct 10, 2010 6:40 pm
Contact:

Re: ECF Grading Proposals

Post by Chris Goodall » Tue Sep 17, 2019 10:00 am

MartinCarpenter wrote:
Tue Sep 17, 2019 8:50 am
Good luck coping with reassinging the date, geographic area etc of an incorrectly entered tournament once its stuck on an immutable chain :)
Don't need to. All that metadata wouldn't live in the ledger. You'd have a wiki page for the tournament that could hold whatever metadata you wanted, and it would reference rows in the ledger.
MartinCarpenter wrote:
Tue Sep 17, 2019 8:50 am
The weighting thing isn't easy. Cutting it to 30 games is slightly silly when some people are playing 50+ games a year - they're all data. The time based cut off that Chessnuts did didn't work that well for 10-15 a season though, as it meant that grades were incredibly volatile early season.
It isn't cut to 30 games. It's cut to 30 times your current grade, which is a number that encapsulates your entire history. It doesn't matter when you played the games.
Donate to Sabrina's fundraiser at https://gofund.me/aeae42c7 to support victims of sexual abuse in the chess world.

Northumberland webmaster, Jesmond CC something-or-other. Views mine. Definitely below the Goodall Line.

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: ECF Grading Proposals

Post by Roger de Coverly » Tue Sep 17, 2019 11:34 am

Chris Goodall wrote:
Tue Sep 17, 2019 10:00 am
It doesn't matter when you played the games.
Which is the reason it won't solve one of the major problems that grading and rating systems struggle with, namely a reliable treatment of players who have improved considerably.

User avatar
Chris Goodall
Posts: 1057
Joined: Sun Oct 10, 2010 6:40 pm
Contact:

Re: ECF Grading Proposals

Post by Chris Goodall » Tue Sep 17, 2019 1:29 pm

Roger de Coverly wrote:
Tue Sep 17, 2019 11:34 am
Chris Goodall wrote:
Tue Sep 17, 2019 10:00 am
It doesn't matter when you played the games.
Which is the reason it won't solve one of the major problems that grading and rating systems struggle with, namely a reliable treatment of players who have improved considerably.
The speedometer in your car shows you the speed you're going this instant. It doesn't show you the speed it thinks you'll be going once you've finished accelerating. Drivers seem to be okay with not having that information.
Donate to Sabrina's fundraiser at https://gofund.me/aeae42c7 to support victims of sexual abuse in the chess world.

Northumberland webmaster, Jesmond CC something-or-other. Views mine. Definitely below the Goodall Line.

User avatar
IM Jack Rudd
Posts: 4818
Joined: Tue Apr 17, 2007 1:13 am
Location: Bideford
Contact:

Re: ECF Grading Proposals

Post by IM Jack Rudd » Tue Sep 17, 2019 1:51 pm

Chris Goodall wrote:
Tue Sep 17, 2019 1:29 pm
Roger de Coverly wrote:
Tue Sep 17, 2019 11:34 am
Chris Goodall wrote:
Tue Sep 17, 2019 10:00 am
It doesn't matter when you played the games.
Which is the reason it won't solve one of the major problems that grading and rating systems struggle with, namely a reliable treatment of players who have improved considerably.
The speedometer in your car shows you the speed you're going this instant. It doesn't show you the speed it thinks you'll be going once you've finished accelerating. Drivers seem to be okay with not having that information.
That is in fact the point. The speedometer shows you the speed you're going this instant, not an average of that speed and the speed you had when you started accelerating.

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: ECF Grading Proposals

Post by Roger de Coverly » Tue Sep 17, 2019 2:20 pm

IM Jack Rudd wrote:
Tue Sep 17, 2019 1:51 pm
The speedometer shows you the speed you're going this instant, not an average of that speed and the speed you had when you started accelerating.
You would use the trip computer for material like average speed and average fuel consumption. This enables comparisons between cars and between journeys. That's what grades and ratings do, they enable comparisons between players as to the standard of their opposition and the quality of their results.

Mike Gunn
Posts: 1022
Joined: Wed Apr 11, 2007 4:45 pm

Re: ECF Grading Proposals

Post by Mike Gunn » Tue Sep 17, 2019 2:25 pm

If your grade was based on current speed then it would be based on just the last game played. Your grade would be even more volatile than when the ECF adopted the last 30 games played method.

User avatar
Chris Goodall
Posts: 1057
Joined: Sun Oct 10, 2010 6:40 pm
Contact:

Re: ECF Grading Proposals

Post by Chris Goodall » Tue Sep 17, 2019 3:09 pm

What we're talking about is just the overfitting problem, yes? Because having your historical data confiscated doesn't make you more likely to overfit that historical data. I would have thought the exact opposite was true! If you can boil the trend down to a single number that represents the improving-player uplift you need to apply to the grade, then you can also work backwards from the improving-player uplift to obtain all the information you need about the trend. So if that's what you wanted to do, you'd just store a second number in brackets next to the current grade, much like the weights on Chessnuts, and you still have an online algorithm. And then your improving-player uplift would also update in real-time rather than once a season.
Donate to Sabrina's fundraiser at https://gofund.me/aeae42c7 to support victims of sexual abuse in the chess world.

Northumberland webmaster, Jesmond CC something-or-other. Views mine. Definitely below the Goodall Line.

John Swain
Posts: 412
Joined: Sun Jun 20, 2010 10:35 pm
Location: Nottingham

Re: ECF Grading Proposals

Post by John Swain » Wed Sep 18, 2019 4:16 pm

Up thread, I asked a couple of times why the results of the ECF Members' Survey into possible changing to monthly and four-figure grading hadn't included the precise total sample size nor constituent sample sizes, but just the percentages and the rather vague "more than 900 responses".

I followed this up under the "Ask the Directors" system with a question to Mike Truran and Dave Thomas about the actual sample sizes. Dave Thomas helpfully provided the data and Mike Truran suggested that quoting from the data would be appropriate.

I can see now why it was simpler to state percentages alone. A small number of the 931 responses did not state their age (22); some (23) didn't answer the monthly question and a few more (30) didn't answer the four-digit question. But the number of incomplete responses don't alter the overall picture significantly.

What surprised me was that only 67 of the 909 responses where age was stated came from under 18s (7%). This contrasts with the ECF membership, as of 1 March, where 3,407 out of 11,580 members were juniors (29.4%):
https://www.englishchess.org.uk/wp-cont ... h-2019.pdf

So it is not the case, as I speculated earlier in the thread, that junior responses swamped the results; quite the contrary.

For the record, the numbers in the other age categories were: 18-30 (64); 31-49 (196); 50-64 (342); 65 or over (240).

As I commented earlier, it's not the Board's fault if they go to the trouble of trying to find out members' wishes and only 931 out of 11,580 (8%) bother to respond. Putting it another way, 655 of 11,580 members (5.7%) support monthly grading and 725 (6.3%) support four-digit grades; 253 oppose monthly grading (2.2%) and 176 oppose four-digit grading (1.5%). To borrow a phrase from Richard Nixon, most ECF members comprise "the great silent majority".

The move to monthly and four-digit grading seems to be proceeding; I wish Brian Valentine and others the best of luck in making both work with minimal inconvenience to graders and organisers.

Post Reply