A possible improvement to grading?

General discussions about ratings.
Robert Stokes
Posts: 119
Joined: Sun Mar 06, 2011 4:51 pm

A possible improvement to grading?

Post by Robert Stokes » Sat Mar 26, 2011 3:55 pm

I have read that a study of many grandmaster games showed about 40% were wins for white, 30% draws, and 30% wins for black. If this is correct, then it shows that white's advantage of the first move is not just inital but potentially lasting. That being so, should it not be reflected in grading? Should the bonus for winning as black be greater than the bonus for winning as white, etc?

Could it be something like this? (OG refers to opponent's grade)
Win playing black - OG plus 50
Win playing white - OG plus 40 or 45
Draw playing black - OG plus 5 or 10
Draw playing white - OG minus 5 or 10
Lose playing black - OG minus 40 or 45
Lose playing white - OG minus 50

You can argue about the exact differential but I would prefer opinions on the principle.

It may well be that graders would argue the slight improvement is not worth the extra complication but since it is all done by computer it would only need an entry of which colour a person was playing.

Geoff Chandler
Posts: 3486
Joined: Mon Jul 06, 2009 1:36 pm
Location: Under Cover

Re: A possible improvement to grading?

Post by Geoff Chandler » Sat Mar 26, 2011 4:47 pm

Hi Robert

(Live games going on, live football going on and this lad posts grading stuff...) ;)

The study is based on GM games, at the lower levels it equals out.
(weaker players tend to put more effort in booking up their Black's).

Also far too much hassle for the graders, TC's and league secs to
sort out the difference in colours.

You would also no doubt get the sharks fighting to get Black in
league matches v opponents they fancy they can beat.
(Some guys would sell their soul for a few points.)

Right David v D'Costa is warming up. (Dave chas swapped off on b5)
Look likes he has this position in mind.

Image

timlawson
Posts: 12
Joined: Sun Nov 14, 2010 11:44 pm
Location: Northampton

Re: A possible improvement to grading?

Post by timlawson » Sat Mar 26, 2011 5:43 pm

It's pretty obvious even to me that the existing grading system is skewed and cannot possibly work as an accurate mathematical model. However, to start tinkering this way would really just create more work and a hell of a lot of confusion!

Things have not changed since I was playing regularly 15 years ago and when I started playing again last season after a 5 year break, things had got so bad that the powers that be had to "adjust" most of the grades.

To add insult to the "skewed maths" statement, we also have grading officers. These guys n gals are humans and with the best will in the world, when you introduce humans into any equation, you get errors. Now, I'm not complaining about grading officers as they do a great job but to me, an ECF rating cannot really be trusted.

Fifteen years ago, I was playing chess on the internet and even the servers back then all had rating systems (based, usually, on an ELO or Glicko system). Clearly the technology is there to have one big computer system whereby you could input your own results. Presumably this would ensure a greater accuracy of data entered and reduce the huge workload on grading officers.

There would, naturally, still be a question of accuracy of data but with two parties entering results and the system clarifying that the results all match up, potential errors could be automatically highlighted.

There are benefits to this system, to name a few:

1) We could drop the utterly useless ECF rating system and "go ELO!" or "Glicko" - It would save me having to get the calculator out from time to time for sure.
2) Ratings would be absolutely bang up to date i.e your rating would be adjusted as soon as data is entered; which means things won't be skewed so much. For example, if someone loses to an improving junior at the end of the season at present this is not accurately reflected. Many juniors can put on 50 rating points over a season so that junior you played last week and lost to (ECF 90) might actually be ECF 150... in reality, their grade might only come out at ECF 135 or 140 which means under the present system a new grade can be out of date as soon as it's published.
3) It would save the ECF money.
4) It would allow more flexibility for the ECF to collect game fees as this could potentially be on a "block" purchase (let's call them ECF Credits) or on a "per click" basis (where the fee would be higher).
5) A system like this would undoubtedly have a lot more internet traffic and there is the potential there for the ECF to jump on "additional revenue streams". For example, it could act as a "one stop shop" for listing events/coaching/junior events etc etc. I know from experience that finding out EXACTLY what's going on in the chess circuit for anything from junior events to open tournaments can be an arduous task.

So, in summary - tinker with the existing system? Not a good idea in my opinion. Nuke the existing system and come up with something more modern, accurate and user friendly? A great idea but it would probably require some sort of funding. But it might be worth looking into - there may well be savings that could be made so it might even pay for itself.
Tim Lawson

User avatar
Sebastian Stone
Posts: 153
Joined: Sat Apr 25, 2009 4:21 pm

Re: A possible improvement to grading?

Post by Sebastian Stone » Sat Mar 26, 2011 7:25 pm

Robert Stokes wrote:I have read that a study of many grandmaster games showed about 40% were wins for white, 30% draws, and 30% wins for black. If this is correct, then it shows that white's advantage of the first move is not just inital but potentially lasting. That being so, should it not be reflected in grading? Should the bonus for winning as black be greater than the bonus for winning as white, etc?

Could it be something like this? (OG refers to opponent's grade)
Win playing black - OG plus 50
Win playing white - OG plus 40 or 45
Draw playing black - OG plus 5 or 10
Draw playing white - OG minus 5 or 10
Lose playing black - OG minus 40 or 45
Lose playing white - OG minus 50

You can argue about the exact differential but I would prefer opinions on the principle.

It may well be that graders would argue the slight improvement is not worth the extra complication but since it is all done by computer it would only need an entry of which colour a person was playing.
With a win percentage of 55% this means white has a +5 grading advantage.

Thus,

Win playing black - OG plus 55
Win playing white - OG plus 45
Draw playing black - OG plus 5
Draw playing white - OG minus 5
Lose playing black - OG minus 45
Lose playing white - OG minus 55

I'm not to sure whether the grading results the ECF receive include which colour the players were.

However, I don't think it's going to make much difference apart from people who play one colour a large amount more than another.

As for people abusing the system, people are going to try and abuse any grading system so it's not an argument against this one.
AKA Scott Stone

"Give a man fire and he's warm for a day, set fire to him and he's warm for the rest of his life."

That's Mr Stone to you, f**kface.

Arshad Ali
Posts: 704
Joined: Mon Feb 22, 2010 12:27 pm

Re: A possible improvement to grading?

Post by Arshad Ali » Sun Mar 27, 2011 5:15 am

The win percentage of 55% translates to a 35-point rating difference (the last I heard, admittedly some years back). So if I'm a 2335 player as black, I should get a 50% score against a 2300 player. Statistically it evens out in the long run (I play as many games as white as I do black), so no particular reason to tinker with the rating formula (i.e., my win with black against a 2300 player should not get as many rating points as against a 2335 player). In any case, the difference in points awarded or lost would be small.

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: A possible improvement to grading?

Post by Roger de Coverly » Sun Mar 27, 2011 9:03 am

Sebastian Stone wrote:'m not to sure whether the grading results the ECF receive include which colour the players were.
The collection of results for English grading or rating includes the facility to indicate who had white and who had black. As it presently stands, it's an optional field which may or may not be implemented and input by the designers and users of data collection systems. So there's a data issue which is a potential problem for anyone wanting to research the practical effects.
The international FIDE data now includes colour, again I don't know whether it is mandatory.

Back in the 1990s, the PCA ratings for top players (which was an Elo style system) did reflect colour.

If you had the data, a possible line of experiment would be a recalculation of grades or ratings with three values for each player , firstly their results with white, secondly their results with black, and finally their combined results. You might expect, at the higher levels anyway, a higher grade or rating with white. Whether this results in the ranking order for players being different is unknown.

Increasingly tournament organisers are calculating grading or rating prizes, not by reference to the actual score by those of a rating group, but the best performance. There's a case of sorts that equal performances by players with three Blacks and two Whites are more meritorious than with three Whites and two Blacks.

E Michael White
Posts: 1420
Joined: Fri Jun 01, 2007 6:31 pm

Re: A possible improvement to grading?

Post by E Michael White » Sun Mar 27, 2011 10:20 am

If there is a need for grades to reflect the colour it should be done by calculating a grade with White and a grade with Black and then averaging the two, rather than tinkering with the +/- 50. In the event that a player has a small number of games or none with one colour an approximate adjustment to the grade needs to be made based on historic evidence.

timlawson
Posts: 12
Joined: Sun Nov 14, 2010 11:44 pm
Location: Northampton

Re: A possible improvement to grading?

Post by timlawson » Sun Mar 27, 2011 11:58 am

E Michael White wrote:If there is a need for grades to reflect the colour it should be done by calculating a grade with White and a grade with Black and then averaging the two, rather than tinkering with the +/- 50. In the event that a player has a small number of games or none with one colour an approximate adjustment to the grade needs to be made based on historic evidence.
In doing it this way, rather than simply averaging the two, would you need to apportion according to how many games had been played with each colour?: eg: average rating with white: 150 (20 games), average rating with black: 140 (10 games) would mean you could only take 1/3 of the difference between the two grades so rather than having a grade of 145, you would end up with a grade of 146.66 recurring or 146.7 if rounding up or 147 if rounding to the nearest integer.

I think that using "approximate" adjustments and basing things on "historic" evidence is a surefire way to distort things even further. In my case, this would be a bit like "pinning the rating on the patzer"! Actually, my results with black are better than with white (I seem to get a lot more games as black). This is probably due to my complete incompetence in the opening/middle and endgame. My guess is it's easier to confuse someone who has the white pieces?! The last question here is "what to do with players who have few games and NO historical data?" - who makes the decision on "approximate" adjustments. I can see some players getting confused!

The best method is to admit that the existing system is not robust enough. Will this happen? My guess is that would be a bit like an alcoholic to admit they have a problem in the first place. Personally, I'm still in denial myself. The solution is to scrap ECF ratings and use an ELO or Glicko system where ratings are all up to date because results are entered "on the go" and therefore a lot more accurate (including when rounding fractions). There are always going to be issues over ratings where not much data is available, eg a small sample of games only and that's where the Glicko system is useful because it also gives you a "rating deviation" based on the number of games over a set period. This then gives you a rating with an RD. If you play frequently, the RD will be quite low (around the 30's) which would mean a 1600 rated player with an RD of 30 has a rating range of 1540 and 1660 (I think the RD is doubled then applied but I might be wrong and was too lazy to check this up when writing this post!) I think this is the method that the Australian's use?

At present, to be fair to the ECF system, it DOES give an indication of the relative strength of most players. Ratings for juniors are obviously distorted (as grades are only published ever 6 or 12 months) and juniors improve and under the current system, with the recent "adjustment" it will take 3-4 years for standard play ratings to "settle down". Some players will be able to cope with their shiny new rating and others won't!
Tim Lawson

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: A possible improvement to grading?

Post by Roger de Coverly » Sun Mar 27, 2011 4:20 pm

timlawson wrote: The best method is to admit that the existing system is not robust enough. Will this happen? My guess is that would be a bit like an alcoholic to admit they have a problem in the first place. Personally, I'm still in denial myself. The solution is to scrap ECF ratings and use an ELO or Glicko system where ratings are all up to date because results are entered "on the go" and therefore a lot more accurate (including when rounding fractions). There are always going to be issues over ratings where not much data is available, eg a small sample of games only and that's where the Glicko system is useful because it also gives you a "rating deviation" based on the number of games over a set period. This then gives you a rating with an RD. If you play frequently, the RD will be quite low (around the 30's) which would mean a 1600 rated player with an RD of 30 has a rating range of 1540 and 1660 (I think the RD is doubled then applied but I might be wrong and was too lazy to check this up when writing this post!) I think this is the method that the Australian's use?

Some points.

I don't think a real time system of rating as used on chess servers can ever work for over the board chess. A couple of simple practicalities. Firstly if you need to rank people for board order, pairing or eligibility purposes, you cannot have the ratings change every day. So you want a list which is fixed for, say, at least a month. Secondly you cannot have a system where a delay of the input of a single game or event can get the calculations out of sequence and require them to be reworked.

The underlying statistical basis for grading, rating and ranking systems is that it enables players to be placed in order of strength , using results as a proxy for strength. I think it can be shown that you need about 30 games to have confidence that if A's grade exceeds B's by 8 ECF points, then A is the "stronger" player.

Elo methods are tried and tested in many chess federations. Glicko on the other hand has a fatal flaw when used in national rating systems. Glicko makes the assumption that just because you didn't happen to play in events subject to its rating system, that it means you have become an inactive player and there can be little confidence in your rating. General reasoning, to my mind, says that's rubbish unless you have world-wide proof of inactivity.

timlawson
Posts: 12
Joined: Sun Nov 14, 2010 11:44 pm
Location: Northampton

Re: A possible improvement to grading?

Post by timlawson » Sun Mar 27, 2011 7:11 pm

You make some good points Roger. My feeling is that there is a consensus amongst many chess players that the current grading system isn't the best but for the reasons you point out, it would be nigh on impossible to have a perfect system.

Since the existing system DOES work after a fashion, it's probably best left alone. Perhaps one day the ideal system will emerge, who knows?!
Tim Lawson

E Michael White
Posts: 1420
Joined: Fri Jun 01, 2007 6:31 pm

Re: A possible improvement to grading?

Post by E Michael White » Sun Mar 27, 2011 9:26 pm

The choice of a basis for any grading system needs to have in mind the purposes for which grades are to be used. I suspect players want to use current ECF grades :- a) to determine board orders in League/County matches b) determine the draw for tournaments c) for players to be able to set themselves personal targets d) for junior or other selection criteria. Any new or present system may not cover all these purposes equally well but the object should be to reduce the variance of any estimates arising from the use of the grades for those purposes..

The current ECF basis does not allow for colour and therefore does not make the best estimate of board order where league/county rules require the order of teams to be declared prior to the toss being made and colours being chosen. You stated that you play better with black than you might expect based on global historic averages ie the 51- 55 % figures quoted by various writers for success with White pieces, which may mean once the toss has been decided you should perhaps be a board higher or a board lower. Calculating grades with White and Black and then averaging the two reduces the likelihood of inequity except where there are insufficient games to calculate both grades, when an approximation is needed.
timlawson wrote:In doing it this way, rather than simply averaging the two, would you need to apportion according to how many games had been played with each colour?: eg: average rating with white: 150 (20 games), average rating with black: 140 (10 games) would mean you could only take 1/3 of the difference between the two grades so rather than having a grade of 145, you would end up with a grade of 146.66 recurring or 146.7 if rounding up or 147 if rounding to the nearest integer.
Your calculation brings you back to the present ECF grade calculation.
timlawson wrote: think that using "approximate" adjustments and basing things on "historic" evidence is a surefire way to distort things even further. In my case, this would be a bit like "pinning the rating on the patzer"! Actually, my results with black are better than with white (I seem to get a lot more games as black). This is probably due to my complete incompetence in the opening/middle and endgame. My guess is it's easier to confuse someone who has the white pieces?! The last question here is "what to do with players who have few games and NO historical data?" - who makes the decision on "approximate" adjustments. I can see some players getting confused!
There are a number of ways to deal with insufficient data. For example with very few games use the existing ECF formula. With enough to calculate only one colours grade, do that and take off from the White grade a fixed number of points or add on a fixed number to the black grade.
timlawson wrote:…. and that's where the Glicko system is useful because it also gives you a "rating deviation" based on the number of games over a set period. This then gives you a rating with an RD. If you play frequently, the RD will be quite low (around the 30's) which would mean a 1600 rated player with an RD of 30 has a rating range of 1540 and 1660
Glicko and FIDE rating bases are suitable in different circumstances but like the ECF basis do not have their footing firmly in scientific probability theory.
FIDE is basically a reward system which tots up past performances and FIDE hope gives a reliable indicator of how a player will perform in their next tournament and whether they have attained a title level.

If Glicko tells you your grade is ECF 118 to 132 how does this help a county captain determined where you play in relation to a player whose grade is determined by Glicko as 124-126 ?

If ECF grades were used for Junior selection, the present ECF grade make no allowance for colour, so it would seem inappropriate to choose one player over another if by chance the second scored similarly but had a much higher percentage of Black piece games against the same level of opposition.

Geoff Chandler
Posts: 3486
Joined: Mon Jul 06, 2009 1:36 pm
Location: Under Cover

Re: A possible improvement to grading?

Post by Geoff Chandler » Sun Mar 27, 2011 9:57 pm

Grades...Grades...Grades.

Why not have a grade for white and a grade for Black.
A grade for league games, a grade for tournaments.
A Sunday morning grade and a last round grade.
A grade for friendly games a grade for games v Grandma.

I'm with the OP. Scrap the whole lot......and just have classes.

1st class (top) 20th Class (bottom)

To advance in a class you must beat 10 players in a higher class.
Once up one class, your score reverts to 0 and you must win another 10.
(losses/draws don't count and you can never drop back to a lower class).

Advatages:
Players will play as much and as often as they can without fear of
a losing to a lower graded player. It's win or bust.

Grade gimps will not play bland chess chasing the few points they get
from drawing with a higher graded player.
Infact the lower classed player has everything to gain chasing a win and
the higher classed player has nothing to lose going for win.

Under the current system a player actually gains points because he
never won and a player can actually lose points because he never lost.
This is utter nonsense. Maths gone mad.

The class system is much easier to implement.
None of this crap about curves, expected ratio's, junior adjustments...
What a lot of tosh. It's a game of chess, not rocket science.

User avatar
Sebastian Stone
Posts: 153
Joined: Sat Apr 25, 2009 4:21 pm

Re: A possible improvement to grading?

Post by Sebastian Stone » Sun Mar 27, 2011 10:39 pm

Geoff Chandler wrote: Grade gimps will not play bland chess chasing the few points they get
from drawing with a higher graded player.
Hey! Some of my best games are bland draws against higher rated players.
AKA Scott Stone

"Give a man fire and he's warm for a day, set fire to him and he's warm for the rest of his life."

That's Mr Stone to you, f**kface.

Geoff Chandler
Posts: 3486
Joined: Mon Jul 06, 2009 1:36 pm
Location: Under Cover

Re: A possible improvement to grading?

Post by Geoff Chandler » Sun Mar 27, 2011 10:46 pm

Then you should be ashamed of yourself.

You did not win the points, you obtained them via a
mathmatical formula invented by some nut who
hated chess and wanted to kill off the game.

Contact your grader and get him to remove the paltry
few points you gained.

Give them to charity.

User avatar
Sebastian Stone
Posts: 153
Joined: Sat Apr 25, 2009 4:21 pm

Re: A possible improvement to grading?

Post by Sebastian Stone » Sun Mar 27, 2011 10:52 pm

Geoff Chandler wrote:Then you should be ashamed of yourself.

You did not win the points, you obtained them via a
mathmatical formula invented by some nut who
hated chess and wanted to kill off the game.

Contact your grader and get him to remove the paltry
few points you gained.

Give them to charity.
Err...If I took away all the points I gained from draws this season my grading performance would go up. :shock:

Yes, I am one of those sad losers who keeps track of it.
AKA Scott Stone

"Give a man fire and he's warm for a day, set fire to him and he's warm for the rest of his life."

That's Mr Stone to you, f**kface.