Side effect of the countback system

General discussions about ratings.
Mike Gunn
Posts: 1014
Joined: Wed Apr 11, 2007 4:45 pm

Side effect of the countback system

Post by Mike Gunn » Mon Jan 30, 2012 7:34 am

Consider a player who plays 40 games a year. Because of the count-back method the 10 games he plays immediately before the end of the cut off period will be counted twice in calculating his grade (i.e. in January and July) whereas the other 10 games will be used just once. Is this what we want under the Clarke system? I can see one advantage is that for improving (or declining) players the most recent results are used, but is it the best system for the rest?

(Of course 40 is just used to make the point, but a similar effect will apply to anybody who isn't in category X.)

User avatar
Christopher Kreuzer
Posts: 8781
Joined: Fri Aug 06, 2010 2:34 am
Location: London

Re: Side effect of the countback system

Post by Christopher Kreuzer » Mon Jan 30, 2012 10:35 am

I think it would be more logical to go back in blocks of entire grading lists until you get above 30 games. So for your hypothetical example, you would end up with using 40 games in the July list instead of 30 games.

E Michael White
Posts: 1420
Joined: Fri Jun 01, 2007 6:31 pm

Re: Side effect of the countback system

Post by E Michael White » Mon Jan 30, 2012 10:36 am

The effect Mike Gunn mentions will speed up the overall deflation and spread of grades throughout the grading list and make the need for 2009 style rehashes more frequent. This may or may not be a price worth paying in achieving the advantages foreseen, when the current changes were made. Out of interest do you know what advantages were foreseen in making the current changes ?

Roger de Coverly
Posts: 21291
Joined: Tue Apr 15, 2008 2:51 pm

Re: Side effect of the countback system

Post by Roger de Coverly » Mon Jan 30, 2012 11:03 am

E Michael White wrote:The effect Mike Gunn mentions will speed up the overall deflation and spread of grades throughout
Can we be careful with terminology? Spread I understand to mean the effect whereby the top and bottom of the list move away from the middle.Thus top players would seem to be increasing their grades, whilst the bottom end would be reducing. But what is the definition of deflation please? Is it the movement downwards of the whole grading scale? If so, you would attempt to detect it by tracking the mean and median grades, adjusting if needed for changes in the overall mix of players included. The historic series of these values showed little movement notwithstanding the claims of a point a year for forty years.

Alex Holowczak
Posts: 9085
Joined: Sat May 30, 2009 5:18 pm
Location: Oldbury, Worcestershire
Contact:

Re: Side effect of the countback system

Post by Alex Holowczak » Mon Jan 30, 2012 11:08 am

The easiest way to get around this problem - if only by causing different problems! - is to switch from Clarke to Elo.

I guess you have to weigh up which is the less problematic system.

mleonard
Posts: 18
Joined: Mon Aug 10, 2009 8:41 pm

Re: Side effect of the countback system

Post by mleonard » Mon Jan 30, 2012 12:40 pm

Mike Gunn wrote:Consider a player who plays 40 games a year. Because of the count-back method the 10 games he plays immediately before the end of the cut off period will be counted twice in calculating his grade (i.e. in January and July) whereas the other 10 games will be used just once.
I'm assuming that your example player had 20 games before the cut-off and 20 after.
To keep it simple let's say he or she played 10 games in each of the 4 months Nov-Feb.

What you're saying is that the 10 games from December are included in the grading calculations both for the January list and for the July list? Can you explain why this would cause a problem? Is it because the new grade depends to some extent on the previous grade? Wasn't this already a factor (using the previous count-back method)?

I guess you're saying that the effect has been amplified by shortening the grading period.

User avatar
Christopher Kreuzer
Posts: 8781
Joined: Fri Aug 06, 2010 2:34 am
Location: London

Re: Side effect of the countback system

Post by Christopher Kreuzer » Mon Jan 30, 2012 1:10 pm

I think Mike is also saying that people previously active enough to avoid count-back effects are now having these effects appearing. Under the old system, those 40 games a year would have been enough for a completely new grade each year, with no lingering effects from previous years.

Mike Gunn
Posts: 1014
Joined: Wed Apr 11, 2007 4:45 pm

Re: Side effect of the countback system

Post by Mike Gunn » Mon Jan 30, 2012 1:28 pm

I am not saying it will create a problem (I am not sure if it does or not) but if your performance varies throughout the year (e.g. because of the different type of events that take place at different times of the year) then the grade calculated under the countback system will certainly be different compared to the old calculation. For somebody whose grade isn't changing much it will be more accurate to base the grade on a large sample of results, e.g. those from the last 12 month) (statistical argument).

Arguably just using the last 30 results will make grades less accurate.

Paul Cooksey

Re: Side effect of the countback system

Post by Paul Cooksey » Mon Jan 30, 2012 9:18 pm

I have not seen anything explaining the aims of the grading changes. I hope it will published.

I can see it makes sense to count back until a minimum is reached. But why not included whole grading periods until the number is >= 30? Surely a whole grading period is more accurate than a handful of games at the end of the period?

Roger de Coverly
Posts: 21291
Joined: Tue Apr 15, 2008 2:51 pm

Re: Side effect of the countback system

Post by Roger de Coverly » Mon Jan 30, 2012 9:59 pm

Paul Cooksey wrote:I have not seen anything explaining the aims of the grading changes. I hope it will published.
There's the report on the 2010 AGM at http://englishchess.org.uk/farthing/?paged=3. This is where the policy was approved. There's also discussion of the implications and attempted clarification of the changes in the Grading section of this forum.

There was also the pre-meeting proposal at
http://www.englishchess.org.uk/wp-conte ... sts-AR.doc

This contains the reasoning in favour of the proposal.

Paul Cooksey

Re: Side effect of the countback system

Post by Paul Cooksey » Mon Jan 30, 2012 10:08 pm

Thanks Roger. I had seen that proposal, I think the case for more frequent lists was made clearly

However I have not seen an explanation for the way it has been implemented. I question counting back exactly 30 games, how the junior grade calculation is done, and using unpublished grades in January and July. (Mainly the former because I think it creates unnecessary volatility, and the latter two because I think I ought to know the grade of a person I am playing)

Quite possible that these things were necessary, but it would be nice to know why.

Roger de Coverly
Posts: 21291
Joined: Tue Apr 15, 2008 2:51 pm

Re: Side effect of the countback system

Post by Roger de Coverly » Mon Jan 30, 2012 10:33 pm

Paul Cooksey wrote:However I have not seen an explanation for the way it has been implemented. I question counting back exactly 30 games, how the junior grade calculation is done, and using unpublished grades in January and July. (Mainly the former because I think it creates unnecessary volatility, and the latter two because I think I ought to know the grade of a person I am playing)

Quite possible that these things were necessary, but it would be nice to know why.
The 30 game count back was inherited from the way they do it for rapid-play grades. No-one complained, either because rapid-play is relatively homogeneous being mostly tournaments, fewer players play it, or no one really cares what their rapid-play grade is.

The junior calculation has been that way since August 2009 when it was brought in as a final fix to get the grading revaluation signed off. It wasn't that easy to reverse engineer the source of junior grades until this month with the detail analysis of not just your own results, but also all your opponents. I had struggled to see the source of Junior grades when analysing my 2009, 2010 and July 2011 results by event. I had missed the trick that the grade in the calculation is neither the start period grade, nor the end period grade. It's claimed that if you play a junior who has just the one game, that you always score your own grade. With the download available, I might try to find a practical example to validate this or otherwise.

(edit) There's an earlier discussion partly about count back at viewtopic.php?f=4&t=3252

In effect Council at the 2010 AGM handed the graders a blank cheque as far as methodology for the six monthly development. They used the method already employed for the rapid play grades. (/edit)
Last edited by Roger de Coverly on Tue Jan 31, 2012 1:03 am, edited 1 time in total.

Richard Bates
Posts: 3338
Joined: Fri Nov 14, 2008 8:27 pm

Re: Side effect of the countback system

Post by Richard Bates » Tue Jan 31, 2012 1:06 am

Christopher Kreuzer wrote:I think it would be more logical to go back in blocks of entire grading lists until you get above 30 games. So for your hypothetical example, you would end up with using 40 games in the July list instead of 30 games.
On the contrary i don't think this is at all a good idea, because it leads to perverse scenarios. Whether it is logical or not is another matter. The reason can be simply demonstrated by a not unlikely scenario as follows: suppose a player plays 30 games between July and December, performing at 140. They then play 29 games between January and, say, May, performing at 160. If they play no more games before the grading cut off then their grade will be 150 (and a little bit). But if they can just play one more game, against any opposition and with any result, their new grade will be substantially higher, because they will have reached the magic number of 30 and they can forget about the games played in the earlier period.

Once you have excluded the principle of operating a 'rolling list' (thereby maintaining the linkage of the grading calculation to a calendar season) then if you want to avoid the issue of games at the end of a period being potentially more important than games at the beginning of a period then the logical solution is to apply a weighting in line with the number of games needed to make it up to 30. So in the above example you would use the 29 games at a performance of 160 and 1 game at a performance of 140 (the average from the previous period).

ELO wouldn't work for ECF grades IMO.

Roger de Coverly
Posts: 21291
Joined: Tue Apr 15, 2008 2:51 pm

Re: Side effect of the countback system

Post by Roger de Coverly » Tue Jan 31, 2012 1:37 am

Richard Bates wrote: Once you have excluded the principle of operating a 'rolling list' (thereby maintaining the linkage of the grading calculation to a calendar season) then if you want to avoid the issue of games at the end of a period being potentially more important than games at the beginning of a period then the logical solution is to apply a weighting in line with the number of games needed to make it up to 30. So in the above example you would use the 29 games at a performance of 160 and 1 game at a performance of 140 (the average from the previous period).
That is the traditional theory as operated for many, many years on the annual lists. It's only when you have the detailed results by date that you can use a different method. It doesn't mean you should.

I know that the roll back method has been used for the rapid-play list. The standard play list is an amalgamation of results played under different conditions. At one extreme you have the 7 hour games (plus) in the 4NCL, the British and similar. At the other extreme you have 30 moves in 75 minutes followed by adjudication, or even all the moves in 60/61 minutes. Selecting a handful of end half-season results for double counting just doesn't feel right.

mleonard
Posts: 18
Joined: Mon Aug 10, 2009 8:41 pm

Re: Side effect of the countback system

Post by mleonard » Tue Jan 31, 2012 2:19 am

Richard Bates wrote:...suppose a player plays 30 games between July and December, performing at 140. They then play 29 games between January and, say, May, performing at 160. If they play no more games before the grading cut off then their grade will be 150 (and a little bit). But if they can just play one more game, against any opposition and with any result, their new grade will be substantially higher, because they will have reached the magic number of 30 and they can forget about the games played in the earlier period.
Out of interest does anyone know where this magic number of 30 games originated?

Also, I wonder why we cannot (starting from the next grading list publication) use data from the last 12 months to calculate the new grades? Sorry if this is already discussed elsewhere, but wouldn't it be possible to include game results from the two most recent (6 monthly) grading periods and not just the latest one?

Post Reply