New Grades

Request amendments or pass comments on this free service.
Edward Tandi
Posts: 82
Joined: Sun Jun 15, 2008 11:29 pm
Location: London, UK

Re: New Grades

Post by Edward Tandi » Mon Sep 01, 2008 3:50 pm

Wow, the 50th percentile, which used to be 110 is now raised to 135.

More interestingly, the distribution curve is more peaky -see the diagram (attached):
Attachments
distribution.png
Distribution plots for old and new grades
distribution.png (9.32 KiB) Viewed 4221 times

Sean Hewitt

Re: New Grades

Post by Sean Hewitt » Mon Sep 01, 2008 3:54 pm

David Robertson wrote:the new grades seem to reflect, not one's own specific performance, but the prior performance of one's opponents and the extent of their exposure to deflated juniors.
Isn't that obvious? Scoring 70% against a field averaging 130 isnt as meritorious as 40% against a 190 field. If the field that you have been playing against has been vastly under-rated then your own performance has been compromised as a result.

Steve Lloyd
Posts: 35
Joined: Sat Aug 30, 2008 12:22 am

Re: New Grades

Post by Steve Lloyd » Mon Sep 01, 2008 9:30 pm

These new grades are certainly interesting. My grade rightfully went down to 114 from 117 after a poor season, but under the new system I've come out as 130. I've had a pretty successful season so far, so I decided to work out my grade using the classic +50 -50 method, based on my opponents' new grades. It came to 151. I very much doubt it will be this high come the end of the season but still there is no way I am a 150 player. I don't really consider myself to be a 130 player unless I choose to believe 130 is the new 110.

I assume congress organisers will need to re think the rating bands for each section? The local Frodsham congress runs sections U90, U130 and U170. I scored just 2 in the U130. If nothing changes I would be expected to enter the U170.

My own random search found a junior 10 years old with a grade of 109 in the old system. Under the new system he comes out at an incredible 170! This 170 overtakes some of better U12s. What's more, is that if this kid joined my club he would be the second best player (according to ratings). I'm not sure I would want to risk a 10 year old on board 2!

Ian Thompson
Posts: 2673
Joined: Wed Jul 02, 2008 4:31 pm
Location: Awbridge, Hampshire

Re: New Grades

Post by Ian Thompson » Mon Sep 01, 2008 10:09 pm

If the method used to calculate the new grades is sound then the new grades are an accurate reflection of players' relative strengths. This means that the old grades are not an accurate reflection of players' relative strengths.

It is quite common for leagues to require teams to play in descending order of current playing standard (which need not be descending grading order). This means team captains should use the new grades as the starting point for determining board order, as they are accurate and the old ones are not. I wonder how many team captains and leagues will do this? Will the ECF enforce this in its own competitions? The County Championship, National Club Championship and Schools Championship all have a 'descending order of current playing standard' rule. If the ECF does not enforce this, what does that say about their confidence in the new grades?

Ian Thompson

John Upham
Posts: 5056
Joined: Wed Apr 04, 2007 10:29 am
Location: Cove, Hampshire, England.
Contact:

Re: New Grades

Post by John Upham » Mon Sep 01, 2008 11:51 pm

I've always advocated being able to play teams in order of playing strenght rather than strict rating order. Indeed, there has been a previous thread on this very topic.

I've encountered many dyed in the wool persons who won't consider this as a valid principle.

As the saying goes...."There are people on the pitch, they think its all over : it is now!"

I'm predicting people suddenly advocating playing teams in order of playing strength having seen the new ratings.

We had a good laugh at one of my clubs tonight with the thought of some of the bunnies being propelled about the board order just because they are juniors and therefore improving. There is no evidence they are improving apart from their new laughable ratings.

The talented juniors have rightly gone up such as Felix Ynojosa and James Adair.

We have someone who is totally hopeless and still cannot castle after two years help. Their rating was zero (0) and now its 60!

Crazy...

Still, taking rating points off them will be like shooting fish in a bucket!

John
Last edited by John Upham on Tue Sep 02, 2008 12:15 am, edited 2 times in total.
British Chess News : britishchessnews.com
Twitter: @BritishChess
Facebook: facebook.com/groups/britishchess :D

User avatar
Carl Hibbard
Posts: 5864
Joined: Fri Dec 08, 2006 8:05 pm
Location: Evesham

Re: New Grades

Post by Carl Hibbard » Tue Sep 02, 2008 12:12 am

My own grade at 164 -177 is about right since I have not moved from that in the last perhaps 15 years

It is always in that range :(
Cheers
Carl Hibbard

James Courtenay
Posts: 11
Joined: Mon May 21, 2007 8:46 pm
Location: Southend, Essex

Re: New Grades

Post by James Courtenay » Tue Sep 02, 2008 1:02 am

A quick look at the easily downloadable Excel spreadsheet shows that there are 11,765 chess players of which 10,262 have a standard play grade.

Taking just these standard grade players....

Unfortunately the spreadsheet does not show the number of games played, only the category of the grades, however there are:

Cat A - 1,911
Cat B - 1,647
Cat C - 2,449
Cat D - 2,433
Cat E - 1,821

If we take the least number of games it is possible for these people to have played in the last year (cat A 30, B 20, C 10, D 5 and E 1) I would calculate the number of results received (which would be twice the number of games played) as:

57,330
32,940
24,490
12,165
1,821
--------
128,746 (and I am sure this is a gross under estimation, as the top 100 players circa 5% of the cat As had over 11,000 results on their own)

so there were (at least) 64,323 games of chess played last year.

How anyone can then say that the new grading calculations are rubbish based on looking at less than 1% (and in some cases less than 0.1%) of these results is beyond me. If you take the most relevant 1% of anything to your argument, you'll "prove" anything you want, and with the most relevant 0.1% you might even have such a convincing argument you'll make people go WOW.

This compared to the ECF bods that have analysed all the 64,323+ games, I would be so bold as to suggest they might just have a better idea.

And then to suggest you can comment on the grade of 8 juniors having played each of them once well….!!

James.
James.

Roger de Coverly
Posts: 19263
Joined: Tue Apr 15, 2008 2:51 pm

Re: New Grades

Post by Roger de Coverly » Tue Sep 02, 2008 8:49 am

How anyone can then say that the new grading calculations are rubbish based on looking at less than 1% (and in some cases less than 0.1%) of these results is beyond me.
I don't think the critics are looking at results, they are looking at outcomes. The absolute level of a grade can be anything you want it to be and they've chosen the level so that "over 210" is virtually unchanged. What has changed are the relativities. So long established relationships that A is higher rated than B who is higher rated than C have been turned on their heads. Players who could happily play in rating restricted tournaments may now need to take on the elite in opens. There will be little point in running under 75 events because there aren't any players.

With only a couple of days since publication, much of the analysis will be superficial. One point that is evident is the calculations for juniors contain a cook factor. in other words, the published grade is the result of the usual +/-50 calculations plus the cook. There has always been a cook factor and it's a measure to reflect the probability that a young player is improving and is stronger at the end of the season than the beginning. It looks as if the cook factor is a lot more aggressive (higher) than it used to be, So what may be happening is that players who play a lot of juniors (which includes other juniors) are being awarded bonus points which results in their grades being or appearing to be overstated.

If I've understood what's happened correctly, then the parallel new results for 2005-6 and 2006-7 also exist. I think these need to be published so that individual calculations for 2007-8 can be audited (ie reproduced on a spreadsheet). In that way we'll at least see where we gained a shedload of grading points.

David Robertson
Posts: 2270
Joined: Tue Apr 03, 2007 6:24 pm
Contact:

Re: New Grades

Post by David Robertson » Tue Sep 02, 2008 8:55 am

James Courtenay wrote:128,746 (and I am sure this is a gross under estimation, as the top 100 players circa 5% of the cat As had over 11,000 results on their own) so there were (at least) 64,323 games of chess played last year.

How anyone can then say that the new grading calculations are rubbish based on looking at less than 1% (and in some cases less than 0.1%) of these results is beyond me. If you take the most relevant 1% of anything to your argument, you'll "prove" anything you want, and with the most relevant 0.1% you might even have such a convincing argument you'll make people go WOW.

This compared to the ECF bods that have analysed all the 64,323+ games, I would be so bold as to suggest they might just have a better idea. And then to suggest you can comment on the grade of 8 juniors having played each of them once well….!!
Before I begin, James, may I be the first to congratulate you on your own 'great leap forward' :) A grading improvement of over 300% provides an obvious cause for satisfaction, albeit at the expense of dulled critical sensibilities.

But to the main point. No one, myself included, have said the new grading calculations are rubbish. A number of us have pointed to examples that give rise to scepticism. And we report an outbreak of mirth in many clubs as players, hitherto displaying barely passing acquaintance with the game, now possess relatively elevated grades. In some cases I can cite, this does not appear to be rooted in any evidence of performance.

One can of course draw reliable conclusions from small samples; opinion polls manage this most of the time. But no one in this discussion has claimed statistical reliability for their comments. They have simply provided a little illustration of some hard-to-explain changes. For my part, I gave examples from eight juniors I'd played very much in that manner: namely, to suggest junior grades seem over-inflated. Believe me, an experienced player can make a pretty accurate assessment of playing strength from the evidence of one game.

I suspect most of us will in the end shrug our shoulders, taking the rough with the smooth, and accept the statistics of the 'ECF bods'. How things pan out from now on will determine the ultimate level of public confidence in the new calculations. I'd have more confidence in yours if you'd managed accurately to divide 128,746 in half :)

David
Atticus CC

Edward Tandi
Posts: 82
Joined: Sun Jun 15, 2008 11:29 pm
Location: London, UK

Re: New Grades

Post by Edward Tandi » Tue Sep 02, 2008 9:14 am

James Courtenay wrote:Unfortunately the spreadsheet does not show the number of games played...
For what it's worth, the CSV file does contain the number of games considered in the calculation:

Code: Select all

Cat   Games         Players    Per-Player (rounded)
A:    90325         1911        47
B:    39688         1647        24
C:    36079         2449        15
D:    24394         2433        10
E:    10190         1821        6
O:    783           1503        1
Total: 201459       11764
Note: The number includes a small percentage of games from previous seasons.

Yes, there appear to be some anomalies, but I expect it will be ironed out by next year, when we start using the (updated) new grade. Of course, I'd like to know how next year's grade will be calculated, will it use the new grade?

User avatar
David Shepherd
Posts: 867
Joined: Fri Nov 23, 2007 3:46 pm

Re: New Grades

Post by David Shepherd » Tue Sep 02, 2008 10:09 am

My understanding is that a juniors grade is intended to represent their average playing strength for the next season (as I believe any yearly increase is added after the grading calculations are done). Thus the grade for longplay would be an estimate of their grade in 6 months time, (assuming a linear increase in playing strength) and not their grade for performances based on last year (which is the case for adults). This probably explains partly why the junior grades appear high as their playing strength for last year could be say 15-20 points below that for the current year.

I think however that they are slightly high for a number of reasons, but probably at lot nearer the mark than the old grades.

Sean Hewitt

Re: New Grades

Post by Sean Hewitt » Tue Sep 02, 2008 10:26 am

Edward Tandi wrote:Of course, I'd like to know how next year's grade will be calculated, will it use the new grade?
If I've read the page on the ECF grading section correctly, the answer is that next years grades will be calculated purely using the "new" grades.

James Courtenay
Posts: 11
Joined: Mon May 21, 2007 8:46 pm
Location: Southend, Essex

Re: New Grades

Post by James Courtenay » Tue Sep 02, 2008 10:31 am

David Robertson wrote:

Before I begin, James, may I be the first to congratulate you on your own 'great leap forward' A grading improvement of over 300% provides an obvious cause for satisfaction, albeit at the expense of dulled critical sensibilities.
And before I continue, David, may I commiserate you on the fact you had the lowest increase (jointly) of anyone on your old grade. A grading improvement of distinctly less than the average provides an obvious cause for dissatisfaction, "albeit at the expense of dulled critical sensibilities." - Couldn't have phrased it better myself.

But, before moving on to more sensible and less personal comments, I am happy to concede that yours is certainly bigger than mine :wink:

David Robertson wrote: Believe me, an experienced player can make a pretty accurate assessment of playing strength from the evidence of one game.
David Robertson wrote: * Rd 10 - 15 year old (131); new grade 164. Too high (draw, dull)
Explain that then? You stat your old grade 167 and new as 170. If person drew against you how can you say they are not due that grade? And if you tell me it is because you made a mistake, or played badly, then fair enough, but that then means no-one "can make a pretty accurate assessment of playing strength" from one game, because having looked at your history you are obviously not a 131 (old grade) strengh player.

David Robertson wrote: I suspect most of us will in the end shrug our shoulders, taking the rough with the smooth, and accept the statistics of the 'ECF bods'. How things pan out from now on will determine the ultimate level of public confidence in the new calculations. I'd have more confidence in yours if you'd managed accurately to divide 128,746 in half :)
And I'd have more confidence in yours if you used more than 8 games against 8 different people to pass comment on! But we can all make mistakes Professor! :wink:


But as you say, we will have to see how things pan out... unfortunately, even with the best will in the world, I don't suppose the ECF bods can please everyone!

James.
James.

Ian Gilbert
Posts: 1
Joined: Tue Sep 02, 2008 5:48 pm

Re: New Grades

Post by Ian Gilbert » Tue Sep 02, 2008 11:29 pm

I have to say I'm astonished that the ECF is pressing ahead with this, I thought it would take them twenty more years to admit there was a problem and fifty to come up with a solution. All credit to the people who've done the work on this.

I believe Howard Grist stated somewhere on the other thread that he's run last year's results through using the new method and the new grades matched player performance far more closely than the old distorted grades.

Unless anybody can come up with evidence that this isn't true, I'd say that was case closed.

The point of grading is to rank players and determine relative playing strength. There is no way to say what it *realy* means to be a 150 strength player, and if being graded 160 rather than 150 is causing anyone an existential crisis I would suggest that it's their problem rather than the ECF's.

On the one hand, Howard and Sean have posted solid mathmatical evidence which makes sense. Firstly there is clear evidence that the old grades are poor indicators of relative playing strength, and hence a need to be recalculated. Secondly the method of iteration used seems to be entirely scientific and thoroughly justified. Thirdly, The recalculated new grades have been proven to be a more accurate predictor of results than the old ones. (That hypothisis can of cause be tested again during the course of this season)

On the other hand what the detractors have said is boils down to 'you see my mate, I don't think he's really 150 strength, I played him in 2006 and you know what, blah blah blah...'

As it is, any time a new list comes out people look at it and say 'he's not really that good' or 'she's better than that' etc. etc. etc. because results don't always totally reflect actual ability. It's just the same with these...

Roger de Coverly
Posts: 19263
Joined: Tue Apr 15, 2008 2:51 pm

Re: New Grades

Post by Roger de Coverly » Wed Sep 03, 2008 12:17 am

The point of grading is to rank players and determine relative playing strength.
Agreed.

Are the "new" grades doing this though?

The junior adjustments seem to be factoring in improvements which haven't happened yet.

If I've understood the process correctly, parallel grades have been calculated for 2008 (published) , 2007 and 2006 (unpublished). The sceptics need to see the results for 2006 and 2007 to demonstrate that rampant inflation has not been permanently injected into the system.

By way of demonstration we can note that FM/IM Jack has an unchanged grade of 215 on 93 games. That's not alone, most of the top 100 are similarly unchanged or show only minor variations. On the current system his 3 year history is 218 214 215 - in other words almost static. What is the parallel sequence on the "new" grades? If it's 218 214 215 or equivalent then there's no inflation. If however it's 205 210 215 or similar then there's built in inflation and the absolute level has been cooked for 2008 to get a "no change" result.

Locked