ECF grades compared with FIDE ratings

General discussions about grading.
Post Reply
Brian Valentine
Posts: 385
Joined: Fri Apr 03, 2009 1:30 pm

ECF grades compared with FIDE ratings

Post by Brian Valentine » Sat Feb 25, 2012 9:52 pm

I have had a look at the relationship of the January ECF grades with the January 2012 FIDE ratings. Where there is a match with both an Elo rating and an ECF grade the relationship is roughly 7*ECF + 785 = FIDE.

The traditional relationship (post the unstretching exercise) is given as 8*ECF +650 = FIDE. The statistics are such that this relationship would be rejected. (The standard deviation of the gradient error is only .07). Where the existing formula is used (e.g. 4NCL), there are grounds for review. However the situation is murkier than the bald statistics suggest.

I don’t know how stable this relationship has been over time. However looking underneath the numbers brings out some notable features.

The first point I noted was that if juniors were excluded, the local seniors (“ENG”) underperformed on FIDE compared with foreigners (not “ENG”). On average the foreigners come out 15 FIDE points higher i.e. 7*ECF + 800. Closer examination suggests that the over-performance is mainly up to around 150-160 ECF and the relationships are fairly similar thereafter.

I then looked at juniors. There is a problem in that the age given on the ECF list does not always tie in with the year of birth given on the FIDE list. Whatever way one decides to sort this out, and to be honest I haven’t tried too hard, there is a glaring anomaly.

Foreign juniors fit 7*ECF + 740, whereas ENG juniors fit 6.5*ECF + 770. Given average grade for these youngsters is around 180, the English are 50-60 Elo points behind for the same ECF. The ENG fit here is not particularly good.

I have some concerns about the method for calculating junior ECF grades. However they are probably OK when used in the averaging process for this purpose. FIDE methods are more unreliable for improving players, but seem to be working OK on the foreign juniors playing in ECF graded events. There does seem to be an issue for local juniors. Given that gradings and/or ratings are expected to be a more important part of international junior selection, understanding the reasons for this feature is quite urgent.

There were 12,289 listings on the January ECF list. 10,385 had a category. 2,813 had an ELO reference of which 2,776 had ECF grades and 1,962 had Elo ratings. 1,147 were listed as ENG. Those with both an Elo rating and ECF grade numbered 1,945 of which 1,139 were coded ENG. There were 244 juniors defined here as having either an age on the ECF list or a year of birth after 1994 on the FIDE list. The people in the list are a select bunch, especially those from overseas and may provide misleading data.

Roger de Coverly
Posts: 17316
Joined: Tue Apr 15, 2008 2:51 pm

Re: ECF grades compared with FIDE ratings

Post by Roger de Coverly » Sat Feb 25, 2012 10:21 pm

Brian Valentine wrote: The statistics are such that this relationship would be rejected.
We've been here before. Given that under an Elo system a difference of 200 points corresponds to a 75% expectation and the same applies under the ECF system at 25 points, doesn't any conversion that ignores this suggest there's something non-linear needed in the formulae? Thus express the conversion as International = 8 * ECF plus some variable function of ECF depending on age and grade.

Brian Valentine wrote: Those with both an Elo rating and ECF grade numbered 1,945 of which 1,139 were coded ENG.
I think ENG players with "active" FIDE ratings only number around 900, 939 from
http://ratings.fide.com/card.phtml?event=422959.

Sample size may be an issue, anyone active in local leagues and Congresses would have many more games ECF graded in the same period than FIDE rated.

At one time, you used to see juniors with 150 grades getting 2000+ ratings in their first 4NCL season. It's now probably the other way round, that they have 170s grades but 1900- international.

Actually you've got three possible grades to use for Juniors.
(1) the performance as used for calculations
(2) the published grade including junior addition
(3) the published grade with the junior addition stripped out.

Brian Valentine
Posts: 385
Joined: Fri Apr 03, 2009 1:30 pm

Re: ECF grades compared with FIDE ratings

Post by Brian Valentine » Sun Feb 26, 2012 8:06 am

Roger,
I think most people would fit a straight line to the data. Haven't you pointed out elswhere in similar circumstances that the effect you mention is hidden by people playing both weaker and stronger players masking the expected result?

It appears the list I downloaded from FIDE includes inactive players.

Roger de Coverly
Posts: 17316
Joined: Tue Apr 15, 2008 2:51 pm

Re: ECF grades compared with FIDE ratings

Post by Roger de Coverly » Sun Feb 26, 2012 9:35 am

Brian Valentine wrote: I think most people would fit a straight line to the data.
I expect they would. If you examine the problem in more depth, it looks inappropriate because it fails to reproduce the (connected) formulae used to generate the two sets of values in the first place which are after all, both based on measurements taken from overlapping data. If you convert two players A and B from one system to another, I would suggest a desirable property of the conversion is that it preserves their respective expected scores should they meet. The formula they came up with in 2005 of (FIDE-1250)/5 = ECF was nonsense for this reason.

It's a least squares method presumably. Shouldn't caution be used interpreting the results when the data pairs are not independent? So using rating as a measure of strength, you attempt to measure strength by results. So if you play 30 games under rating system x and a totally separate 30 games under rating system y, it's valid to compare the outcomes. If some or all of the 30 games overlap, don't some of the independence assumptions start to break?

The Scots had a slightly different problem with average English players. The revaluation pushed the "average" English player from around 120 to around 140 but the average Scots player stayed put. They didn't want to incorporate this 20 point inflation into their domestic system when converting results, so they put a 10 in the multiplier.

E Michael White
Posts: 1301
Joined: Fri Jun 01, 2007 6:31 pm

Re: ECF grades compared with FIDE ratings

Post by E Michael White » Sun Mar 18, 2012 10:35 am

It helps to see whats going on in the ECF v FIDE comparison if the ECF grading formula is rewritten:-

g1 = g0 +(A-EE) x 100/n

where

- g1/g0 are the grades at the start/end of the period.
- A/EE are the total actual/expected results.
- n is the number of games played.

This formula gives identical results to the usual +/- 50 scoring where there is no 40 point rule and > 29 games are played. Otherwise it’s an approximation.The equivalent FIDE formula is

f1 = f0 + ( A – EF) x k

f1/f0 are the ratings at the start/end of the period
EF is the expectation from the FIDE table

If n x k = 800, say n=32 games and k = 25 then the ECF formula becomes:-

g1 = g0 + (A – EE) x k/8

which suggests a conversion formula of FIDE = 8 x ECF + C. However life isn’t as simple as that because the EE and EF are not equal and also why choose n x k = 800. Very few English players will play 32 FIDE games in 2 months.

If nk is assumed to be 750 ie n=30 k=25 and the wild assumption that EF and EE are approximately equal then changes in the ECF grade will be the same size as changes in the FIDE rating scaled down. This is possibly the origin of the not very useful approximation on the FIDE website “When K = 10, the rating turns over in approximately 75 games; K = 15, 50 games, K = 30, it is 25 games.” as at that point the FIDE changes seem relatively equal to scaled ECF changes and so independent of the previous grade.

The difference between EE and EF is at its % greatest for differences in grade +/- (30,240) (ECF/FIDE) which is where most games are played. Among others this means that the increasing use of accelerated pairings will accentuate the errors in the linear conversion formula.

If A and EE are close in value it is possible that a player’s ECF grade may increase and their FIDE rating decrease when the only games played are rated under both systems.

Normally grades/ratings will stabilize when A = E under either system which suggests a TPR based system could be more suitable and RdeCs point that matching the E domains should be an initial stage in establishing an approximate conversion.
Last edited by E Michael White on Wed Mar 21, 2012 9:27 am, edited 1 time in total.

Sean Hewitt
Posts: 2190
Joined: Sat Mar 10, 2012 8:18 pm
Contact:

Re: ECF grades compared with FIDE ratings

Post by Sean Hewitt » Sun Mar 18, 2012 12:28 pm

My analysis suggests that FIDE = ECF *8 +631

Brian Valentine
Posts: 385
Joined: Fri Apr 03, 2009 1:30 pm

Re: ECF grades compared with FIDE ratings

Post by Brian Valentine » Sun Mar 18, 2012 12:53 pm

Sean, Great to see you contributing again - can you tell me what data you used?

Subject to Sean not putting me right, I think my argument is not that the logic outlined by Michael is wrong, but what I observed was that it was not working out like that. I think Michael's analysis might break down if expected strength in the formulae is not aligned with actual strength - that is if something is wrong with the grade/rating.

As to Roger's point about independence, I'm not sure I understand. I think we have two independent measures working on an overlapping sample space.

Roger de Coverly
Posts: 17316
Joined: Tue Apr 15, 2008 2:51 pm

Re: ECF grades compared with FIDE ratings

Post by Roger de Coverly » Sun Mar 18, 2012 1:48 pm

Brian Valentine wrote: I think we have two independent measures working on an overlapping sample space.
That's the case as far as the connection between ECF and International is concerned. It's probably not the case for Welsh and Scottish systems where visiting players are introduced into the system either at their established International Elo or their converted English. That presumably sets up a feedback loop where the outcome in the Welsh or Scottish rating depends on the conversion formula used for the initial estimate. Rightly or wrongly, the new player estimates in the ECF system are blind to ratings established elsewhere.

Roger de Coverly
Posts: 17316
Joined: Tue Apr 15, 2008 2:51 pm

Re: ECF grades compared with FIDE ratings

Post by Roger de Coverly » Sun Mar 18, 2012 7:39 pm

E Michael White wrote: If A and EE are close in value it is possible that a player’s ECF grade may increase and their FIDE rating decrease when the only games played are rated under both systems.
There's a growing disconnect between the ECF grades and the International ratings, even at levels as high as 2000. For example, I have just finished a tournament with a TPR of 2050, converting directly to my Jan ECF grade of 175. By contrast the ECF performance, junior recalculation permitting, comes out at just under 200.

I don't really know whether to regard the ranking implied by ECF grades as more correct than FIDE ratings. It must make a difference to pairings if the seedings have been done with an incorrect ranking.

E Michael White
Posts: 1301
Joined: Fri Jun 01, 2007 6:31 pm

Re: ECF grades compared with FIDE ratings

Post by E Michael White » Tue Mar 20, 2012 9:05 pm

Roger de Coverly wrote:I have just finished a tournament with a TPR of 2050, converting directly to my Jan ECF grade of 175. By contrast the ECF performance, junior recalculation permitting, comes out at just under 200.
If that was High Wycombe e2e4 - I dont believe your final TPR. I know TPRs are calculated a varity of ways but the TPR at the end of round 4 shouldnt stay the same at 1940 when you have an additional result of a win v a 1950. Is there a simple explanation ? Perhaps you were repaired ? I esitmate your final TPR should be around 2130 which is within expectation as most of your oppos have lagged FIDE grades. This is the basic problem with the linear relation when some grades are lagged; if you take an overall sample some players fit F = A x E + C and others fit smaller values of either A and/or C; then the next years playing patterns change and the A and C need revision.

Richard Bates
Posts: 2740
Joined: Fri Nov 14, 2008 8:27 pm

Re: ECF grades compared with FIDE ratings

Post by Richard Bates » Tue Mar 20, 2012 9:12 pm

E Michael White wrote:
Roger de Coverly wrote:I have just finished a tournament with a TPR of 2050, converting directly to my Jan ECF grade of 175. By contrast the ECF performance, junior recalculation permitting, comes out at just under 200.
If that was High Wycombe e2e4 - I dont believe your final TPR. I know TPRs are calculated a varity of ways but the TPR at the end of round 4 shouldnt stay the same at 1940 when you have an additional result of a win v a 1950. Is there a simple explanation ? Perhaps you were repaired ? I esitmate your final TPR should be around 2130 which is within expectation as most of your oppos have lagged FIDE grades.
The tournament performance software only includes rated games. The 1950 was unrated so the game was excluded.

E Michael White
Posts: 1301
Joined: Fri Jun 01, 2007 6:31 pm

Re: ECF grades compared with FIDE ratings

Post by E Michael White » Tue Mar 20, 2012 9:22 pm

Yes. Just noticed a * against the 1950. So RdeCs ECF TPG is on a score of 4/5 ie 80% and his Fide TPR on 3/4 ie 75% which would explain all , ... well .. most of all. Lagging can produce a similar effect.
Last edited by E Michael White on Wed Mar 21, 2012 9:24 am, edited 1 time in total.

Ian Kingston
Posts: 1070
Joined: Sat Apr 14, 2007 3:16 pm
Location: Sutton Coldfield
Contact:

Re: ECF grades compared with FIDE ratings

Post by Ian Kingston » Tue Mar 20, 2012 10:21 pm

Richard Bates wrote:
E Michael White wrote:
Roger de Coverly wrote:I have just finished a tournament with a TPR of 2050, converting directly to my Jan ECF grade of 175. By contrast the ECF performance, junior recalculation permitting, comes out at just under 200.
If that was High Wycombe e2e4 - I dont believe your final TPR. I know TPRs are calculated a varity of ways but the TPR at the end of round 4 shouldnt stay the same at 1940 when you have an additional result of a win v a 1950. Is there a simple explanation ? Perhaps you were repaired ? I esitmate your final TPR should be around 2130 which is within expectation as most of your oppos have lagged FIDE grades.
The tournament performance software only includes rated games. The 1950 was unrated so the game was excluded.
TPRs can be extremely misleading. I shared first place in the May 2010 e2e4 Sunningdale Major, and despite playing very badly I ended up with a TPR of 2572 (details here). It's completely meaningless - I just happened to beat all three of my rated opponents. 100% and 0% scores are inherently undefined in rating terms, so the number might just as well have been pulled out of a hat.

Alex Holowczak
Posts: 8506
Joined: Sat May 30, 2009 5:18 pm
Location: Oldbury, Worcestershire
Contact:

Re: ECF grades compared with FIDE ratings

Post by Alex Holowczak » Wed Mar 21, 2012 12:51 am

I was given the data for this at the weekend, and did the same test using the March list.

Excel spewed out a formula of ECF*7.13 + 771.

Take out juniors, and a formula of ECF*7.06 + 793 results.

If you assume ECF*8 to be correct logically given the way the system works, which I do, then ECF*8 + 615 is the best fit using the March data. Take the juniors out, and you get ECF*8 + 625.

I plotted a graph of ECF*8+650 v FIDE. If the conversion is correct, then you should get a gradient of y=x. You actually get y = 0.9321x + 173.8. The constant is created by the gradient not being 1. Is a gradient of 0.93 against an expected gradient of 1 enough to suggest that it is wrong?

The other test Excel can do is the R^2 test, which is a measure of how correlated the data is. I get 0.83 on the whole data, but 0.84 when juniors are taken out. I.e. the fit is better when you take juniors out. This is probably not a conclusion that will astound anyone, but nonetheless...

Brian Valentine
Posts: 385
Joined: Fri Apr 03, 2009 1:30 pm

Re: ECF grades compared with FIDE ratings

Post by Brian Valentine » Wed Mar 21, 2012 8:53 pm

Like Alex, I have been comparing the March FIDE list with the January ECF list. I get similar results.

There are 2813 players listed on the ECF list with a FIDE code. Of these 2776 have a current ECF grade. Of these I've found 1958 with a FIDE rating of which 1601 were not flagged as inactive. I also looked for those who had played at least 1 game according in one of the last 6 FIDE lists. These more recently active numbered 1542 and gave almost identical results to the active (hardly surprisingly).

The fits I obtained were:
Using the 1958: 7.07ecf+783=Fide
Using the 1601: 7.24ecf+744=Fide

So I've addressed RdC's point about inactive players. I also calculated the average K factor for the 1958 - it came out at 18.7. I think under EMW's analysis the factor might be as 6 rather than 8 (I'm not sure if the average is the right number, but the outcome shows the "25" is suspect). This does not change the 8 derived from the .75/.25 method outlined by RdC. I did count the number of "ENG" in my analysis and this came to 868, which given the source RdC gave is now listed as 954th suggests that some of the FIDE codes may be missing from the ECF list (unpaid up members?).

My original post was more about subgroups (especially those where the ECF has this start-from-scratch method) were not aligning with the main alternative measure of strength/performance. Would Alex care to publish his fit of the juniors only comparison to show whether or not it is significantly different from the non-juniors?

Post Reply