Another way of looking at the ecf grading method(?)

[Brian Valentine]

I am attaching two files. The first demonstates that the ECF grading iteration method approximates to a couple of matrix equations that can be solved without iteration. This approach is impractical with the number of players in the system, but gives a basis for looking at some of the wilder things seen on the latest list.

The second is an excel workbook which looks at an imaginary population of up to 10 players and shows the iteration approach and the matrix approach gives the same answer. Annotations are provided.

Users can input some results and see the effect. The workbook contains one macro.

Michael White has logged a similar approach and Roger de Coverly, amongst others, has been talking matrices/vectors. However I hope this work can shed some light on the situation. All comments welcomed - especially those that address the unfathomable bits!

[E Michael White]

I think the point of the first estimate of the grade used in the iterative approach is that the iterative approach is not guaranteed a unique solution. A reasonable first guess should get the method to the most sensible solution. I put an oblique comment on this point in my paper on the matrix method.

Brian ,
We may have to disagree on that but it depends on what the grading team do with the data. My comment related to the iterative approach used for new starters as described on the ECF website : -

Estimating a starting Grade for an ungraded player
A Rapid Grade, where available, will be used in default of a Standard Grade; and vice versa. If the player has no Grade at all, a starting grade is calculated as follows.
Stage 1 is to calculate a 'grade' for each ungraded player on his games against graded opponents. The 40-point rule is not used. If all his opponents are graded, it stops there and the result will be used as his starting grade.
Stage 2 brings in games between the ungraded players. Once again the 40-point rule is not used. The players are 'graded' on all their games, using as starting grades the figures obtained from Stage 1.
The resulting 'grades' will not be very accurate. So they are fed in again as new starting grades, and Stage 2 is repeated. This continues, with increasing accuracy each time, until the figures (more or less) stop changing. The starting grades can then be considered accurate.
These starting grades are then used in the grading proper.

The iterative process in use, is step 2 and its repetition. I take the description to mean that the F matrix and equation in my posting should contain, as it does, only graded opponents of ungraded players,those results, other ungraded players and the results against other ungraded players. Results for other games of graded opponents and other players, who did not play against ungraded players are not included. The results against graded players, and the grades of those graded players remain the same, for each step of the iterative process. This was confirmed by Richard Hadrell on this forum in Nov 2007. As a result of this the lower LH corner of my F matrix contains zeros. If the results against graded players were carried through just via starting estimates for the iterative process this will not give accurate weight to the results v graded players. However I do not think this is what the ECF say in their description.

To form your N matrix take my F matrix and transcribe the graded players to the top. In addition, and this is the important difference, you have increased the scope of your N matrix to cover games by graded players against other graded players to give an overall picture.

The F matrix which I described is a single order row stochastic Markov matrix which means that the process will always converge to a unique solution as it reduces to the solution of n simultaneous linear equations for n ungraded players.

“The natural solution (it is not necessarily unique, but it is a solution that makes sense)for p* is:
P* = [I- N^^UU ]^^-1 { N^^UR p^^0 + S^^UR [1] + S^^UU[1]}.................... (1) “

Where a ^^ means superscript follows

My posting showed that the solution is unique according to the information given by the ECF grading team. I realise this is sparse and open to misinterpretation.

One way this could not be the case is if the group of ungraded players contains one or more groups of players, who only play one another (lighthouses again), in which case an estimate of start grades is needed and the process may still not produce a unique solutions for those players. However Richard Hadrell confirmed that groups like that are not included.

Another source of discrepancy could be in the treatment of a player who is ungraded LP but has a RP grade (As above - A Rapid Grade, where available, will be used in default of a Standard Grade; and vice versa). If this player has played against other ungraded players, are those results included in the iterative process to obtain start grades for the remaining ungraded players ? If so then is the RP graded player assumed to have a constant grade during the iterative process or can it float with each round of iteration ? If the latter an otherwise closed group of ungraded players have one of their number with a game against this RP graded player and then the whole group would be graded on the basis of this unwise estimate which could cause considerable over or under rating not just owing to it s being RP but also floating.

[Brian Valentine]

Michael,
Thanks for your constructive response. I don't think we are disagreeing on a point that changes anything on what either of us has done. It's just an archane mathematical point and I think we agree where the grading team gets to. I also agree with you that there are bits that Hadrell's summary misses out (not surprising given its context) which might affect things - though I doubt if they change the mathematics, just possibly the implications.

Now I look at the appendices of your post with an enlightened eye, I think that in what I wrote I have stumbled on what your final F matrix looks like, without any need for markov matrices.

I am interested on how the process affects prior rated player's grade, and that is why I went the way I did. My premis is that now one has the matrices one can use them to look at bias in and efficiency of the grading team's estimates. It also gives another line of enquiry into Sean Hewitt's original work. However I'm making very slow progress on these issues.

[Roger de Coverly]

I believe it is generally agreed that if you repeated apply the same result set to a list of grades, it shouldn't matter where the grades start, they will eventually converge to the same relative list.

Or expressed in another way, if the grades at the start of a season are "incorrect", then the grading system should, over time, self correct to bring itself back in line with the results. One should bear in mind that there are a number of pools where the players play the same opposition every year, so the real world has some resemblance to the model recursion. Does it follow therefore that you need stronger evidence than just one season's results before you can decide that the natural self-correction process is inadequate?

[E Michael White]

I believe it is generally agreed that if you repeated apply the same result set to a list of grades, it shouldn't matter where the grades start, they will eventually converge to the same relative list.

Roger this may be what is agreed by some players but it is not generally true.

What is true is that if there is at least one player who has a grade which is taken as fixed and every other player can be reached by a link of :- --- an oppo of an oppo of an oppo of .... ......... an oppo of the player is the fixed player then the process does converge and is independent of the start grade of players other than the fixed grade player.

[Roger de Coverly]

What is true is that if there is at least one player who has a grade which is taken as fixed and every other player can be reached by a link of :- --- an oppo of an oppo of an oppo of .... ......... an oppo of the player is the fixed player then the process does converge and is independent of the start grade of players other than the fixed grade player.

You can also give the one "fixed" player an arbitrary start grade and still get the same relative distance between the players - that's allegedly how they did the revaluation. The reason I'm 186 is only because Howard Grist says so. It could equally have been 178 but then everyone else would also be 8 points lower. For any group of players that can't be reached by the main process, you can determine their relative grades but not how they fit into the larger list.

The point I was making is that the grading system is self-correcting over time unless there are forces "round the edges" which override its natural convergence.

[Brian Valentine]

For anyone interested Amazon are stocking a paperback reprint of " The rating of chessplayers" By Elo for a reasonable £15-ish.

[Peter Rhodes]

I was trying to find a copy of Harkness's book "How I dreamt up a rating system while I was in the pub", and his sequel "How I hoodwinked a nation".

[Brian Valentine]

The attached note builds on the previous note to set out the equations of bias in the ecf rating system. If the system is initially unbiased then there is no reason to expect it to become biased. However there are situations where bias exists and this note develops multipliers of any existing bias. These can be used to look at the robustness of the model, although these test will be for further research.

[Roger de Coverly]

If the system is initially unbiased then there is no reason to expect it to become biased.

Is "stretch" just a form of bias? If you consider the ECF grading system to be a measure of how much worse you are than the top players, then for 2008 and earlier years the graders expressed the opinion that grades in the 180s' had been 6-8 points too far away, those in the 120s about 25, those in the 90s about 30 and in the 50s about 50. Then they removed the "bias" with the 2009 revaluation. We won't know until next year whether they've overcorrected.

It's something they never tested, but even if the 2008 grades were biased it was difficult to see any signs of it increasing over time. In fact isn't the natural tendency of the grading system to self-correct provided "most" players are correctly ranked - so you have a player 30 points biased who plays 30 separate (active) people, each picks up one just one point of bias. If you have relatively inactive biased players, even that may not matter since they hardly play anyone.

[Brian Valentine]

Roger,
Stretch is not the same as bias - the way I have defined it! In the note g is fixed and known. I'm stuck at the moment getting this linked to the stretch concept.

While the graders never published any tests of the system, I think it is because the tests to use are not obvious. The thread is an attempt to put the foundations in place to develop such tests.

At present I agree with you, in that I have not yet found a situation where the "multipliers" are other than benign. However there are situations where players playing many games are actually linked to a small (possibly biased) community of rated players. But then we knew that anyway!

[Roger de Coverly]

Bias, positive or negative, exists when the grade, an estimate of player’s strength, differs from true strength.

I was seeing "estimate of strength" minus "grade" as a sort of error function. The error function contains all the reasons why the "grade" isn't correct. These could include lag, stretch, grader's cook factors ( junior increments) etc as well as bias/distortion.

[Brian Valentine]

Roger,

I have no problem with your concepts - it's just I haven't got all the maths for all this yet. I tried to cover some of this in the introduction, but I think you are looking to update my plagiarism of EMW's ideas.

I think there is a need to develop some common definitions, if this thread starts moving forward in a promising way. The note has only been accelerated when I realised that EMW's F matrix was not the same as my N matrix. Maybe we should consider how to build a glossary that can be easily be updated on this site.

[E Michael White]

Anyone who finds this sort of thing interesting might be interested in this link.
http://wwwmaths.anu.edu.au/~brent/pd/rpb237.pdf

The authors' list of references could be extended to include a paper by Roger Penrose from the 1950s but formula 13 is mildly interesting as are the comments in section 7. Also the remarks in the second sentence of section 1 might not hold up following the arbiter story in the Canterbury tales. Additionally a footnote to page 7 shows the method turns up in a Google search algorithm.

[Adam Raoof]

Can I recommend Professor Chris Bishop's 2010 Turing Lecture?

'Embracing Uncertainty'

http://conferences.theiet.org/lectures/turing

I think that this;

http://scpro.streamuk.com/uk/player/Def ... x?wid=7739

is the direct link. Slide 32 onwards, about 32 minutes in (Bayesian ranking) is the bit you might be especially interested in if you want to see how professional mathematicians view the Elo System. I would be interested in your opinions!