Estimating the Strength of Ungraded Players

General discussions about ratings.
Ian Thompson
Posts: 3551
Joined: Wed Jul 02, 2008 4:31 pm
Location: Awbridge, Hampshire

Estimating the Strength of Ungraded Players

Post by Ian Thompson » Wed Jul 02, 2008 9:57 pm

If a league runs grading limited divisions they are likely to have to estimate the grades of ungraded players to determine which divisions they should be allowed to play in. The initial estimate may be little more than a guess, but after a few games it should be possible to make a more accurate estimate, with an indication of how accurate it is likely to be.

Can anyone provide a formula to do this, along the lines of:

N games played at an average grade of R. There is an X% confidence level that the player's true strength is between A and B.

N, R and X are inputs to the calculation. A and B are the outputs.

Ian Thompson

Ian Jamieson
Posts: 203
Joined: Tue Apr 03, 2007 4:00 pm

Re: Estimating the Strength of Ungraded Players

Post by Ian Jamieson » Tue Jul 08, 2008 2:47 pm

Interesting

A possible approach is:-

SD = 25
Y = 100 - X
A = R - PPND(Y/2) x SD / SQRT(N)
B = R + PPND(Y/2) x SD / SQRT(N)

where

SD is short for standard deviation. According to Hooper and Whyld in the Oxford Companion to Chess 200 points is the standard deviation for Elo ratings so 200/8 = 25 is the standard deviation for ECF grades if my elementary statistics is correct and using the traditional formula for converting Elo to ECF.

PPND(p) is the (100-p) percentage point of the Normal distribution (with mean 0 and standard deviation 1). Common values are as follows

X Y/2 PPND(Y/2)

99 0.5 2.5758
98 1.0 2.3263
95 2.5 1.96
90 5.0 1.6449
80 10 1.2816

I can provide other values if necessary or you can obtain them from the function NORMSINV(1-Y/200) in Excel. Indeed if you use Excel you can replace PPND(Y/2) * SD / SQRT(N) by CONFIDENCE(Y/100,SD,N)

Example 1

An ungraded player draws one game against a player graded 150.

There is a 90% confidence level that the ungraded players strength is between

150 - 1.6449 x 25 / 1 = 109

and

150 + 1.6449 x 25 / 1 = 191

which seems reasonable to me.

Example 2

An ungraded player plays 30 games against opposition with an average grade of 150 and scores exactly 50%

There is a 90% confidence level that the player's true strength is between

150 - 1.6449 x 25 / SQRT(30) = 142

and

150 + 1.6449 x 25 / SQRT(30) = 158

I have no problem with this but you may feel that the interval should be narrower after 30 games.

If you are going to adopt this approach for ungraded players presumably you should also use it for graded players using last seasons' N and/or the new season results.

As asked I have provided a 2-sided interval but depending on how your league's grading limits operate a 1-sided interval may be more appropriate.

Please note I am not a grader and have no link with the grading team.

Ian Jamieson

Alex Holowczak
Posts: 9085
Joined: Sat May 30, 2009 5:18 pm
Location: Oldbury, Worcestershire

Re: Estimating the Strength of Ungraded Players

Post by Alex Holowczak » Fri Jun 05, 2009 8:43 pm

I have an idea on this, which may be statistically useless, but anyway... You need to know your initial grade, Gi, in order to calculate the final grade, Gf. How statistically valid is it to run a loop through testing all integer values for Gi, and the end grade is where Gi = Gf (or | Gi - Gf | is the lowest value). Would there be instances though where Gi = Gf for more than one value of Gi?

User avatar
IM Jack Rudd
Posts: 4818
Joined: Tue Apr 17, 2007 1:13 am
Location: Bideford

Re: Estimating the Strength of Ungraded Players

Post by IM Jack Rudd » Fri Jun 05, 2009 8:45 pm

Alex Holowczak wrote:I have an idea on this, which may be statistically useless, but anyway... You need to know your initial grade, Gi, in order to calculate the final grade, Gf. How statistically valid is it to run a loop through testing all integer values for Gi, and the end grade is where Gi = Gf (or | Gi - Gf | is the lowest value). Would there be instances though where Gi = Gf for more than one value of Gi?
You may run into trouble when the player's percentage score is 100 or 0.

Alex Holowczak
Posts: 9085
Joined: Sat May 30, 2009 5:18 pm
Location: Oldbury, Worcestershire

Re: Estimating the Strength of Ungraded Players

Post by Alex Holowczak » Fri Jun 05, 2009 9:06 pm

FM Jack Rudd wrote:
Alex Holowczak wrote:I have an idea on this, which may be statistically useless, but anyway... You need to know your initial grade, Gi, in order to calculate the final grade, Gf. How statistically valid is it to run a loop through testing all integer values for Gi, and the end grade is where Gi = Gf (or | Gi - Gf | is the lowest value). Would there be instances though where Gi = Gf for more than one value of Gi?
You may run into trouble when the player's percentage score is 100 or 0.
You're right, I just tried it on a spreadsheet. It'll go on forever, because you gain 10 points if you win no matter what the rating difference is. Maybe in that scenario, it could be set to the lowest graded defeat (if you lost) or the highest graded win (if you won). However, even with 1 win and 10 losses to players graded from 50 to 150 with a difference of 10 between them, it never converges. However, if you get a good mix of results, it seems to work quite well. It's almost like a normal distribution in terms of how reliable the results are... Hmm.