Universal Rating System

Discuss anything you like about chess related matters in this forum.
User avatar
Paolo Casaschi
Posts: 1089
Joined: Thu Jan 08, 2009 6:46 am

Re: Universal Rating System

Post by Paolo Casaschi » Wed Jan 04, 2017 11:47 am

Roger de Coverly wrote:
Paolo Casaschi wrote: using more relevant data makes your statistical model more accurate
What are you trying to model though?
Reading their own website, their driver seems the ability to correctly predict the result of future games. Whatever works, it works. It can be very misleading to think only in abstract terms. You can make an assumption, then run the data through the model and see what happens. Given the academic experience of the authors of the new system, I'm sure they follow all proper methodology to validate their assumptions, for example tuning the model using data up to 2015 and using 2016 data for validating the results, pretty standard stuff.

Roger de Coverly
Posts: 18096
Joined: Tue Apr 15, 2008 2:51 pm

Re: Universal Rating System

Post by Roger de Coverly » Wed Jan 04, 2017 11:55 am

Paolo Casaschi wrote: Reading their own website, their driver seems the ability to correctly predict the result of future games.
The current FIDE approach tells us that even if A and B are equal at long time controls, that A may be somewhat better at Rapid and Blitz. If you discard this information how does this help predict the result of a future game, if the expected result depends in part on the time control?

Whatever fancy method they have come up with could be run three times or more to get calibrated ratings for different forms of chess.
Last edited by Roger de Coverly on Wed Jan 04, 2017 12:07 pm, edited 1 time in total.

Angus French
Posts: 1604
Joined: Thu May 15, 2008 1:37 am
Contact:

Re: Universal Rating System

Post by Angus French » Wed Jan 04, 2017 11:56 am

NickFaulks wrote:
Angus French wrote:
Graham Borrowdale wrote:I see no circumstances where Grand Chess Tour would be collating results from all FIDE events, including rated rapidplays, and producing their own parallel rating list.
I think that must be right.
I'm not convinced, and suspect that GK really does still harbour plans to set up a rival rating system. However, we have seen no details at all - presuming that Dr Iqbal's paper, referenced above, is indeed the joke that it appears to be.
So... the URS has in fact used all the results of all the games rated by FIDE in the last 6 years (and there are 246,022 players in the initial URS list - as I found after downloading a copy). And presumably this data has been willingly provided - for a fee? - by FIDE.
Last edited by Angus French on Wed Jan 04, 2017 12:07 pm, edited 1 time in total.

User avatar
Paolo Casaschi
Posts: 1089
Joined: Thu Jan 08, 2009 6:46 am

Re: Universal Rating System

Post by Paolo Casaschi » Wed Jan 04, 2017 11:59 am

Roger de Coverly wrote:Their rating system will lack acceptance until they publish the detail workings. Otherwise there might be a secret formula that adds a few points for players approved of by Kasparov and subtracts points for those who don't.
I agree that more details of the inner workings are required in order to gain acceptance. It's the same for all rules and regulations, for example I find appalling the methodology of the anti-cheating commission to operate in secrecy without any details of their automatic detection systems.

However, as much as I enjoy conspiracy theories, I would not worry about manual tweaking under Kasparov direction.
Also keep in mind, from the little they explained on their website, even if they published lot more details, just by the nature of the system I suspect it would be extremely difficult to replicate the calculations. Certainly it won't be as simple as taking the previous rating of yourself and the 5 people you played against since the last rating update in order to calculate with pen and paper your own new rating.

If you do not trust the people running the system, you might as well make your mind now about it.

MartinCarpenter
Posts: 2439
Joined: Tue May 24, 2011 10:58 am

Re: Universal Rating System

Post by MartinCarpenter » Wed Jan 04, 2017 12:03 pm

The reason for scepticism is that it seems inherently implausible.

Peoples playing strengths at long play/rapid/blitz are only moderately loosely collorated, so trying to combine them into one value rather than tracking them independently basically automatically loses you accuracy.

Obviously if you feel like you must have one figure for predicting how people will do over games at a combination of the time limits then sure, but why?

Also perhaps a few potential gains if there aren't enough games at each individual time limit to do individual ratings.

User avatar
Paolo Casaschi
Posts: 1089
Joined: Thu Jan 08, 2009 6:46 am

Re: Universal Rating System

Post by Paolo Casaschi » Wed Jan 04, 2017 12:06 pm

Roger de Coverly wrote:
Paolo Casaschi wrote: Reading their own website, their driver seems the ability to correctly predict the result of future games.
The current FIDE approach tells us that even if A and B are equal at long time controls, that A may be somewhat better at Rapid and Blitz. If you discard this information how does this help predict the result of a future game, if the expected result depends in part on the time control?
They did not say anywhere that they discard the information about A being better at rapid and blitz. The fact they keep those rapid and blitz "gaps" suggests otherwise. We do not know at what extent they use it though.
What do we know is that the FIDE Elo completely ignores all rapid and blitz games when calculating the main rating, this we know for sure.

Ultimately, whatever works, it works. You just need to look at the data. Making qualitative assumptions only points you to a direction when looking for a new model, but as much as your qualitative assumptions make sense, if the resulting model is worse, then you have to accept their model is better fit for the prediction purpose.

Ian Thompson
Posts: 2127
Joined: Wed Jul 02, 2008 4:31 pm
Location: Awbridge, Hampshire

Re: Universal Rating System

Post by Ian Thompson » Wed Jan 04, 2017 12:40 pm

I can't see anything in the information provided telling me how I calculate the expected result between two players given their universal ratings. Without that information, I don't think it will gain much acceptance.

I'm also curious to know why, in the downloadable rating list, there is no-one ranked 134th in the world.

Roger de Coverly
Posts: 18096
Joined: Tue Apr 15, 2008 2:51 pm

Re: Universal Rating System

Post by Roger de Coverly » Wed Jan 04, 2017 1:04 pm

Paolo Casaschi wrote:The fact they keep those rapid and blitz "gaps" suggests otherwise. We do not know at what extent they use it though.
There are a couple of unexplained columns in the download, namely RGap and BGap. Default values for anyone who hasn't played FIDE rated rapid or blitz appear to be 32 and 84 respectively.

For anyone not playing FIDE rated Rapid or Blitz, the Universal ratings are reporting on the same results as the FIDE ones. Checking some ENG players with ratings under 1900 shows up some wide divergences. I think we knew that anyway, that by comparison with rankings using ECF grades, the parallel ranking using FIDE Elos is all over the place below about 1900 or 2000.

Angus French
Posts: 1604
Joined: Thu May 15, 2008 1:37 am
Contact:

Re: Universal Rating System

Post by Angus French » Wed Jan 04, 2017 1:11 pm

Ian Thompson wrote:I'm also curious to know why, in the downloadable rating list, there is no-one ranked 134th in the world.
Also, I notice there are differences between the list given at chess.com and the downloadable Excel/CSV lists:

Chess.com list:
ChessDotCom.PNG
ChessDotCom.PNG (116.25 KiB) Viewed 728 times
Excel list:
URSRatingsJan2017.png
URSRatingsJan2017.png (22.38 KiB) Viewed 728 times
For example, Grischuk is 2771, the same as Anand, in the Chess.com list but 2772, one point ahead of Anand, in the Excel/CSV download. This is important for determining who qualifies as a wildcard in the 2017 Grand Chess Tour.

Ian Thompson
Posts: 2127
Joined: Wed Jul 02, 2008 4:31 pm
Location: Awbridge, Hampshire

Re: Universal Rating System

Post by Ian Thompson » Wed Jan 04, 2017 1:13 pm

Roger de Coverly wrote:There are a couple of unexplained columns in the download, namely RGap and BGap. Default values for anyone who hasn't played FIDE rated rapid or blitz appear to be 32 and 84 respectively
They are explained:
http://www.universalrating.com/about-us.php wrote:Rapid Gap = the Universal Rating advantage the player would need for a 50% expected score in rapid (Game in 30 minutes each) against an opponent whose quality and consistency of play do not worsen at quicker time controls.

Blitz Gap = the Universal Rating advantage the player would need for a 50% expected score in blitz (Game in 5 minutes each) against an opponent whose quality and consistency of play do not worsen at quicker time controls.

Roger de Coverly
Posts: 18096
Joined: Tue Apr 15, 2008 2:51 pm

Re: Universal Rating System

Post by Roger de Coverly » Wed Jan 04, 2017 1:27 pm

Ian Thompson wrote: They are explained:
It also says
Published URS™ Ratings represent the system's assessment of a player's strength at Classical chess.
It's a rather core question. If you have two players, one of whom never plays rated Blitz, whilst the other plays events like the Hastings pub evening and scores badly. They play an identical field at Classical chess with identical results. Who has the higher rating and if so, why?

The idea of just having a Classical rating and handling Rapid and Blitz by applying an uncertainty factor makes some sense. It's not what the proponents of Universal Rating appeared to be advocating.

(edit) There's a FAQ on their website.

http://www.universalrating.com/faqs.php

Ignoring the Rapid and Blitz adjustments, the underlying premise appears similar to the method used by the ECF for new players/juniors and indeed the method used by the ECF when it recalculated all the grades a few years ago. They are taking six years of results and running them through some process. As it's a rolling six years with higher weightings for more recent results, ratings can change even if you don't play. It has the possible advantage that you get some credit for beating the ten year old when he or she becomes a GM six years later. For anyone running or organising events where eligibility or other rules depend on ratings continuously moving ratings can be a nuisance.

At a time when FIDE might be moving on the 60 moves in two hours issue, their system is far from universal if it excludes any games with time limits of over an hour and less than two.

There's an underlying historic premise in rating systems that two players of the same rating should have a 50-50 chance. In that context I really don't see what the Rgap and Bgap are trying to achieve. As a measure of uncertainty perhaps, but isn't it a nonsense to assert that for players of equal rating and gap, that A has to exceed B's rating by the gap to achieve a 50% expected score, when the converse must also apply?

(/edit)

MartinCarpenter
Posts: 2439
Joined: Tue May 24, 2011 10:58 am

Re: Universal Rating System

Post by MartinCarpenter » Wed Jan 04, 2017 2:18 pm

That last is easy - they're actually more or less internally calculating rapid/blitz ratings but for some political/marketing reason they're publishing a single grade. The gaps are then the statisticians insisting on putting things half right ;)

With how many rapid/blitz events there are in the grand tour now a single combined rating does of course actually make lots of sense for picking the people to play in it.

Ian Thompson
Posts: 2127
Joined: Wed Jul 02, 2008 4:31 pm
Location: Awbridge, Hampshire

Re: Universal Rating System

Post by Ian Thompson » Wed Jan 04, 2017 2:44 pm

Roger de Coverly wrote:There's an underlying historic premise in rating systems that two players of the same rating should have a 50-50 chance. In that context I really don't see what the Rgap and Bgap are trying to achieve.
Are they not, effectively, giving you Rapid and Blitz ratings. Take Adams and Short, for example. Their ratings and gaps are:

Code: Select all

PlayerName		URating	RGap	BGap
Adams, Michael		2730	29		78
Short, Nigel D		2672	54		112
So Adams is 58 points better at classical time limits, 83 points better at rapidplay and 92 points better at blitz.

If the method had been disclosed, you could then calculate Adams expected score against Short at each of these time limits.

MartinCarpenter
Posts: 2439
Joined: Tue May 24, 2011 10:58 am

Re: Universal Rating System

Post by MartinCarpenter » Wed Jan 04, 2017 2:50 pm

Since the main grade is some sort of combination of performance at all time limits, the gap at classical limits will logically be a little bit smaller - and the one at blitz/rapid correspondingly higher - than that raw comparison might suggest.

Nick Grey
Posts: 1153
Joined: Thu Dec 01, 2011 12:16 am

Re: Universal Rating System

Post by Nick Grey » Wed Jan 04, 2017 10:07 pm

Wow:
99074 in the top 100,000
Grey, Nicholas D
1793
32
83
M
ENG
1256 in ENG list.

I don't believe it.

Post Reply