Universal Rating System

Discuss anything you like about chess related matters in this forum.
Graham Borrowdale
Posts: 124
Joined: Thu Jun 09, 2016 10:54 pm

Re: Universal Rating System

Post by Graham Borrowdale » Tue Dec 20, 2016 12:52 pm

Is this a hypothetical discussion, or did I miss something? I thought the proposal was from the Grand Chess Tour to produce some combined ratings for the purpose of eligibility to its events. That would involve the top players in the world, but would be an irrelevance to the masses unless FIDE adopted it as well.

Roger de Coverly
Posts: 17982
Joined: Tue Apr 15, 2008 2:51 pm

Re: Universal Rating System

Post by Roger de Coverly » Tue Dec 20, 2016 1:11 pm

Graham Borrowdale wrote: I thought the proposal was from the Grand Chess Tour to produce some combined ratings for the purpose of eligibility to its events.
That is their proposal, what isn't known is how far they propose to extend their combined ratings.

Graham Borrowdale
Posts: 124
Joined: Thu Jun 09, 2016 10:54 pm

Re: Universal Rating System

Post by Graham Borrowdale » Tue Dec 20, 2016 2:46 pm

Roger de Coverly wrote:
Graham Borrowdale wrote: I thought the proposal was from the Grand Chess Tour to produce some combined ratings for the purpose of eligibility to its events.
That is their proposal, what isn't known is how far they propose to extend their combined ratings.
Personally I would not worry. I see no circumstances where Grand Chess Tour would be collating results from all FIDE events, including rated rapidplays, and producing their own parallel rating list.
Professional players might be more concerned about the 'closed shop' nature of the GCT events, and, as you say, the possible exclusion of certain players.

Angus French
Posts: 1582
Joined: Thu May 15, 2008 1:37 am
Contact:

Re: Universal Rating System

Post by Angus French » Tue Dec 20, 2016 3:53 pm

Graham Borrowdale wrote:I see no circumstances where Grand Chess Tour would be collating results from all FIDE events, including rated rapidplays, and producing their own parallel rating list.
I think that must be right. And if it is, the URS could just use the same data used by http://www.2700chess.com/.

NickFaulks
Posts: 4965
Joined: Sat Jan 02, 2010 1:28 pm

Re: Universal Rating System

Post by NickFaulks » Tue Dec 20, 2016 4:37 pm

Angus French wrote:
Graham Borrowdale wrote:I see no circumstances where Grand Chess Tour would be collating results from all FIDE events, including rated rapidplays, and producing their own parallel rating list.
I think that must be right.
I'm not convinced, and suspect that GK really does still harbour plans to set up a rival rating system. However, we have seen no details at all - presuming that Dr Iqbal's paper, referenced above, is indeed the joke that it appears to be.

Nick Grey
Posts: 1130
Joined: Thu Dec 01, 2011 12:16 am

Re: Universal Rating System

Post by Nick Grey » Tue Dec 20, 2016 10:12 pm

So a system to select wildcards. Completely irrelevant for everyone else. Sorry but I thought those hosting a tour event selected a wild card
Personally using something like live ratings with bugs on the day before All Fools day is about right.

Chris Rice
Posts: 2725
Joined: Thu Nov 08, 2012 5:17 am

Re: Universal Rating System

Post by Chris Rice » Wed Jan 04, 2017 7:54 am

Here's the new homepage for the URS. http://universalratingsystem.com/ of which the highlights include:

"It is critical to note, however, that the URS™ does not incorporate Elo ratings anywhere within its actual calculation."

"Universally Better Than Elo"

"One commonly-held (though admittedly subjective) belief is that classical chess is categorically different from rapid chess and even more different from blitz chess and the three types of chess ought to be kept separate."

Finally, right at the end is a wildcard for Karjakin for the GCT. With his showing in the WC rapid and winning the blitz his URS will certainly be challenging for the top spot I would assume.
Last edited by Chris Rice on Wed Jan 04, 2017 8:42 am, edited 1 time in total.

Brian Valentine
Posts: 419
Joined: Fri Apr 03, 2009 1:30 pm

Re: Universal Rating System

Post by Brian Valentine » Wed Jan 04, 2017 8:15 am

It is a rather disappointing launch. We know through the Kaggle competition that there are many methods that could perform better than Elo. Although the developers are recognised as leaders in this field, normally a group of academics would publish alongside a paper to allow scrutiny by peers. On their websites Sonas still associates himself with Chessmetrics and Glikman has all sorts of other sports evaluation systems although he mentions URS.

At present all we know about the detail is that it is a black box that combines all speeds of chess in one rating. Other than that the only clue is that it resembles the Yorkshire Grading System (that also uses an "exponential decay rate").

Mick Norris
Posts: 7403
Joined: Tue Apr 17, 2007 10:12 am
Location: Bolton, Greater Manchester
Contact:

Re: Universal Rating System

Post by Mick Norris » Wed Jan 04, 2017 8:49 am

Nepo, Karjakin & Vishy have the 3 GT wildcard nominations

They join So, Caruana & J'adoubeamura who qualified from the 2016 GT; Carlsen, Kramnik & MVL qualify from 2016 by rating
Any postings on here represent my personal views and should not be taken as representative of the Manchester Chess Federation www.manchesterchess.co.uk

Roger de Coverly
Posts: 17982
Joined: Tue Apr 15, 2008 2:51 pm

Re: Universal Rating System

Post by Roger de Coverly » Wed Jan 04, 2017 9:09 am

Brian Valentine wrote:We know through the Kaggle competition that there are many methods that could perform better than Elo.
Didn't some of the entrants to that competition exploit the feature that much of the data would have been taken from seeded Swiss competitions? Thus their models were starting with partial knowledge of existing rankings.

I would have thought it self evident that some players are better at "classical" chess than at rapid or blitz or vice versa. If you require a system that ranks players at classical chess for pairings, eligibility or whatever, then adding rapid and blitz into the mix introduces distortions. If you have two players of identical strength at normal time controls, does it help or hinder to tie break them for rankings using rapid and blitz? At the very least it distorts the predicted result at classical chess.

Brian Valentine
Posts: 419
Joined: Fri Apr 03, 2009 1:30 pm

Re: Universal Rating System

Post by Brian Valentine » Wed Jan 04, 2017 10:01 am

Roger de Coverly wrote:
Brian Valentine wrote:We know through the Kaggle competition that there are many methods that could perform better than Elo.
Didn't some of the entrants to that competition exploit the feature that much of the data would have been taken from seeded Swiss competitions? Thus their models were starting with partial knowledge of existing rankings.

I would have thought it self evident that some players are better at "classical" chess than at rapid or blitz or vice versa. If you require a system that ranks players at classical chess for pairings, eligibility or whatever, then adding rapid and blitz into the mix introduces distortions. If you have two players of identical strength at normal time controls, does it help or hinder to tie break them for rankings using rapid and blitz? At the very least it distorts the predicted result at classical chess.
Some entrants, but not all did. I know the YGS did not (it beat FIDE).

While there were some limitations in that competition, the idea that other information might improve things is not. It is more about whether it is worth the extra effort against the difficulty in checking the outputs.

There is merit in considering one rating for young juniors. The extra information should better monitor their improvement. If the distortions you outline are real then there is a question about what age juniors start being treated as adults, allowing the various grades to diverge.

Richard Bates
Posts: 2878
Joined: Fri Nov 14, 2008 8:27 pm

Re: Universal Rating System

Post by Richard Bates » Wed Jan 04, 2017 10:23 am

I see the English flag has changed from the St Georges Cross to the Union Jack.

User avatar
Paolo Casaschi
Posts: 1087
Joined: Thu Jan 08, 2009 6:46 am

Re: Universal Rating System

Post by Paolo Casaschi » Wed Jan 04, 2017 10:32 am

Roger de Coverly wrote:I would have thought it self evident that some players are better at "classical" chess than at rapid or blitz or vice versa.
That is true. However these are also self evident: there is some correlation between the chess abilities at different time controls; using more relevant data makes your statistical model more accurate. Ultimately it's a trade-off and only empirical experience will show which of the two factors prevails over a very large sample.

My first reaction to the announcement: why do we need another rating system, we already have too many and should consolidate those instead of creating a new one. On second thought, this is possibly what they are aiming for: building a rating platform that initially uses only FIDE rated games but eventually rates more and more games, such as those currently rated at the national level but not (yet) by FIDE. For example, they might offer to small federations with only a few thousand players to rate all their events by uploading results to their server; the small federation could then dismantle their own rating system and better use their resources to concentrate on supplying high quality data to the universal rating platform. This plan might actually work, either by itself by getting rid of national ratings of limited value and use or just by pushing FIDE to extend their Elo rating in a similar way, achieving the same result.

I looked at their website, they do not share much details of the inner workings of their rating algorithm: this is probably the main concern at the moment, the lack of adequate peer review.

Reading their own description, they clearly choose accuracy versus simplicity. As a result, no surprise that their ratings might is more accurate than FIDE's Elo by whatever metric you choose.

It looks like they take into account the past results at every update of the ratings, thus understanding better old results based on the new information. For example, their system is probably much better than FIDE's Elo dealing with results with juniors. With FIDE's Elo nothing will compensate/correct my loss against a junior 200 points lower than me and performing higher than me; I suspect that the more that junior player further improves their results, the more the universal rating will correct the impact of my previous loss against it. This would definitely be a plus.

The cost for this accuracy comes with the complexity associated with the new system. FIDE's Elo is designed to allow each player to calculate very easily the impact of a win/draw/loss against any other rated player. With the universal rating I suspect this will be impossible. In fact it seems even worse, because of each month the rating system understands better old results, even a player that has not played any games since the last rating update might see their rating change. For example, a draw against an underrated junior 200 points lower than me might drag my rating down this month, but in the next few months, even if I do not play any game, consistent high-performing results from that junior might push the system to reconsider the weight of that loss and lift my rating up.
This difficulty in predicting the impact of individual results on ratings might upset someone as a major flaw: what about a player achieving the rating they require for a GM title in a month when they do not play any games? On the other end, others might even find this unpredictability as an advantage, stopping players from worrying too much about rating at every game and forcing them to just play their games. Personally, I'm not sure which side I'm on for this issue.

Roger de Coverly
Posts: 17982
Joined: Tue Apr 15, 2008 2:51 pm

Re: Universal Rating System

Post by Roger de Coverly » Wed Jan 04, 2017 11:27 am

Paolo Casaschi wrote: using more relevant data makes your statistical model more accurate
What are you trying to model though? If it's prowess as a chess player, it makes sense to include all forms of chess. But why not correspondence play as well to bring in the ability to choose between engine moves? If it's prowess at a particular speed of chess, you reduce the reliability of the results by introducing partly irrelevant data.

It's an interesting if complex idea that a rating result for a game can change retrospectively. It's a feature of sorts in the ECF system where your performance against new players and junior players treated as new players can be changed by games not yet played.

Their rating system will lack acceptance until they publish the detail workings. Otherwise there might be a secret formula that adds a few points for players approved of by Kasparov and subtracts points for those who don't.

Richard Bates
Posts: 2878
Joined: Fri Nov 14, 2008 8:27 pm

Re: Universal Rating System

Post by Richard Bates » Wed Jan 04, 2017 11:30 am

Leaving aside the theoretical merits of combining differing time controls (of course FIDE already do this because the faster 90' + 30" is very different to the longer versions, and even more so when factoring in two or three rounds a day on the back of it), I would suggest that a potential flaw in the idea is that it has to make an assumption that players are (attempting to) play at their true/maximum capability in all versions. The reality is that many will distinguish between the three and will see the faster time controls as an opportunity for experimentation and/or approach them as having a "social" aspect. Should a Blitz tournament played at 8.30 in Hastings, immediately after a long classical game, and possibly having taken in some liquid sustenance, really feed into a universal list? This is similar to some arguments against, for example, FIDE rating evening league chess in the UK, except even more so.

Further more it is the nature of rapid/blitz competitions, that the large number of games played in a short period will create a linkage between performance in early rounds and performance in later games. Of course "collapses" happen in slow play events as well, but probably not to the same extent.

Post Reply