4NCL Online

[Matthew Turner]

I do worry a tiny bit about aiming for multiple appeals given how people seem to react so far. Its obviously quite an emotive thing.

I'm not surprised that Ken is being sensible 25 really isn't much leeway though, not when you've shifted the playing conditions from >4 hours FTF to somewhat quicker online from homes. Most of the population will obviously play slightly worse due to that.

I would also expect a small but real %age of the playing population to notably improve their absolute underlying playing strength.
(Only +100 or something, not +5/6!)

If you're being cautious I think you'd want to take that effect away. It shouldn't weaken the power of the statistics too much.

Martin,
It is all about determining how many false positive you are prepared to accept. Increasing everyone’s rating or looking for a higher z score to ‘prove’ cheating amounts to the same thing. Time will tell whether the balance is right. I think it is clear that a Ken takes a conservative approach and I haven’t seen anything as yet to suggest the balance is wrong (that is just my opinion of course).

[Matthew Turner]

The calculations that Ken Regan does initially sets the ratings at 25 points higher than the actual rating. A further adjustment is made for lower rated players, but I am unclear as to what that is.

Just to be clear is that 25 ECF or ELO? 25 ECF is equivalent to ~200 ELO currently?

25 ELO

[MartinCarpenter]

Martin,
It is all about determining how many false positive you are prepared to accept. Increasing everyone’s rating or looking for a higher z score to ‘prove’ cheating amounts to the same thing. Time will tell whether the balance is right. I think it is clear that a Ken takes a conservative approach and I haven’t seen anything as yet to suggest the balance is wrong (that is just my opinion of course).

All true. I'd be a bit worried by this though. 4NCL online is a very different set of playing conditions to those under which people gained their FIDE ratings.

The time limit is also slow enough that you would probably expect a small %age of people to play an actively better quality of moves. Anyone who does this is obviously much more exposed to a false positive than the rest of the population.

They probably deserve systematic protection, not least because its such an easy - and effective - route for someone to take on appeal, good faith or otherwise.

[Matthew Turner]

Martin,
This is all covered by natural variation, of course I might prefer Blackpool Congress to St Albans or 4NCL online to 4NCL OTB and might play better as a result, but that is just natural variation. I don't suddenly become the equivalent of 800 FIDE rating points better at playing computer moves.
Of course, I might perform very well in the Online 4NCL because I win some games through my opponent disconnecting, or knocking back a case of San Miguel and I might have a stellar rating performance, but that is not the same as selecting computer moves with a high degree of accuracy.

[MartinCarpenter]

No I explicitly don't mean natural variation. People consistently playing at a different performance level in different events/playing conditions is often systematic, not chance.

Of course it won't make people play 800 points higher. What it will do is make some people play 1-200 pts higher.
(Especially when the grade used to predict performance contains no games played under those conditions.).

If you don't incorporate that then it really messes with your statistics. Especially if you've reached the happy point of agreeing with the captains that you want to convict at X point, with Y% false positives. Because then all someone needs to do is claim this and you morally have to acquit them.

[Adam Raoof]

Martin,
It is all about determining how many false positive you are prepared to accept. Increasing everyone’s rating or looking for a higher z score to ‘prove’ cheating amounts to the same thing. Time will tell whether the balance is right. I think it is clear that a Ken takes a conservative approach and I haven’t seen anything as yet to suggest the balance is wrong (that is just my opinion of course).

All true. I'd be a bit worried by this though. 4NCL online is a very different set of playing conditions to those under which people gained their FIDE ratings.

The time limit is also slow enough that you would probably expect a small %age of people to play an actively better quality of moves. Anyone who does this is obviously much more exposed to a false positive than the rest of the population.

They probably deserve systematic protection, not least because its such an easy - and effective - route for someone to take on appeal, good faith or otherwise.

As I said. Ken does adjust for time controls, for theory, for forced moves, for age.

[MartinCarpenter]

That's not at all the same thing that I'm talking about here.

[Ian Thompson]

Just to be clear is that 25 ECF or ELO? 25 ECF is equivalent to ~200 ELO currently?

25 ELO

That's less than the average variation in performance due to playing White or Black, so surely inconsequential.

[Thomas Rendle]

Of course it won't make people play 800 points higher. What it will do is make some people play 1-200 pts higher.
(Especially when the grade used to predict performance contains no games played under those conditions.).

It might make people PERFORM (TPR) higher, but absolutely not PLAY (% computer top moves) higher. The idea that someone consistently plays better moves online and at a faster time control, compared to over-the-board doesn't make any sense.

[Mick Norris]

Martin,
It is all about determining how many false positive you are prepared to accept. Increasing everyone’s rating or looking for a higher z score to ‘prove’ cheating amounts to the same thing. Time will tell whether the balance is right. I think it is clear that a Ken takes a conservative approach and I haven’t seen anything as yet to suggest the balance is wrong (that is just my opinion of course).

All true. I'd be a bit worried by this though. 4NCL online is a very different set of playing conditions to those under which people gained their FIDE ratings.

The time limit is also slow enough that you would probably expect a small %age of people to play an actively better quality of moves. Anyone who does this is obviously much more exposed to a false positive than the rest of the population.

They probably deserve systematic protection, not least because its such an easy - and effective - route for someone to take on appeal, good faith or otherwise.

There's the not driving for a couple of hours to get to the 4NCL/county match in my case versus the question of whether I take it less seriously at my computer

[MartinCarpenter]

Of course it won't make people play 800 points higher. What it will do is make some people play 1-200 pts higher.
(Especially when the grade used to predict performance contains no games played under those conditions.).

It might make people PERFORM (TPR) higher, but absolutely not PLAY (% computer top moves) higher. The idea that someone consistently plays better moves online and at a faster time control, compared to over-the-board doesn't make any sense.

For the overall population, yes, obviously you're right.

For a small subset of that population then why not? This isn't 5 minute online blitz, its still quite a 'workable' time limit.

There's no travel. They're at home, maybe that helps them mentally. Maybe they've basically grown up playing on computers vs boards. Maybe they simply don't have the mental stamina to sustain their quality of play for more than 2 hours. Maybe just inspired by playing in a new competition.

People are really odd things.

That goes especially strongly if Ken's algorithm (very reasonably) automatically slightly degrades the expected quality of peoples moves for the slightly shorter time limit.

[Thomas Rendle]

There's no travel. They're at home, maybe that helps them mentally. Maybe they've basically grown up playing on computers vs boards. Maybe they simply don't have the mental stamina to sustain their quality of play for more than 2 hours. Maybe just inspired by playing in a new competition.

People are really odd things.

That goes especially strongly if Ken's algorithm (very reasonably) automatically slightly degrades the expected quality of peoples moves for the slightly shorter time limit.

Yes. But from what I understand there are players in the online 4NCL performing higher than ANY player from the OTB 4NCL seasons. By all means make a small adjustment if you want, but it will pretty insignificant to the kind of performances we're seeing.

[Li Wu]

Back from a break from the forums/chess.

@Martin- this stuff can all be analysed and improved upon. I've done it myself in my job. Test an idea, then it comes back inconclusive- so ignore it, otherwise incorporate it. You are describing some effects that I personally doubt, but could be there. Ken's algorithm (and other detection methodologies) is not set in stone, and being an IM at chess with a professional understanding with a history of published and unpublished work on the subject, doesn't he deserve some degree of assumed competency on the subject?

It's always easy to attack a fixed target by coming up with potential whatifs. What is a detection method you (not you specifically, but all naysayers in this thread) are happy with? Are all statistical methods untrustworthy in your view? 4-sigma not good, what about 5-sigma?

If you can't come up with a solution, and don't believe 4NCL online chess is viable in its current form and can't be persuaded, then no real argument can be had.

[Nick Burrows]

In Div 2, Exeter are in the QF despite finishing 3rd behind Wigston who are omitted?

[MartinCarpenter]

Back from a break from the forums/chess.

@Martin- this stuff can all be analysed and improved upon. I've done it myself in my job. Test an idea, then it comes back inconclusive- so ignore it, otherwise incorporate it. You are describing some effects that I personally doubt, but could be there.

Ken's algorithm (and other detection methodologies) is not set in stone, and being an IM at chess with a professional understanding with a history of published and unpublished work on the subject, doesn't he deserve some degree of assumed competency on the subject?

I'm presuming that his work is basically perfect.

The issue under immediate discussion affects the input data. Its definitely real, and relatively easy to fix. Not sure of the magnitude.

There's a given set of grades based on 4+ hour OTB games. This is being used to predict performance in Internet, ~2 hour(?) games.

Ken's algorithm will be internally presuming a constant degradation derived from population statistics in the move performance of each player. ie at player at grade X from set 1 will play at move quality Q at the longer games and Q - Z% with less time. That's absolutely the right thing for it to do.

What isn't right to use the grade X from the longer time limit as the test grade when checking for extreme over performance.

The effects on specific individuals are some way from linear and if you want to be remotely fair you need to avoid 'automatically' penalising people who do well vs the population as the time limits shift down.

So you give the person you're testing a generous mapping - something like the 95th percentile of people who retain their playing strength. That's the two hypothesises you're testing against anyway of course, person with good season, liking time limit, conditions etc vs person cheating.

This would be really obvious/ruinous if you tried using long play grades to predict blitz performance, obviously much smaller magnitude of effect here.