4NCL Online

[JustinHorton]

Incidentally lichess appears to have just gone down, so that'll be interesting if it happens on Tuesday evening.

Also, something happened a bit earlier today that I've heard other people complain about - I moved, the clicks stopped, there was a screenfreeze, and when I was able to get back to the game I was told I'd left the game and lost. Which was OK for me because as it happened I'd just blundered, but again, you can't see it going down well if it happens tomorrow.

[Joseph Conlon]

I'm still struggling to understand how the Lichess model assesses quality of games. Their 'Insight' facility yields this result for one of my few games:

Learn from your mistakes

RogerLancaster +55
0 Inaccuracies
0 Mistakes
0 Blunders
5 Average centipawn loss

Should I assume from this that the game in question, which ran to 36 moves, is regarded as suspect? I hope not because, if it is, it's a nonsense - my opponent, and I trust he's not a member of this forum and so won't mind my saying this, was likely a near-beginner and I had little chance to go wrong. I'd appreciate clarification from someone who understands the assessment method better than I do.

One thing I do know is that in terms of inaccuracies, mistakes, or centipawn loss the analysis regards every move retaining an advantage beyond
a certain level (something like +10) as 'perfect'. So games where one side goes a queen up in the opening and then retains that advantage will result in a similar acpl score and a similar 0/0/0 error profile (even if the winning side blunders from Q+R up back to Q up).

So this will not be suspect as there are very many games like this, almost all of which involve a beginner on one side.

[Jonathan Rogers]

Incidentally lichess appears to have just gone down, so that'll be interesting if it happens on Tuesday evening.

Also, something happened a bit earlier today that I've heard other people complain about - I moved, the clicks stopped, there was a screenfreeze, and when I was able to get back to the game I was told I'd left the game and lost. Which was OK for me because as it happened I'd just blundered, but again, you can't see it going down well if it happens tomorrow.

After years and years of play being interrupted by fire alarms at 4NCL venues, any such incident would arguably give the online experience the authentic touch!

[Kevin Thurlow]

"After years and years of play being interrupted by fire alarms at 4NCL venues, any such incident would arguably give the online experience the authentic touch!"

Deep in a lair South of London, much Diet Coke is being consumed whilst the mastermind calculates how he can set off multiple fire alarms remotely...

[Nick Grey]

How many got bans last week?
Do not drink much Diet Coke. I'get out of my seat far too much to consider playing online.

[Roger Lancaster]

Before taking this any further, I'd like to check that my thinking on the probabilities here accords with that of others. This is a situation where I tend to expect Bayes Theorem to apply. For those who aren't mathematically inclined, I'll set out below three examples of how this operates. In each case, I'll assume that 95% of 4NCL/J4NCL players wouldn't dream of consulting computers while playing their games but that 5% are prone to cheat. The 95% figure doesn't actually matter - you can rework these examples after substituting 90% or 99% but you'll get to the same result.

1] Suppose there's a 5% error in the anti-cheating mechanism, meaning that 5% of those deemed to be honest are actually cheats and 5% of those deemed to be cheats are actually honest. Taking a population of 10,000, that means that 5% of the 9,500 honest players - that is, 475 - are singled out as cheats while 95% of the 500 real cheats - another 475 - are correctly identified as cheats. In total, the mechanism identifies 950 'cheats' but is wrong 50% of the time. That's obviously unacceptable.

2] Let's assume it's more accurate with just a 1% error. The mechanism now identifies 1% of the 9,500 honest players - that is, 95 - as cheats while 99% of the 500 real cheats - another 495 - are correctly identified as cheats. In total, the mechanism identifies 590 players as cheats but is wrong on 95 occasions - meaning every sixth case results in an injustice. That's an improvement but still not, I think most will agree, good enough.

3] To get to anything approaching "beyond reasonable doubt", we need to assume something like a 0.1% error rate, when the mechanism now identifies 9.5 of the honest players as cheats along with 499.5 of the real cheats. The injustices are now down to 2%, a figure which would probably satisfy some but not others. And not everyone will want to believe - I'm certainly sceptical - that errors occur only once in every thousand occasions.

Is there a fallacy in this reasoning? The above throughout assumes that the mechanism is equally likely to identify honest players as cheats as vice versa, which isn't a "given", and others may well have better and greater insight.

[Li Wu]

I think your reasoning is sound, and I do think the sites know this well and that they really go for "beyond reasonable doubt".

One can also make some educated estimates for rates of cheating beyond a random 5%.

There is some survivorship bias involved with older players less likely to be cheating (the only ones still playing in old age are ones that enjoy the game without being hyper competitive- some GMs IMs give up chess later on with no financial benefit/mental decline let alone normal players, and also they "survived" being caught from cheating for many years, so are less likely to be cheaters now). Note also that in general younger players are more risk-seeking in their behaviour.

So I would be extra careful with J4NCL (read- its going to be higher prevalence of cheating), and also not judge cheating from younger players as harshly as I would, say, a 40 year old GM.

[Matthew Turner]

Roger,
I can go on the Lichess website now and it tells me that there are 67,000 active players now and 33,000 games in progress. If we use your figures, what would that mean for the number of false positives that we would be getting each day? and is that really credible.

[Li Wu]

Also I think you can breakdown player population more than prone/not prone to cheat. Something like (I stress it's made up numbers):

-5% wouldn't dream of cheating and would go out of their way to avoid it
-5% wouldnt actively cheat but wouldn't switch off live streams of commentators
-?% some people in the middle somewhere
-5% would read consult opening books they tell themselves that they would have memorised eventually
-5% consulting engines sometimes
-5% consulting engines always, have no qualms with cheating

With maybe some overlaps in the above categories- and this is for online chess alone. Lichess will flag the last 2 or 3 categories and maybe some of the others (would prob need bigger samples).

[Li Wu]

@Matthew- someone can be flagged as 99% cheating in an alerting system and then put on some observation, where additional testing can happen (more resources and more depths of engine analysis for next few games for closer look say) that puts it to 99.99%.

It doesn't have to be a straight up simple formula for cheat/not cheat- you can go beyond that to build a whole process.

It would be very costly and inefficient if the entire player pool games are subject to 20ply stockfish analysis every move and centrally analysed for cheating. They must have a few alerting flags in place.

[Ian Thompson]

Roger,
I can go on the Lichess website now and it tells me that there are 67,000 active players now and 33,000 games in progress. If we use your figures, what would that mean for the number of false positives that we would be getting each day? and is that really credible.

Isn't the relevant figure the number of newly created accounts?

If we assume X% of players are cheating and cheats will be caught after playing Y games on average, we want to know how many of those 67,000 players are recently created accounts that have played less than Y games.

[Roger Lancaster]

Roger,
I can go on the Lichess website now and it tells me that there are 67,000 active players now and 33,000 games in progress. If we use your figures, what would that mean for the number of false positives that we would be getting each day? and is that really credible.

Matt, first thing to say is that I'm assuming that 4NCL players [and, to a lesser extent, J4NCL players] are less likely - by comparison with the wider population - to cheat because the downside is greater, notably, that any ban will be noticed by their team-mates.

But, if we assume that roughly 95% of those 67,000 players - say, around 64,000 - are honest then a 5% error rate would result in around 3,200 false positives, a 1% error rate would result in 640 and a 0.1% error rate in 64. If it's an automated process then, spread around the world, I'd guess any of those figures is possible - I assume that only a small proportion would arise in the United Kingdom.

[Roger Lancaster]

Roger,
I can go on the Lichess website now and it tells me that there are 67,000 active players now and 33,000 games in progress. If we use your figures, what would that mean for the number of false positives that we would be getting each day? and is that really credible.

Isn't the relevant figure the number of newly created accounts?

If we assume X% of players are cheating and cheats will be caught after playing Y games on average, we want to know how many of those 67,000 players are recently created accounts that have played less than Y games.

Yes, that's related to the survivorship bias to which Li Wu refers.

[Roger Lancaster]

@Matthew- someone can be flagged as 99% cheating in an alerting system and then put on some observation, where additional testing can happen (more resources and more depths of engine analysis for next few games for closer look say) that puts it to 99.99%.

It doesn't have to be a straight up simple formula for cheat/not cheat- you can go beyond that to build a whole process.

It would be very costly and inefficient if the entire player pool games are subject to 20ply stockfish analysis every move and centrally analysed for cheating. They must have a few alerting flags in place.

It seems to me very arguable that, if the accuracy were 99.99% [the most extreme of my three examples earlier was based on 99.90%] then that would very likely be acceptable. The reason I'm sceptical about 99.99% is the number of cases - Northumbria springs to mind as a likely example - where games are analysed by strong human players who can find nothing suspicious. That means either {a} a lot of 0.01% occurrences which by definition is highly unlikely [b} humans, allegedly expert humans, are wrong or {c} mechanism is wrong.

[Li Wu]

No disrespect to the "strong human players", but some things can be very counterintuitive in probability/stats, and have an understanding or review the data from thousands of human games (which lichess has, and I doubt most strong chess players have).

Each move not being remarkable and having a string of them- may not cause any alarm, but it should.

A famous example from statistics- a professor asks half of his class to flip a coin 200 times and record it, whilst the other half were asked to make it up. Then when they had in the results- he could always tell whether it was made up or not.

The reason? Students making up coin tosses never had 6 heads in a row or 6 tails in a row, when it's virtually impossible for the longest run to be less than 6 in 200 coin tosses.

https://www.csun.edu/~hcmth031/tlroh.pdf