British Blitz Championships

[Jonathan Bryant]

There's probably a very different standard of proof used by LiChess/chess.com etc vs what you'd really want to accuse someone in person.

Totally agree. And with Mick’s point above

I wasn’t suggesting the ECF should auto ban just because chess.com did. Or even that they should ban at all necessarily.

I was just saying that the ECF have to do and say something. They surely can’t just pretend it didn’t happen?

Or maybe they can. We shall see

[Jaimie Wilson]

Well I thought the event was a lot of fun. Overall, I'm quite relaxed and enjoying the little bit of normality events like this and 4NCL Online are providing.

One thing that does worry me is this issue of 'false positives' which innocent people can potentially get caught up in. I'm not sure how often this happens but I feel that the important issue of online cheating should be treated separately for this reason.

[MartinCarpenter]

The false positive rate is really hard to estimate, because to calculate it you need to ‘know’ the prevalence of cheating in the overall population. Much looser standards to accuse someone if there’s 10% cheating than if its 1 in a million!

We just don’t have that bit of data, and its really hard to really think how to get it.

[Li Wu]

It's hard for us to get it sure, but chess sites have the data. As long as they have some record of "clean" human play as control to analyse with (which they do).

I'm pretty sure it's extremely small.

[JustinHorton]

You would like to think so, but I'm not going to believe it until we're given something more substantial than "like to think".

(Also, do they have the false positive rate? How would they know?)

[MartinCarpenter]

They can’t (I think.). They can calculate the probability of a given game being produced by a human by chance vs with engine assistance with decent confidence intervals.

That absolutely (epidemiology 101) doesn’t give you the false positive rate. At the extreme, no cheaters in the tested population means an automatic 100% false positive rate.

In practice you have to decide roughly how many cheaters you think there are in your population and what false positive/negative rates you’ll accept. Then you calibrate the thresholds in your statistical methods to match.

These assumptions are actually very much something they should be happy disclosing.
(Pedantically, I guess they might have a known minimal number of cheaters from people they’ve caught absolutely red handed rather than via statistics)

[JustinHorton]

In practice you have to decide roughly how many cheaters you think there are in your population and what false positive/negative rates you’ll accept. Then you calibrate the thresholds in your statistical methods to match.

What could possibly go wrong

[Nevil Chan]

In practice you have to decide roughly how many cheaters you think there are in your population and what false positive/negative rates you’ll accept. Then you calibrate the thresholds in your statistical methods to match.

What could possibly go wrong

Not a lot. That said, I decided to throw last two games on Saturday just to be on the safe side.

[DavidWalker]

There's probably a very different standard of proof used by LiChess/chess.com etc vs what you'd really want to accuse someone in person.

Note also that if a player is banned from some site, this doesn't necessarily tell you much about the probability of them having cheated.

Suppose 1,000 players join a site and over the time that they have an active account, 5% of the players cheat. Also assume a 1% false positive rate by the anti-cheat algorithm over the entire lifetime of an account. Note that if an honest player maintains an account for 5 years, this false positive rate is equivalent to a monthly false positive rate of about 1 in 5,970.

Therefore, if all cheating players are caught, 50 cheats are banned, along with 1% of the remaining 950 honest players - say 10 honest players banned to keep the numbers round. That means that 60 players are banned in total, but 10 of then were honest, even if the algorithm is perfect at detecting cheats and has a small monthly probability of a false positive.

If the false positive rate is higher, or the number of actual cheats lower, then it is possible for the population of banned players to include more honest players than cheats.

[Paul Cooksey]

David's assumption of 1 false positive to 5 cheats surprises me. I would probably have gone 1 to 100 on the anecdotal evidence.

[Li Wu]

To be completely fair there is also a huge scale of cheating.

If we are talking about ANY cheating at all, then the prevalence of it has to be higher than 15% (someone quoted ICC numbers from before). It's not apples to apples, and both detecting and punishment should be a sliding scale.

In JH's case, it makes no sense to punish someone "cheating" at daily chess the same way as someone cheating in "titled tuesday", or just any live chess. Underlying prevalence is different, level of maliciousness is different, "detectability" is different. Someone that shakes your hand in otb chess and goes through the trouble of hiding devices in toilets is another level of maliciousness. If said person is a chess professional doing well in life cheating against his colleagues should be dealt the worst kind of punishment.

But sites are not evil. They decide on a level of depth/detail to their rules and enforce it. I think most players who think they know better would do a much poorer job.

[JustinHorton]

But sites are not evil. They decide on a level of depth/detail to their rules and enforce it.

Yeah, but they also decide on how far they're going to risk mistakes and how they're going to behave when that happens, and in practice that's a pretty cynical reckoning. What the relationship is between cynicism and evil is a larger matter.

[NickFaulks]

In JH's case, it makes no sense to punish someone "cheating" at daily chess the same way as someone cheating in "titled tuesday", or just any live chess.

But the point is that they did precisely that. They then insisted for ages that he was just a whingeing cheat and, when finally forced out of that position, offered what he says was a lame apology. There was no suggestion of "we've clearly got something systemically wrong and need to deal with it".

But sites are not evil.

I'd like to believe that too. So why do you think Justin's case was handled in the way it was?

[JustinHorton]

offered what he says was a lame apology.

To be fair I think the apologies were fine as far as the individuals who gave them are concerned. As far as chess.com itself (and specifically Danny Rensch) is concerned, not so much.

[Li Wu]

They did that since it was easier and cheaper to do so.

From a purely business perspective- having a proper system in place to handle Justin's case and others like it properly would have made a net loss on their site. Human review system and customer support isn't free- about as cynical as I would be here.

Note because of the same reason- banning ppl with high false positive rates is very bad for their business- they would lose potential paying customers with not much upside (not banning player X who might be cheating wouldn't hurt playing experience of rest of the site enough to offset potential revenue loss). So in terms of false positive rate- I can't see this as being high at all. But review process is going to be sh**ty for everyone except titled players (where they are likely even more careful of false positives in the first place).