Cheating in chess

[Matthew Turner]

I think you have to be quite careful about all of this, because each of us comes at it from our own perspective. Let me offer a question which I am not sure people will have considered

What proportion of players will have reliable Wi-fi in their own homes?

[MartinCarpenter]

Yes, its just hard. I do find the idea of effectively setting out to catch only the casual cheats a bit 'off' somehow, but obvious practical reasons.

As for Wifi? Most vs Ethernet on a landline I think. Some won't have good internet at all of course but they're already excluded by it being an online event.

There will be some people with no landline internet and 'just' mobile phones though, especially world wide. I don't think you can screen share on a mobile?

The regulations are basically assuming you're playing on a reasonably modern laptop with rooms free of obvious electronics.

I've got a desktop and a variety of mobile devices here, but nothing that would suit.

[Matt Bridgeman]

I think you have to be quite careful about all of this, because each of us comes at it from our own perspective. Let me offer a question which I am not sure people will have considered

What proportion of players will have reliable Wi-fi in their own homes?

Not that we’re that interested in the event, but with an open plan downstairs and unreliable WiFi upstairs, we’d have to vacate the whole downstairs area and hope that they wouldn’t mind the camera not covering all the exits so to speak. Plus I’d presume the huge smart tv in the corner would be an electrical device! Lol Roll on a post-vaccine world I say!

[Jacob Ward]

I don't get the argument. It is similar to saying: with this measure, 1% of cheaters will still be able to cheat, hence this measure is horrible. Let's not implement it, and have 100% of cheaters cheat, because is a measure is not perfect, it's not worth having.

I think you can make a more persuasive case than that. I am undecided about these types of measures, but one way of looking at it is:

1. This will deter or catch unsophisticated cheats, who would probably have been caught anyway
2. It is not difficult to evade, so a cheater sophisticated enough to evade the statistical tests will find a way to evade this as well (as demonstrated by forum members finding ways round it in a few minutes)
3. Meanwhile, a certain number of innocent players will be put off by the requirements or disqualified when a spouse or family member walks into the room. These measures all have costs for innocent players.

I don't agree as:

1.a- those are the vast majority of cheats.
1.b- those are not caught when they get human help (e.g a 150 or 130 helping a 130)
1.c- they are caught AFTER the cheating is done, which means the rankings are impacted, the honest players' experience is soured, etc... It's the whole point of having a deterrence/detection upstream that can weed out most of the cheats.

2.a- I disagree, for most opportunistic cheats, this will deterrence enough. I think at the French junior championship, the cheating ratio was about 0.7% with these measures. Instead of god knows how much in tournaments with no audio/video/desktop monitoring.
2.b- What can you do if 1%, 5% or 10% are determine cheaters? Nothing. You at least make their life more complicated, and hope they get caught with the post game analysis. And as I said, if we prevent 80% of cheating, it's still better than having 100% of cheats.

3- that is indeed the main problem. Some will not be able to play with these measures. But the alternative is a cheat infested tournament, which nobody will enjoy. So it is better to have 100% of players play in a tournament infested with cheats, or 80% play in a much cleaner tournament, where only the very determined cheaters play hide and seek with the analytical reviews.

These measures work, have been in use in organised tournaments in England, Spain, etc..., used by foreign federations, now used by FIDE.
Yet some players still prefer no measures and rampant cheating rather than less cheating.

I think everyone agrees that more measures are likely to reduce cheating, but also that the more measures you put in place, the more people will be put off. I assume it's also generally agreed that there are diminishing marginal returns from adding measures on top of measures.

What is needed is some actual figures so that we know what measures are worth using - for example, do video-monitored events result in catching cheats using the video who would not be caught by the stats, or conversely catching fewer cheats overall (which might indicate a deterrent effect)? I've never seen any tournament attempt that sort of analysis, and without those figures I think it is very hard to know whether video is worth it for the reduction in cheating. I do know that, speaking only for myself, I find the idea of playing classical games alone in a room on video sufficiently unpleasant that it puts me off entering anything. I prefer the risk of playing a cheat but in comfort over a more secure but more invasive atmosphere.

I wonder if that sort of monitoring of tournaments to identify what works and what doesn't is a job for governing bodies?

[Wadih Khoury]

What is needed is some actual figures so that we know what measures are worth using - for example, do video-monitored events result in catching cheats using the video who would not be caught by the stats, or conversely catching fewer cheats overall (which might indicate a deterrent effect)? I've never seen any tournament attempt that sort of analysis, and without those figures I think it is very hard to know whether video is worth it for the reduction in cheating. I do know that, speaking only for myself, I find the idea of playing classical games alone in a room on video sufficiently unpleasant that it puts me off entering anything. I prefer the risk of playing a cheat but in comfort over a more secure but more invasive atmosphere.

I wonder if that sort of monitoring of tournaments to identify what works and what doesn't is a job for governing bodies?

UKCC: for most of the tournament, the only measured were lichess detection and Regan's software.
In an interview, it was claimed that there has been an estimated 100 cheater out of 2000 participants at the UKCC. This is a 5% cheat ratio.

French online junior championship: audio, video, screen sharing and software analysis.
The federation found 5 cheaters out of 989 players. This is 0.5% cheat ratio.

So here you are: measures on a similar junior population resulted in a reduction of cheating by a factor of 10.
With no proactive measures, over 9 round a UK junior had 37% chance of meeting a cheater. One in 3 will have had a soured experience.
With measures in place, they would have had 4% chance of meeting a cheater.

Well worth the effort and the inconvenience I would say.

"The solution should never be to do nothing". Taken from the french federation's debrief of the tournament.

[Ian Thompson]

UKCC: for most of the tournament, the only measured were lichess detection and Regan's software.
In an interview, it was claimed that there has been an estimated 100 cheater out of 2000 participants at the UKCC. This is a 5% cheat ratio.

French online junior championship: audio, video, screen sharing and software analysis.
The federation found 5 cheaters out of 989 players. This is 0.5% cheat ratio.

Lies, damned lies, and statistics.

If we assume those numbers are accurate, or at least the proportions, I don't think too many conclusions should be drawn without more information.

The UKCC is likely to have consisted of mainly casual players who never have, and never will, play chess other than for fun and may not care much if they get caught cheating. The French event might, and I don't know, have consisted of more serious players who would care more if they got caught.

The consequences of getting caught in the UKCC are minimal. They might, and again I don't know, be more significant in France.

[Wadih Khoury]

Lies, damned lies, and statistics.

If we assume those numbers are accurate, or at least the proportions, I don't think too many conclusions should be drawn without more information.

The UKCC is likely to have consisted of mainly casual players who never have, and never will, play chess other than for fun and may not care much if they get caught cheating. The French event might, and I don't know, have consisted of more serious players who would care more if they got caught.

The consequences of getting caught in the UKCC are minimal. They might, and again I don't know, be more significant in France.

Why would you assume they are not accurate? I can provide links if it helps for bot sets of numbers. (The UKCC one was in the Longson interview in the Guardian, the FFE one was in their debrief of the tournament)

Both pools of players were very similar. Ratings went from 700 national (sub 50 ECF level I would say) to Masters. One game you could be facing a junior hanging his queen, the next you could be playing the top of the age group.

Whether they cared getting caught was probably due to the camera. It's easier to cheat without being watched. The same way some would cheat in an exam without any teacher present, but would not attempt it if one was there.

[Roger de Coverly]

(The UKCC one was in the Longson interview in the Guardian, the FFE one was in their debrief of the tournament)

The UKCC have gone public with some estimates and also pointed a finger of blame at "posh kids". The 4NCL and ECF haven't been especially forthcoming about what they may or may not have detected during the events they have run.

[Ian Thompson]

Why would you assume they are not accurate?

Because there's no way of knowing how many people might have got away with it, or how many false positives there might have been.

[Joseph Conlon]

In terms of the UKCC, I think it's a matter of public record that at least three of those who would have qualified for the terafinal knockout stage (i.e. were in the top 4 following the first day of the terafinal) got lichess bans and were disqualified between the first and second day of the terafinal.

But let me also say why I am not sure that the online world juniors is such a good idea, even with videos and arbiters. The worry is that this is hanging a very large prize (a world junior title, even if preceded by an 'online' moniker) by the very thin thread of human frailty and temptation. It is one player from each federation, so the players participating are by some definition the best junior in a country across two age groups - not really the kids you want to leave the game.

Now suppose (as almost certainly will happen) some of these players, whether under pressure from their parents (who also suffer from human frailty) or not, succumb to the desire to hide a phone under their seat and consult during the games, and are then caught and publicly thrown out of the tournament in disgrace. Will they ever play again?

And even if a player wins cleanly, it will always be an asterisked title, as how can anyone truly know that a player was clean? With stakes this high, is 5 in 1000 an acceptable cheating rate - indeed is anything higher than 0% acceptable?

You may recall chesskid.com organised a knock-out tournament of some of the best preteens in the world, with them on camera throughout, and one player was busted. Will that player (who had set records for being the youngest in their country ever to achieve a master title) ever play again? Or will they quit the game in shame? Was that online tournament a net win for chess or a net loss?

[Matt Bridgeman]

In terms of the UKCC, I think it's a matter of public record that at least three of those who would have qualified for the terafinal knockout stage (i.e. were in the top 4 following the first day of the terafinal) got lichess bans and were disqualified between the first and second day of the terafinal.

But let me also say why I am not sure that the online world juniors is such a good idea, even with videos and arbiters. The worry is that this is hanging a very large prize (a world junior title, even if preceded by an 'online' moniker) by the very thin thread of human frailty and temptation. It is one player from each federation, so the players participating are by some definition the best junior in a country across two age groups - not really the kids you want to leave the game.

Now suppose (as almost certainly will happen) some of these players, whether under pressure from their parents (who also suffer from human frailty) or not, succumb to the desire to hide a phone under their seat and consult during the games, and are then caught and publicly thrown out of the tournament in disgrace. Will they ever play again?

And even if a player wins cleanly, it will always be an asterisked title, as how can anyone truly know that a player was clean? With stakes this high, is 5 in 1000 an acceptable cheating rate - indeed is anything higher than 0% acceptable?

You may recall chesskid.com organised a knock-out tournament of some of the best preteens in the world, with them on camera throughout, and one player was busted. Will that player (who had set records for being the youngest in their country ever to achieve a master title) ever play again? Or will they quit the game in shame? Was that online tournament a net win for chess or a net loss?

Perhaps not so much in the British Isles, but in some countries a big performance in this tournament will probably open a door for funding, government sponsorship and special chess programs and the like. The temptation to fiddle some of the results must be huge. The traditional method of a hidden earpiece, covert camera and an assistant with a well-tuned suboptimal engine would probably beat the measures quite easily, if they were determined to take that path.

[Kevin Thurlow]

"Now suppose (as almost certainly will happen) some of these players, whether under pressure from their parents (who also suffer from human frailty) or not, succumb to the desire to hide a phone under their seat and consult during the games, and are then caught and publicly thrown out of the tournament in disgrace. Will they ever play again?"

It partly depends on their Federation's attitude. Some get angry and ban them for disgracing their country. Some wring their hands and say, "Well, they're only juniors. They didn't understand", and then take no action at all.

[JustinHorton]

And what would be the best approach to address the problem Matt identifies

[Kevin Thurlow]

"And what would be the best approach to address the problem Matt identifies"

I don't know - if you can definitely prove someone cheated, there has to be some sort of ban. But it is unacceptable to ban someone when they're innocent. As has been said earlier, it is relatively easy to catch the people who use an engine all the time, but if they're skilful enough to only use it occasionally, that is obviously a problem...

[Wadih Khoury]

Because there's no way of knowing how many people might have got away with it, or how many false positives there might have been.

That's a bit of a strawman argument! "We should not hold OtB tournaments nor attempt to implement counter measures because there's no way of knowing how many people might have got away with cheating".

• On the specific argument of false positive, I found confirmation that both UKCC and the French federation used Prof. Regan's software as the last barrier. Hence they should have broadly the same ratio of false positives (the french may capture cheats on video/audio evidence, but the effect is quite small, especially on their sample)
• On the argument of false negatives, I do not know whether Prof Regan's analysis has 1% or 99% false negatives. It is the international gold standard currently, and few methods (beyond catching someone red handed) are better than it. However, since both tournaments used it, it should not matter in any comparison between both tournaments
• If we assume that false negatives and positives are roughly identical in both tournaments (they should be a bit better with audio/video), then we do get that the probability of a cheater is 10x higher in the UKCC than the FFE tournament.

Overall, I do not therefore believe the order of magnitude in my first analysis to be materially incorrect: adding audio/video/screen controls reduces the level of cheaters by a factor of 10 (whether it's 8 or 12 is irrelevant to the discussion. The improvement is significant).

Is it perfect? No. Are OtB anti-cheating measures perfect? No. But people still play OtB in spite of knowing that someone might be checking the position in the toilet, or having a potentially rigged cap.

I do not understand why there is such a resistance to organising OtB, to organising robust online events. Almost as if some players would rather see chess die and other players be deprived of the fun and experience, rather than see it played in a format they do not personally like.

The one argument that does resonate is the injustice of false positives. Unfortunately, if you are not caught on camera/screen share, there is only an analytical presumption which can be incorrect.

Interestingly, out of the 5 cases in the FFE tournament, the highest profile one (a winner) is still pending appeal. There was nothing against her on audio or screen, some oddities on video, but she failed spectacularly the Regan test. Why? She beat 2 players 2100+ while having a rating of 1303 FIDE rapid. Part of the appeal argues that her real level was more around 1800, and using this number in Regan's analysis reduces the anomaly level.
Worth noting that the appeal commission also thinks that Prof Regan's analysis on its own may not be sufficient proof, and that Master analysis of the game does not constitute additional analysis to Regan's software. Will be interesting to see what happens.

So yes, not sure how to handle the risk of false positives, but the alternative cannot be "do nothing". And until we resume OtB, we cannot let serious chess be just the privilege of Magnus and friends. Everyone is entitled to play serious chess, without rampant cheating. There will always be some cheating, OtB or online, but for that we need to accept robust counter measures. An imperfect solution is better than no solution, especially if it does not increase false positives (it does not).