Cheating in chess

[Li Wu]

This is by definition what this sort of evidence is. It assigns a probability that such moves can be produced by any human/Magnus Carlsen/the player in question.

That's where the interpretations and assumptions come in. There's an underlying assertion that rating calculations are reliable down to the level of individual moves. A hypothetical player who could match GM choices 19 moves out of 20 but puts a queen or equivalent en prise on the 20th would have a modest rating as results would be poor.

i suppose there are two ways this could get to courts. One is where FIDE or a national federation attempted a publicised ban and this was hotly contested. The other is a dispute between players where accusations were made and the accused party took civil action for defamation etc.

The moves are analysed for all players. Rating calculations are ~reliable down to the level of individual moves only as a probability distribution not an exact prediction. Player to player deviation/variance can be taken into account, and doesn't render the analysis useless. In any case we can use a player such as Carlsen (or pre 2000 correspondence games) for analysis rather than the player's rating.

There is a line, and at some point on that line I say we have evidence to convict. Are you just arguing for more added variance in these calculations, or are you disputing the general concept of statistical analysis itself?

[JustinHorton]

Roger can answer for himself, but I personally might be arguing for more science.

[Roger de Coverly]

Rating calculations are ~reliable down to the level of individual moves only as a probability distribution not an exact prediction.

Can that assertion be proved? Particularly for the International Elo system it's known that the rating can be several hundred points incorrect as a measure of strength under various circumstances. A rating itself is an estimate. When describing the system, Elo expressed the opinion that "chess strength" was a random variable, it being possible to estimate the mean of the distribution from observed results.

[Li Wu]

I actually agree with you that FIDE ratings are less reliable. When it comes to online rating- there's a lot more sample within a shorter time frame, making move correlation analysis have better predictability.

I don't know what to prove here tbh? we can choose some thresholds with a training data set, and show that 99.99% of players in a separate sample set fall in line with prediction? We can move the threshold to make it 99.999% as well, but obviously this means cheaters go on longer before detection (and reduce false positive rate at the same time).

[Matt Bridgeman]

Rating calculations are ~reliable down to the level of individual moves only as a probability distribution not an exact prediction.

Can that assertion be proved? Particularly for the International Elo system it's known that the rating can be several hundred points incorrect as a measure of strength under various circumstances. A rating itself is an estimate. When describing the system, Elo expressed the opinion that "chess strength" was a random variable, it being possible to estimate the mean of the distribution from observed results.

Now you’re getting to the nub of it. Some poor solicitor for the ECF/FIDE hasn’t even got anywhere near presenting Prof. Regan as his star witness and they’ve got to wade through a voir dire on a topic like this!

[Roger de Coverly]

Roger can answer for himself, but I personally might be arguing for more science.

i doubt there's ever been a blind experiment.

Take a group of trusted volunteers, give them secret instructions which selectively tell them to consult engines all the time, none of the time and various permutations of some of the time. Have then play some games and run the detection software to see how much or how little of the engine use it can spot.

[Kevin Thurlow]

"Particularly for the International Elo system it's known that the rating can be several hundred points incorrect as a measure of strength under various circumstances."

Rating systems aren't a measure of strength. I call Otto Clarke as expert witness. (For the younger readers, he invented the BCF grading system.)

[JustinHorton]

When it comes to online rating- there's a lot more sample within a shorter time frame, making move correlation analysis have better predictability.

It's a subjective judgement, but I do not think I find that lichess ratings are an accurate predictor of my opponents' playing strength compared to FIDE or ECF ones.

I don't know what to prove here tbh?

I don't know that I want people on here to try and prove anything. I would like people to distinguish between what they know and what they infer.

[Li Wu]

I have to say that this is a good idea to get around the fact that methodology can't be revealed.

In practice I don't know if there is enough funding in FIDE to do such a thing for OTB, and whether the cheated participants would be OK with this in various tournament/league games (if you inform them beforehand then it's no longer blind).

As for online- we would need to organise external testers that don't mind being labelled a cheat. And basically the main gain from such an experiment is to restore trust in the methodology?

[Li Wu]

^Referring to conducting a blind experiment.

[JustinHorton]

the fact that methodology can't be revealed

This isn't a fact, though, is it? It might be undesirable to do so, for all manner of very good reasons, but it's not a fact, and indeed without revealing the methodology there's not going to be any proper science.

[Matthew Turner]

the fact that methodology can't be revealed

This isn't a fact, though, is it? It might be undesirable to do so, for all manner of very good reasons, but it's not a fact, and indeed without revealing the methodology there's not going to be any proper science.

I have posted a video where a computer expert and a chess expert show the evidence in a case. They use a detection system which is freely available to everyone and the coding is all available online. That does seem to reveal quite a bit about the methodology, but apparently not enough for you to comment on it.

[NickFaulks]

In practice I don't know if there is enough funding in FIDE to do such a thing for OTB

Nothing to do with funding, obviously nothing of this sort could be done in a rated competition and what would be proved otherwise?

[JustinHorton]

That does seem to reveal quite a bit about the methodology

You understand that science doesn't involve "seeming to reveal quite a bit", yes?

[Li Wu]

the fact that methodology can't be revealed

This isn't a fact, though, is it? It might be undesirable to do so, for all manner of very good reasons, but it's not a fact, and indeed without revealing the methodology there's not going to be any proper science.

Sorry I just used those words for convenience. I will try to make distinctions between what I know and what I infer in future, but no guarantees!