Some thoughts on anti-cheating systems

[Mick Norris]

Mick,
You have identified a problem and I don’t know the answer. Maybe in years to come county matches will be a different format, I.e each player playing 6 games, so that it is likely that a player will build up enough evidence after one round to detect cheating.

The ECF online group stages will have between 3 to 5 matches, but of course that doesn't mean that any one player plays in each match, nor that anyone inclined to cheat would do so in each match

I guess if I play I'd just assume my opponents are on the level (they probably won't need to cheat to beat me )

[NickFaulks]

(edit) There's this stuff from around six years ago.
viewtopic.php?f=31&t=5085&p=138476#p138476 (/edit)

Reading some of that stuff again, it's the same arguments or similar arguments about the reliability of statistical tests as in this thread.

Certainly is. Looking back, it is disappointing to see how little we have moved on.

I remember Ken's presentation in Tallinn very well - it was a high point.

http://www.cse.buffalo.edu/~regan/Talks ... ssTalk.pdf

For some reason his laptop wouldn't talk to the local technology so he borrowed mine. I asked a mildly technical question, but it was evident that nobody else in the room had the least understanding of it, or therefore interest, so we moved on.

[Roger de Coverly]

It seems to me that something's not right if you're prepared to ban someone, but not prepared to go public about it.

Aren't their hands tied by the refusal of lichess to disclose why players are banned? In the world of physical sports, when players are banned from participation, the reasons are usually disclosed.

[NickFaulks]

(edit) There's this stuff from around six years ago.
viewtopic.php?f=31&t=5085&p=138476#p138476 (/edit)

Reading some of that stuff again, it's the same arguments or similar arguments about the reliability of statistical tests as in this thread.

Certainly is. Looking back, it is disappointing to see how little we have moved on.

I remember Ken's presentation in Tallinn very well - it was a high point.

https://cse.buffalo.edu/~regan/Talks/FI ... ssTalk.pdf

For some reason his laptop wouldn't talk to the local technology so he borrowed mine. I asked a mildly technical question, but it was evident that nobody else in the room had the least understanding of it, or therefore interest, so we moved on.

[Roger de Coverly]

Certainly is. Looking back, it is disappointing to see how little we have moved on.

Wasn't the idea that players could be sanctioned purely by statistic measures abandoned? What looks to have happened is that where an event has a full set of games available, that these are run through the Regan black box. If any players are flagged that gives arbiters of the next event a tip off as who to look out for in terms of suspicious behaviour, unusual attire, over attentive spectators etc.

[Roger Lancaster]

Roger,
If I understand you correctly, you are now talking about a hypothetical example which has nothing to do with anything that is happening in British Chess or anything that has happened historically. You’ve got me, I don’t know.

Matt - You evidently don't. I'm not discussing a hypothetical example but actual examples which occur hundreds, possibly thousands, of times each day when someone signs up to Lichess. Since Lichess frequently have no idea who that person is, they can have no idea of his or her playing strength. Without this, they can make no remotely accurate assumption of the 'expected value' in the statistics.

Roger, this has nothing to do with the 4NCL or the ECF where we have established grades for the players. Is what you describing a problem for Lichess, well in a word Yes. There are two points that I would make
1. Titled players are caught cheating, so your cheat would have to be adept enough to avoid using assistance to the extent that they would be flagged up regardless of their rating.
2. Major events are limited to titled players so everyone has an established OTB history

Now you have moved on to thinking about how Lichess can improve their systems for other events, I take it you are completely happy with the ECF fair play policy.

Matt, I started this thread because I had concerns about certain aspects of anti-cheating systems. That should be fairly clear from the subject-line.The 4NCL and the ECF don't themselves operate anti-cheating systems so the thread has little to do with them. I'm not sure I would go quite as far as your words "nothing to do with" because 4NCL, in particular, relies in the first instance on Lichess [meaning that, at least in the past, where someone was accused by Lichess and their appeal received the usual response, they were banned from 4NCL] where, as I think you now agree, there is an identifiable problem. In general, although I touched on a rare probable exception, there isn't an equivalent problem in over-the-board chess.

I agree with your two further points but my criticism of Lichess related to their treatment of 'unknowns' so titled players are a quite separate issue. Finally, there's an implication in your final paragraph that my thinking has somehow "moved on", ie. changed. Actually, it hasn't, I've been making this same point about Lichess for some little while here. The new ECF Fair Play policy is another matter altogether - it was published only a few days ago and, to be honest, it's the sort of thing I like to ponder at my leisure rather than rush to an immediate judgment.

[Roger Lancaster]

Now I must admit that although I have downloaded Ken Regan’s papers I haven’t (yet) got round to reading them, so I’m not really in the area of being able to do a proper critique of his method, but when I heard him going on about Z values in Tuesday’s webinar I realised that it is all based on the area under the tail of a normal (or similar statistical) distribution. These calculations are all very well when it comes to truly random physical events (coin tossing etc) but perhaps not to events based on human decision taking.

I've reproduced just the last paragraph of Mike's earlier post although the whole is worth a read. Ken Regan's presentation on Friday mentioned, and I'm here mainly addressing those who didn't take part in the webinar, a game between Carlsen and Anand. Some way through this game, Anand decided that the correct plan was to advance his a-pawn, which he did on moves n, [n+1] and [n+2]. It was actually the wrong plan, as a result of which all three moves were identified as blunders.

The essence of a normal distribution is that it relies on successive events being random or independent, which should certainly be the case if one is tossing a coin. In the Carlsen-Anand case, successive events plainly weren't random - Anand's thinking at move n governed his decision at move [n+1] and one or both of these governed his decision at move [n+2]. Logic, rather than statistics, suggests that - all other factors being equal - someone who makes three independent blunders during a game is playing worse than someone whose makes one false assessment which results in three closely-related blunders. Ken Regan's solution was to attach a lower statistical weighting, determined empirically, to Anand's blunders at moves [n+1] and [n+2].

The Carlsen-Anand case was a particularly obvious case of a very common phenomenon with which most players will be familiar - that of following a previous line of thought when fresh thinking was required - but identifying such occurrences from a bare scoresheet strikes me as very difficult indeed. In those cases where one can't identify, one can't compensate. That indicates some variation from a normal distribution - I can't say how much, and my totally unscientific guess is 'not very much' - but it's typical of situations where the human factor complicates statistically-based decision-making.

[Matthew Turner]

Roger,
I have a titled player account at Lichess and a normal account at chess.com.
I could give you access to my chess.com account which has an established history of playing at a reasonably high level and you could attempt to use your computer to mimic that level of play or push it higher. You would be cheating and you might not get caught, you could get away with cheating, but so what?

[Matthew Turner]

Roger,
“ The essence of a normal distribution is that it relies on successive events being random or independent,”
This is just wrong

[John McKenna]

Can't see anything wrong with this -

I think one big difference between OTB and online chess with regard to cheating is that when playing online they is the constant spontaneous temptation based on opportunity that simply isn’t there in OTB chess, particularly at slower time controls.

Somebody could go into a game with no preconceived intention to cheat, or indeed thought about what method of cheating might be feasible. And play absolutely fair and straight right up until the moment when that critical complicated moment arrives. Success or failure in the match may depend on it. It might even be a fifty:fifty call where any player or any strength may opt for one move or the other.

And at that moment, through some element of weakness deep in their character, they take the risk and access a computer to tell them the right path. There is no chance they will be caught, the moment will pass every casual or probably expert adviser by, and any doubt can always be explained away as “I just rolled the dice”. And there’s no protection beyond an individuals guilty conscience and even that’s not much good after the game.

Whereas in OTB chess a great deal of the anti-cheating work has focussed as much on prevention (denying the means to cheat and at least without drawing attention to unusual behaviour) as detection (based on computer analyses probability). And I would suggest that the former has been far more successful.

I think we’ve had discussions before about how much stronger a player (particularly somebody starting from a high base) might be if given access to a computer for a single move. Online chess simply can’t combat this issue.

I might also note that with OTB chess is has been generally accepted that computer algorithms are only enough to generate suspicion - the method of cheating must also be determined. Whereas to some extent the approach to online chess seems to have become that the method need not be determined (because in most cases it would not be possible to do so) it is enough that the means exists. Which creates an uncomfortable intersection in the Venn diagram of players potentially cheating at chess - those who are ultimately assumed to be innocent OTB in the absence of any discernible means to cheat, but assumed guilty in online chess because it is not realistically possible to create playing conditions to restrict the ability to cheat and push the burden of proof into the organisers.

[Roger Lancaster]

Roger,
“ The essence of a normal distribution is that it relies on successive events being random or independent,”
This is just wrong

See, for example, https://www.merriam-webster.com/diction ... stribution

Definition of normal distribution:
a probability density function that approximates the distribution of many random variables (such as the proportion of outcomes of a particular kind in a large number of independent repetitions of an experiment in which the probabilities remain constant from trial to trial) and that has the form
f(x)= and my keyboard won't permit repetition of the formula here where μ is the mean and σ is the standard deviation

[Roger Lancaster]

Roger,
I have a titled player account at Lichess and a normal account at chess.com.
I could give you access to my chess.com account which has an established history of playing at a reasonably high level and you could attempt to use your computer to mimic that level of play or push it higher. You would be cheating and you might not get caught, you could get away with cheating, but so what?

Sorry, Matt, but you'll have to explain how that is in any way relevant to the issue of calculating 'expected' performances from no data.

[Adam Raoof]

Roger,
“ The essence of a normal distribution is that it relies on successive events being random or independent,”
This is just wrong

See, for example, https://www.merriam-webster.com/diction ... stribution

Definition of normal distribution:
a probability density function that approximates the distribution of many random variables (such as the proportion of outcomes of a particular kind in a large number of independent repetitions of an experiment in which the probabilities remain constant from trial to trial) and that has the form
f(x)= and my keyboard won't permit repetition of the formula here where μ is the mean and σ is the standard deviation

Ken did point this out, and explained how he modified his algorithm to take account of this. In fact, I thought that was the whole point of his talk - he has convinced me that his methods at least are reliable. They are reliable enough to use in court.

[Richard Bates]

Can't see anything wrong with this -

There is a typo is the first paragraph

[Roger Lancaster]

Adam, in my original post I made a point of saying: "I don't for one moment doubt the integrity of Professor Regan, nor do I doubt that he has taken a scholarly approach to this. Whether, in 10 years time, it will still be regarded as the Holy Grail strikes me as rather less certain".

That's still my position and the slight qualification is because I don't doubt that, over the next decade, Ken will further refine his algorithm - in which case, with hindsight, the present version won't be seen as perfect. I was less complimentary about the Lichess and Chess.com equivalents largely on the grounds that, if they were assembled with comparable rigour to Professor Regan, there would be no need to hide them from view.