Cheating in chess

[Matthew Turner]

Because we were told it on the Zoom call with Professor Regan that you were part of

[NickFaulks]

Because we were told it on the Zoom call with Professor Regan that you were part of

I know he said that, but he has never been willing to give even the faintest hint as to how he does it. I assume you have a reason for believing that his methods, whatever they are, produce reasonable answers?

[Matthew Turner]

He uses a dependence matrix which we've commented on before. In simple terms the idea is that when we add up the number of computer matches so if you have three in a row it wouldn't be 1 + 1 + 1, but instead say 1+0.85 +0.83
I commented that I wasn't sure that this modification was actually necessary, you didn't appear to express an opinion.

[NickFaulks]

I don't remember that. Just to be clear, this is the conference I paid 10 euro to attend?

[Geoff Chandler]

I read it as, "Aaaagh!!! it's a super-GM, we had better be careful. This is an 1800, who cares, just ban him."

That is how I read it as well. I wonder many totally innocent players have been 'found guilty.'
No right of appeal, no proof, just those unlucky enough to rounded up as part of this month's randomiser.
(The randomise in this case is set to take out only under 1800 players.)

It's kind of like Benjamin Franklin in reverse: (let's call it the Franklin Benjamin rule.)

Not: "it is better a hundred guilty persons should escape than one innocent person should suffer.

It's: "Better we ban 100 players to makes sure we get a guilty one (who ever that might be.)

[NickFaulks]

He uses a dependence matrix which we've commented on before. In simple terms the idea is that when we add up the number of computer matches so if you have three in a row it wouldn't be 1 + 1 + 1, but instead say 1+0.85 +0.83
I commented that I wasn't sure that this modification was actually necessary, you didn't appear to express an opinion.

I have an idea that you are talking about the meeting of 4NCL captains, which I was not eligible to attend and the record of which is lost. Am I right?

[Matthew Turner]

The meeting we both attended

https://www.chesstech.org/events/2020/c ... -cheating/

My comments about interdependence are included in this thread

[John McKenna]

The meeting we both attended

https://www.chesstech.org/events/2020/c ... -cheating/

My comments about interdependence are included in this thread

I think GM Matt Turner should get a medal for patiently trying to answer the regular barrages of repetitive questions from the usual doubters such as -

"How do you know this? (Nick F.)

(This soon leads to - how does anybody know anything?)

"Do they get to know who has signed them off, or on what basis? Do they get to challenge any irregularities in the procedure?" (Justin H.)

(This line of questioning is part of an extremely long, if not endless, chain of more detailed examinations in the hope that something of interest that can be useful to the questioner will eventually come up.)

Given that things are not likely to change since Prof. Regan himself has done quite a bit to try to explain his own methods AND platforms are stating things like -

"It is strict policy not to discuss the specifics of cheat detection with ANYONE EVER. Doing so would make it easier for cheaters... and be an invitation to UNPRODUCTIVE DEBATES... "

It must be time for the doubters here to answer some questions such as those that follow.

What will satisfy you, by way of answers to your questions, on this topic on this forum? (Or, in other words, where are you trying to get to with your questions?)

If you doubters already know that no amount of Q & A here will ever be satisfactory because of the nature of the problem then what is your purpose in pursuing the unobtainable?

Aren't you really just trying to increase doubt and the number of people here who think like you? (Or, in other words, conducting a recruitment drive to raise an army of doubters?)

What do you doubters want to achieve here exactly?

If it is to save the one righteous player, who is falsely accused, you are in danger of doing it at the cost of facilitating cheating (by exposing weaknesses in the platforms' methods) and even encouraging more to join the largely invisible legion of cheaters.

(NB: Because I do not doubt to the nth degree the answer for me is simple - DO NOT PLAY ONLINE if you do not agree with the providers' policies, procedures and terms & conditions. By all means come here to relate your online experiences to others but do not expect to change outcomes from there on here.)

[Matt Bridgeman]

I'd presume on some level over-the-board cheating in chess is reduced by the lack of high value prizes open to the masses. I think if there was a high frequency of large cash value Open events, with more of a sense of anybody could win - such as in the poker world - there could be more issues in chess. In a lot of sports detection of cheating often does lag behind the most determined exponents. A great documentary which nicely shows of the extremes some people will go to beat supposedly very secure anti-cheating measures, is the 2017 Netflix film Icarus, about the famous Russian doping scandal. https://en.wikipedia.org/wiki/Icarus_(2017_film)

[Roger de Coverly]

I'd presume on some level over-the-board cheating in chess is reduced by the lack of high value prizes open to the masses. I think if there was a high frequency of large cash value Open events, with more of a sense of anybody could win - such as in the poker world - there could be more issues in chess.

The 4NCL Congresses were offering £ 500 prizes to the winners of the rating restricted sections. There is only one known episode of cheating using computer engines to win such a prize. The arbiters were coy about how they detected it, but the rating performance was a give away of sorts. The bigger problem would be those whose ability exceeds their rating by virtue of a run of bad form before the Congress.

It's difficult to see how someone can be consulting an engine under the conditions of scrutiny during OTB play without resorting to subterfuge. Use of the Regan program was quite rightly rejected as a sole source of accusations. The method of consulting external help has to be established.

[Matt Bridgeman]

I'd presume on some level over-the-board cheating in chess is reduced by the lack of high value prizes open to the masses. I think if there was a high frequency of large cash value Open events, with more of a sense of anybody could win - such as in the poker world - there could be more issues in chess.

The 4NCL Congresses were offering £ 500 prizes to the winners of the rating restricted sections. There is only one known episode of cheating using computer engines to win such a prize. The arbiters were coy about how they detected it, but the rating performance was a give away of sorts. The bigger problem would be those whose ability exceeds their rating by virtue of a run of bad form before the Congress.

It's difficult to see how someone can be consulting an engine under the conditions of scrutiny during OTB play without resorting to subterfuge. Use of the Regan program was quite rightly rejected as a sole source of accusations. The method of consulting external help has to be established.

Probably it won’t be an issue, as there’s not the incentive to go to the extreme effort required. But there could be some chance maybe of an unknown online only player coming in hot to the 2022 over-the-board scene. To my mind that has been done once already with a total failure to act or to figure out how they did it.

[NickFaulks]

"How do you know this? (Nick F.)

So this is a question that it is no longer permissible to ask? We just have to express unquestioning belief or be cast out of polite society?

You ask further down what the sceptics hope to achieve. Personally, I do not give a fig for the platforms - they are like Twitter and Facebook, totally unaccountable and you deal with them at your own risk. They will never be brought to book, however questionable their methods.

My concern is for FIDE, who are subject to the law. I have no doubt that if FIDE were ever taken to CAS and had to rely on their current undocumented methods, they would be torn apart, suffering expensive and humiliating defeat. Many on this forum would find this highly entertaining, but I would not. I am therefore keen for the quality of the analysis on which they rely to be brought up to some kind of minimal standard.

As it happens, I am highly doubtful that this is possible. If I am right, then it would at least be useful to know that.

[John McKenna]

"How do you know this? (Nick F.)

So this is a question that it is no longer permissible to ask? We just have to express unquestioning belief or be cast out of polite society?

You ask further down what the sceptics hope to achieve. Personally, I do not give a fig for the platforms - they are like Twitter and Facebook, totally unaccountable and you deal with them at your own risk. They will never be brought to book, however questionable their methods.

My concern is for FIDE, who are subject to the law. I have no doubt that if FIDE were ever taken to CAS and had to rely on their current undocumented methods, they would be torn apart, suffering expensive and humiliating defeat. Many on this forum would find this highly entertaining, but I would not. I am therefore keen for the quality of the analysis on which they rely to be brought up to some kind of minimal standard.

As it happens, I am highly doubtful that this is possible. If I am right, then it would at least be useful to know that.

Thank you very much for that reply, Nick.

On that basis I think you can proceed here with a clear conscience and because I cannot question your motives - now that you've explicitly given them - I'll not question the reasons for your questions, again.

I also tend to agree with you but I believe Prof. Regan is not part of the problem. He is part of the solution.

[John McKenna]

Here are the frequencies of their playing the best through the tenth-best move in such many-choice positions: Rank / Pct. 1 / 17.76%, 2 / 13.22%, 3 /9.95%, 4 / 7.66%, 5 / 6.25%, 6 / 5.18%, 7 / 4.41%, 8 / 4.55%, 9 /3.50%, 10 / 3.03%, 11+ / 24.49%..."

I take that to mean among the players FIDE-rated between 1000-1200 who played a move in the 6,082 positions the best move was found by 17.76% of them and so on... down 24.49% of them who played the 11th or lower, "best" move.

Compare that to what Prof. Regan says about the best FIDE-rated players -

"For the elite class—2600 to 2800—in the same kind of many-choice positions, the new model does even better.

Much more data on elite players is available, so I have 49,793 such positions faced by them:

Whereas elite players found the best move in 30.85% of these difficult positions, my new model finds their move in 34.64% of them..."

(PLEASE NOTE: Prof. Regan, above, is comparing his computer model of human best move selection there with that of the best GMs. Neiither of which were capable of finding the best engine - Stockfish/Komodo - moves in those difficult ramified positions more than 30.85% (GMs) and 34.64% (Regan's model) of the time. Which means Regan's computer model of best human-like move selection was better than that of the best humans. And he goes on in the article to try to explain how and why that helps make his cheating-detection algorithms better.)

I take that to mean when the top GMs faced equivalent, but much more dificult, positions (than the tyros FIDE-rated 1000-1200 faced in their games) then the GMs found the best move in 30.85% of them (much better than the 17.76% best moves found by weak players in easier positions).

Jacob, I hope you can follow the above answer enough to realise how impossible it is for humans to play strong moves in difficult positions by "happenstance".

And if weak players play such moves multiple times even in a single game against a much stronger opponent then they are almost certainly not playing moves they're finding themselves alone at the board.

Hi John

Thank you for your considered reply. I quote only part of it above just to avoid a very long quote - I am not singling out part of your post and ignoring the rest.

That's OK, Jacob.

Apologies for the delay in replying.

I take your point that there is not a hard-line distinction, such that one game can never provide sufficient data but multiple games can. I am sure there are examples of games that would provide sufficient information.

That's agreed. I was merely pointing out that sometimes a single game can be a very strong indicator of a performance so exceptional that an ordinary player could not have done it without real-time assistance.

However, it seems to be that there are two significant limitations on the conclusions to be drawn from looking at multiple moves in a single game.

I agree there are "significant limitations" but not necessarily the two you give.

First, to what extent are the moves actually independent, and therefore providing additional evidence? If you have a very long game I imagine moves towards the start and end would be entirely independent of one another, but in a shorter game if a player, for example, happens to find the computer's preferred plan or tactical line, wouldn't that mean that you would expect a higher degree of correlation for perhaps quite a number of moves? That is leaving aside the question of preparation - if you play into a player's preparation you might find a significant number of matches with the engine's preferred moves, but you could not regard them as independent data points.

I believe GM Matt Turner has already answered that earlier in this thread -

i can recall a relatively recent world championship where the coverage showed the top three moves from three engines. Almost all the moves were distinct and almost all gave an assessment of 0.00 or very close.

However it's a valid point that once it' is known that a spectacular or speculative move works or doesn't work, that the subsequent play falls into place. Given that the mathematics of "you aren't good enough to play those moves without assistance even by chance" relies on independence of the samples, it's not so clear how much evidence a single game or handful of games really offers.

So this is an important point. Professor Regan argues that whether you select the 'computer' move has some impact on the likelihood of you selecting the 'computer' move on the next turn, but has pretty much no impact on the probabilities for subsequent moves. This is all taken account of in his software. This seems counter-intuitive to chess players because we all think about games in terms of 'lines of analysis', so we would expect there to be much greater interdependence seen in the moves.
Now you need to take some time to consider examples. The spectacular or speculative move that Roger talks about will almost always be discounted by the software because it will be excluded because one side is winning (we talk about winning combination, not 'good' combinations).
We might see an example from an endgame where one side was pushing a pawn through, so a8 naturally follows a7, follows a6, follows a5 follows a4. In the overall scheme of things this isn't very likely and it its impact is fairy small, but it is exactly the sort of thing that people would look for before signing off a high profile case.

Thanks, Matt.

Second, how many of the type of multi-option moves that Prof. Regan's analysis relies on are there in the game?

Prof. Regan is not relying on these "multi-option moves" positions to detect cheating "in the game", per se. He is using them to fine tune his cheat-detection algorithms

I haven't managed to find any information in his online papers as to how many he usually finds, and I have no idea whether you would normally find a handful or a large number.

Somewhere in his reports Prof. Regan writes about the number of games he has used to "train" his s/w.

In the extract of his Predicting Chess & Horses article you quote (that I quoted) at the top of this post he has 6,000+ such positions from games between players FIDE rated 1000-1200. He must have got them from a much larger set of such games.

He also says that he has 49,000+ much more difficult positions from games played by players FIDE rated 2600-2800.

I would imagine they form a larger proportion of the total set of games than is the case with the games and positions of the 1000-1200 rated players.

You suggested in another comment that 3 moves would be enough for real suspicion, but on the statistics above the strong player would manage that in (30.85%)^3 = approx. 3% of games. Is that really a solid basis for concluding someone is cheating?

I am not sure of all your figures, just above, but I never suggested 3 moves would be a "solid basis for concluding someone was cheating" in all cases, or even a very many cases...

Three such moves in game by a relatively weak player rated 1700-2000 against a player rated say 2200-2500 might be enough to raise suspicions high enough to warrant a search of the weaker player if playing otb. Or an equivalent procedure if playing online, perhaps.

Arbiters decide such things.

And does an average game contain three of these particularly difficult multiple-option positions? It's not clear from the extract above how often these positions come up.

Again, GM Matt Turner answered that with -

I think the point is that if I play Mickey Smith 700 rated I am likely to zero. If I play Mickey Adams, I could well have three or more.

Which means the stronger the players in the game the more such positions will appear.

My thanks, again, to GM Matt Turner.

[Li Wu]

Nice video by Naroditsky. He also explained some potential reasons for cheating whilst never condoning it.

I'm not aware of the differences between Stockfish and Houdini, especially at recognising mistakes near the endgame (in that R+P Berlin) but would imagine they (sites) have engines analyse with and without a tablebase for cheat detection purposes- and this is maybe why his engine instantly saw the errors?

In terms of cheating, using an engine in the endgame with access to tablebase is probably the least smart way to do it.