Chess Player Strip Searched

[Roger de Coverly]

Is the proposed test for convicting a player solely based on statistical analysis really designed to generate a false positive every two years? Am I doing the math right or do I miss something?

With leagues now reporting per month, the average number of games per event is likely to be quite small. But if it was an average of up to 5 per player per event, you might be talking of a half game count of the order of 2 million a year. Actually the true figure must be known, but not I think published.

They aren't going to tell, since it might question their hypothesis, but if they check games going back far enough that no advantage would be obtained by consulting a computer engine, how many false positives would be generated?

Actually you may expect more false positives than the statistics would suggest, because it isn't illegal to consult an engine or even a book before play starts.

By way of a comparison, the ECF figure for all domestic graded half-games including rapid is around 300,000.

If they ever got the monitoring to work, I would have thought a website where you could submit your games and be told in the opinion of a computer engine, how accurately you had played, would have commercial value in the training sphere.

[Roger de Coverly]

If you do not like what is in this paper, you are encouraged to present alternative solutions to the problems the paper is trying to solve.

I'm not sure what problems the paper is trying to solve. What is the problem with having a switched off device in your jacket pocket or a bag during play? If it is that you might retire to the rest room with the device, switch it on and analyse, why not state that as the problem and find a way to solve it?

Actually as that is the main problem at amateur level and even in the German Championship and League, I fail to see how statistical analysis can give much useful help. It's plausible to argue that "external assistance" at critical positions during the game can add a bit to effective player strength, but I would have thought next to impossible to separate that from "got lucky" or that the skill level you are being measuring against is not in fact correct, or not correct for the type of position being analysed.

[Paolo Casaschi]

If you do not like what is in this paper, you are encouraged to present alternative solutions to the problems the paper is trying to solve. I, in my capacity as Secretary of the FIDE Anti-Cheating Commission, are more than happy to receive proposals and distribute them to the rest of the committee.

With respect to the proposed paper I propose the following amendments:
1) have separate people writing the rules (and developing the tools) and separate people ruling about the complains
2) publish publicly the details of the statistical analysis tools and make it possible to replicate those results
3) exclude the possibility of a determination of guilt based solely on statistical analysis

[NickFaulks]

I fail to see how statistical analysis can give much useful help.

Of course, you're absolutely right. This is just a case of having to look as if they're doing something.

I seem to remember a story about a chap who was unable to demonstrate how to checkmate with K+R v K, but beat a GM, finishing the game by announcing checkmate in 11. Statistical analysis will nail him all right, but that's about all it will do. Once serious cheaters know their moves are being compared to Houdini, it will be a trivial matter for them to stay off the radar.

[Roger de Coverly]

Once serious cheaters know their moves are being compared to Houdini, it will be a trivial matter for them to stay off the radar.

Engine v Engine tournaments still take place and the results aren't that all games are drawn. Engines disagree on evaluations as we see when multiple engines are part of live game coverage. So what's the success rate of using engine X to detect games played by engine not X? Engines first got good around fifteen to twenty years ago and would do better today than then because of faster hardware. How good are modern engines at detecting the hand of their predecessors? Can the methods now prove that Deep Blue was in fact 100% computer and not, as Kasparov sometimes suggests, given human assistance?

[Paolo Casaschi]

Once serious cheaters know their moves are being compared to Houdini, it will be a trivial matter for them to stay off the radar.

Engine v Engine tournaments still take place and the results aren't that all games are drawn. Engines disagree on evaluations as we see when multiple engines are part of live game coverage. So what's the success rate of using engine X to detect games played by engine not X?

Something might actually be done there. Of course you do not compare a player's move with only the first choice of Houdini. If you do that and the player is cheating using Stockfish then your concern is very valid. However, all engines have few things in common, for example they do not blunder material accidentally; also there are moves that look weird to the human eye but all engines might find acceptable. Instead of comparing with the exact first choice of Houdini you might take the evaluation of Houdini in each position and check if the Houdini evaluation for the move actually played is "significantly different". For example, if Houdini evaluates the position as +0.3 and plays Ne5 and the player played Ng5 instead with an evaluation of +0.25 then it's still possible that Ng5 is the recommendation of a different engine (or Houdini on a different hardware or with a different amount of time available). If instead the Houdini evaluation of Ng5 is -0.55 you can be relatively sure that no engine recommended that. Something along those lines. If you have a million moves and the Houdini score for those moves is always in the neighborhood of the best choice of Houdini then you might suspect computer use. But you really need a lot of moves... and it assumes the cheater uses the computer for every move. What if the cheater is using the computer advise only for a few (critical) positions in the game and plays himself otherwise? That is much harder to catch and statistical analysis most likely fails. I guess the assumption is that, once you devised a "safe" method to peek at the engine during the game it's very difficult to resist the temptation to peek at every move (more or less the same that happens when you analyze a game at home, if you have the engine on it's very difficult to think by yourself and ignore the computer evaluation).

[NickFaulks]

If I were to use computer assistance during a game, I would only want it as a blundercheck. I like to think that I usually have a general plan, even if it is sub-optimal, and I'm not going to abandon that just because Houdini thinks that its own move is 0.6 better.

You might say that if I go through a tournament without leaving anything en prise, then I'm obviously cheating, but are you really going to say that about a 2700 player?

[Roger de Coverly]

What if the cheater is using the computer advise only for a few (critical) positions in the game and plays himself otherwise? That is much harder to catch and statistical analysis most likely fails.

If you set aside worries about those who would come to tournaments wearing devices in their shoes or whatever, the concern must be those who would leave the board and consult an engine or someone with access to an engine from time to time during play. Solutions that essentially accuse all amateur players of cheating and would default them if they have a switched off mobile phone in their possession are not the answer unless the intention is to disconnect amateur chess from FIDE. But I don't have much faith in the wisdom of the FIDE General Assembly.

[Paolo Casaschi]

You might say that if I go through a tournament without leaving anything en prise, then I'm obviously cheating, but are you really going to say that about a 2700 player?

Just look at the recent candidates tournament. Anyone without any blunder throughout the whole tournament? Even Anand left the game with Andreikin as a draw when every engine sees a tactical win.

The threshold for qualifying a mistake as a blunder might be lower for a 2700 than for you, but on practical terms it means letting a win (draw) slide as a draw (loss).

[NickFaulks]

But I don't have much faith in the wisdom of the FIDE General Assembly.

You're obviously correct there. The GA will approve whatever is shoved under their noses. Where we part company is that you have somehow convinced yourself that the large and sensible federations are swamped by the small and foolish ones. I have been going to these things for years, and have not seen a shred of evidence to support that view.

If the ACC's final proposal is to become at at all sensible, this will happen during the consultation process which now starts. This may turn out to be a sham, but it is too early to say that. Your own federation will no doubt boycott this process, but there is nothing to stop individuals from sending their views to the ACC, c/o the Secretary and, I suggest, copy to the FIDE office. I fear that people on this forum will not do that, because it's so much easier to snipe from the shadows.

[NickFaulks]

You might say that if I go through a tournament without leaving anything en prise, then I'm obviously cheating, but are you really going to say that about a 2700 player?

Just look at the recent candidates tournament. Anyone without any blunder throughout the whole tournament? Even Anand left the game with Andreikin as a draw when every engine sees a tactical win.

The threshold for qualifying a mistake as a blunder might be lower for a 2700 than for you, but on practical terms it means letting a win (draw) slide as a draw (loss).

So you're saying that if Anand had spotted that win, it would have taken him over the threshold where he could be accused of cheating?

[Roger de Coverly]

I have been going to these things for years, and have not seen a shred of evidence to support that view.

What about the "we will do what Kirsan asks" club who so nearly got the zero default times incorporated into the Laws of Chess? How many rated players did they represent?

It's completely rational to have as little to do with FIDE as possible. Until such time as it reforms its "Rotten Boroughs" system of voting, General Assembly voting will be on the basis of who has the deepest pockets or the greatest powers of patronage. Representing a majority of the world's active players doesn't feature.

It remains a threat to FIDE that if it introduces rules that seriously annoy a majority of amateur players, that those Federations controlled by the affected players could turn their back on FIDE. As you note, most of the amateur chess in England is outside of FIDE and likely to stay that way. I expect FIDE could live without the ECF's approval as it's just a nuisance and resistant to attempts to subvert its likely voting. But what if the rest of the Western European Federations thought likewise? Events such as Hastings and the big European opens could then continue almost unchanged in terms of participation.

[NickFaulks]

What about the "we will do what Kirsan asks" club who so nearly got the zero default times incorporated into the Laws of Chess?

The problem in Dresden was that the delegates from the big federations supported it. Your own delegate supported it. Yes, I know that he was subsequently voted out of office, but I don't see how that helps.

[JustinHorton]

I expect FIDE could live without the ECF's approval

Indeed. And so could everybody else. Nobody gives a damn what we think.

[Paolo Casaschi]

You might say that if I go through a tournament without leaving anything en prise, then I'm obviously cheating, but are you really going to say that about a 2700 player?

Just look at the recent candidates tournament. Anyone without any blunder throughout the whole tournament? Even Anand left the game with Andreikin as a draw when every engine sees a tactical win.

The threshold for qualifying a mistake as a blunder might be lower for a 2700 than for you, but on practical terms it means letting a win (draw) slide as a draw (loss).

So you're saying that if Anand had spotted that win, it would have taken him over the threshold where he could be accused of cheating?

No. I'm saying that the decision of Anand to draw that game suggests he was probably not cheating with an engine during that game. Just an example, I haven't run through all the games with all the engines (nor I feel the need to).