Chess Player Strip Searched

[Steve Collyer]

I also analysed ICCF WC 22 finals for van Oosterom in 2006, when engines were used & got the following results using standard match rate methodology:
http://www.iccf-webchess.com/EventCross ... x?id=13580
Houdini 1.03a x64 4_CPU Hash:512 Time:40s Depth:Max 22ply
{ Oosterom, Joop J. van (Games: 20) }
{ Top 1 Match: 587/876 ( 67.0% ) Opponents: 616/872 ( 70.6% )
{ Top 2 Match: 742/876 ( 84.7% ) Opponents: 747/872 ( 85.7% )
{ Top 3 Match: 794/876 ( 90.6% ) Opponents: 796/872 ( 91.3% )
{ Top 4 Match: 826/876 ( 94.3% ) Opponents: 822/872 ( 94.3% )
Clear, blatant engine use by both van Oosterom & his opponents, according to the standard thresholds.

Those are only a few of the benchmarks, but I think you get the picture!

[Roger de Coverly]

Clear, blatant engine use by both van Oosterom & his opponents, according to the standard thresholds.

Engine use perhaps, in fact probably for tactical checking. It's allowed under ICCF rules anyway. If you think the engine is actually playing the game, surely you should expect to see results much closer to 100%? But can Rybka detect the use of Houdini and vice versa? If there's a choice of reasonable moves, isn't first choice just a matter of the style programmed into the engine's evaluation function?

[Steve Collyer]

Yes...
van Oosterom's cumulative match rate % is interestingly a little lower than his opponents.
A strong centaur will rarely lose to a decent player using an engine.
Compare the results above with the benchmarks on the previous page & you can see just how obvious engine use is, even if it's only to avoid blunders.

The first choice move issue isn't such a problem if you have a large sample size of non-database moves. Typical cheat suspect batches yield 800+ moves, so the odd difference here & there doesn't mean much to the over all %'s.
Usually either someone is a blatant cheat (65, 80, 90%+ for top 1, 2 & 3) or they aren't.
Very few of the 150 or so online players I've analysed have been borderline.

[Roger de Coverly]

"The consensus is that Ivanov played almost perfect chess and followed Houdini in at least 9 out of 10 cases, for each move it was always one of Houdini's top three proposals.

I gather he analyses the first round game starting from the opening choice, where dxe5, Qxd8 and Nxe5 in the Exchange variation is apparently Houdini's weapon of choice against the Kings Indian. That's only possible evidence of pre-game preparation using Houdini, which is completely legal. A superficial check against a database tree suggests they followed previous games for a fair number of moves before his opponent varied.

Speculation continues as to how the cheating, if there was any, was carried out. It's different from the casual approach possibly used in the German cases which was to retire to a cubicle and switch on the smart phone. Specialist hardware might be involved. Some years ago, Casinos ran into problems with card counters, gamblers who would remember previous cards dealt and use that to compute the adjusted odds. Some even built devices into their shoes or clothing to assist them. Given that you only need International Postal or even binary to communicate moves, a device that worked by touch and vibration is presumably plausible given the hardware and software skills to get it operational and practice at interpreting the signals.

[Roger de Coverly]

Usually either someone is a blatant cheat (65, 80, 90%+ for top 1, 2 & 3) or they aren't.
Very few of the 150 or so online players I've analysed have been borderline.

I struggle with the assertion that this proves cheating in the absence of a demonstrated method. Quite obviously if you sit at home playing on-line correspondence, it's very easy to switch on the engine between moves and if the rules of the site attempt to ban this, then there's a plausible case. More difficult if you are playing in a match or tournament.

So if your style of play is such that 13 out of 20 moves match those of a computer engine possibly randomly chosen, or that a computer engine has been programmed with a style to match your moves 13 moves out of 20, that is sufficient grounds to accuse players of cheating? Personally I don't think so and if you were playing over the board, you wouldn't be able to receive and choose between one of four moves.

[Steve Collyer]

Don't you think the remarkable consistency of the benchmarks demonstrate the legitimacy of the approach?
If there are several hundred matches from the best players in the world analysed by many different analysts & the extreme upper end thresholds are all around
top 1 match: 60%
top 2 match: 75%
top 3 match: 85%
and then we come to Mr no-name who, from the safety of an internet connection, manages to consistently play far more engine-like chess in many games over time than every single benchmark ever tested.
What good reason is there for that?
What likely reason is there that Joe Shmoe plays far more like a 3200 Elo rated engine in many (supposedly unassisted) games over time than Carlsen, Fischer, Kasparov, Kramnik, Berliner, Rittner...?
I would imagine he would be better served by getting to the next FIDE world championships rather than loafing around on chess sites!

[Roger de Coverly]

and then we come to Mr no-name who, from the safety of an internet connection, manages to consistently play far more engine-like chess in many games over time than every single benchmark ever tested.
What good reason is there for that?

If you apply the benchmarks to online correspondence and you have an obsession with preventing any engine use in the games, then fine. Even so, a 65% match up on first choice either says they are not using the engine every move, they are selecting from a range of engine options, or the test has failed to detect which engine, if any, is being used. What sort of match up do you get if you use games known to have been played by engines?

If you apply it to over the board chess by reasonable players with no apparent means of them taking advice during play, then it's just witch hunts and false accusations against players producing results above their station.

[Steve Collyer]

Roger, the point is that it already has been applied to the very best OTB players...
This is how the benchmarks are created!

I did say one game means nothing using this method. We all get the odd freak very high match rate game, even OTB. The method applies to batches of games which have been objectively selected & is very much geared towards finding online idiot cheats. The OTB part relates to getting high quality, reliable benchmarks in the first instance.
I can analyse Ivanov's few suspect games, but the results have little if any meaning using the methodology I refer to.

[Roger de Coverly]

Roger, the point is that it already has been applied to the very best OTB players...
This is how the benchmarks are created!

In the absence of additional evidence as to how it was done, you should not accuse players of cheating just because they exceed these benchmarks. The benchmarks also show the extent to which top engines, whilst accurate in their calculations, fail to reproduce the style of top human players.

"Book" is a moveable feast as well, players know things which haven't been played yet and may well be computer assisted.

[Steve Collyer]

A 65%+ top 1 match from let's say 800 non-database moves simply means the suspect plays considerably more engine-like chess than any of the benchmarks.
You then have to delve into why this should be the case.
Using a Page View Log is useful. Most chess server sites have them. It generates an output of a user's activity whilst visiting the site.
An idiot cheat will pull down FEN/PGN then plug them into his engine, then move moments later, then repeat the process etc...
If these moves consistently correlate to engine choice moves, especially in balanced positions, then you're all clear to hit the "ban" button.

[Steve Collyer]

Roger, the point is that it already has been applied to the very best OTB players...
This is how the benchmarks are created!

In the absence of additional evidence as to how it was done, you should not accuse players of cheating just because they exceed these benchmarks. The benchmarks also show the extent to which top engines, whilst accurate in their calculations, fail to reproduce the style of top human players.

"Book" is a moveable feast as well, players know things which haven't been played yet and may well be computer assisted.

Roger, I'm sorry but this is nonsense.
If Kasparov took to playing long time control online chess whilst being looked in a room with no access to any engine then maybe you'd have a point. The benchmark thresholds for human achievable unassisted engine-like play could well be under threat & need to be altered.
As it is, very few titled players play online chess, simply because as one told me "the internet is full of cheats".

[Roger de Coverly]

Roger, I'm sorry but this is nonsense.

I am referring to accusations made against over the board players. Downloading the position and task switching to an engine is the on-line equivalent of switching on your phone, tablet or PC whilst sitting at the board in OTB chess. It's seeking external assistance and has always been banned.

The original context was whether you could accuse and convict a player in an OTB tournament of cheating purely on the basis of one of these match up tests. I think we both agree that you need additional evidence, in other words at least a balance of probabilities as to how the cheating was done.

[Steve Collyer]

Yes I agree.
I thought you meant "players" in reference to online users too.
Only if Ivanov had played a significant number of games which were selected in an objective way, analysed, and match rates in excess of the benchmarks achieved, could this methodology be applied.
A sample of 600 non-database moves would, in my opinion, constitute a significant OTB sample.
Some strong players have looked at the suspect Ivanov games & found "smoking gun" top engine choice moves which make no sense to them. That, along with his rating performance is probably more relevant than looking at match rates from a couple of games, especially given Ivanov's non-titled status.

[Geoff Chandler]

Hi Roger

I too was very sceptical about this method and would defend known good OTB players.

But after they were banned the 'evidence' was allowed to be posted
I had to eat humble pie.

At one time anyone posting evidence on RHP was given a month forum ban
and the post lifted. Nowadays it appears they have stopped banning known cheats
and the posting of evidence is allowed.

When you have a 2200 player getting higher computer match up's than
Fischer, Capablanca, Lasparov etc...at the peak then you can see why eyebrows are raised.
I'd argue that the computer moves may not always be the best moves to play v a human.

Which is another indicator that a player with high match up's over a 20 game period
is almost certainly using computer assistance.

That link to the GM petition proves that the top players agree that there is something
going on and FIDE need to act before the OTB game at the very top level is destroyed by paranoia.

[Roger de Coverly]

match rates in excess of the benchmarks achieved, could this methodology be applied.

I struggle with the 65% benchmark for first choice as indicative of cheating. That's saying that 13 out of 20 moves match your chosen engine, which means that 7 moves out of 20 don't. Assuming that the naive cheater is just using an engine's first choice, that presumably means he's using a different engine or different settings. So you have engines not agreeing the best move in 7 out of 20 moves. Doesn't this make a nonsense of the 65% benchmark as that becomes conditional on the engine and settings chosen? Suppose you checked the human benchmarks against a panel of engines so that you checked whether any of the human moves matched one of the engine first choices. Outright blunders would never be an engine first choice, but what sort of match up would you get if the test was to match against at least one engine first choice?

I could accept that if you test Rybka, Houdini, Fritz, Stockfish etc. against top human play, you get 60-65% matches on the first choice for each of them. But are they always the same moves? If not, then isn't the benchmark of whether human choices are the same as engine choices somewhat higher when you weaken the test to ask whether the moves played match any competent engine?