Chess Player Strip Searched

[Steve Collyer]

so it would be interesting (and some would say fair) to remove that game from the results.

It depends how you think the hypothetical cheating method worked. If you postulate that it involved the live feed in some way or other, then yes. If it was a concealed device on his person where he was inputting the moves himself, then it's not going to make a difference.

{ White: Borislav Ivanov }
{ Top 1 Match: 13/27 ( 48.1% )
{ Top 2 Match: 18/27 ( 66.7% )
{ Top 3 Match: 20/27 ( 74.1% )
{ Top 4 Match: 22/27 ( 81.5% )

is the rather mundane match rate for round 8, so if the data does support anything, then I think it's reasonable to postulate that the very much higher engine correlation in all other games in the batch relates to having the live feed up.

[Roger de Coverly]

If I've done my sums correctly, the chance of getting 293/314 or better given probability of .85 of being "correct" is about 1/50,000.

Aren't you making an assumption of independence between individual moves? If many positions, play a, then b and c automatically follow. It's not as if you are testing against randomly selected moves.

[Leonard Barden]

so it would be interesting (and some would say fair) to remove that game from the results.

It depends how you think the hypothetical cheating method worked. If you postulate that it involved the live feed in some way or other, then yes. If it was a concealed device on his person where he was inputting the moves himself, then it's not going to make a difference.

{ White: Borislav Ivanov }
{ Top 1 Match: 13/27 ( 48.1% )
{ Top 2 Match: 18/27 ( 66.7% )
{ Top 3 Match: 20/27 ( 74.1% )
{ Top 4 Match: 22/27 ( 81.5% )

is the rather mundane match rate for round 8, so if the data does support anything, then I think it's reasonable to postulate that the very much higher engine correlation in all other games in the batch relates to having the live feed up.

The suggestion is that the live feed stopped in round 8 at move 15, and before that Ivanov's moves correlated closely with Houdini, while from move 16 onwards they only correlated in the case of forced moves and captures.

[Brian Valentine]

I have made a lot of simplifying assumptions. Nonetheless I'm struggling to see how the computer does not treat each position as independent. I agree the human must string together combinations and hence each of their "solutions" are not independent.

[Roger de Coverly]

Nonetheless I'm struggling to see how the computer does not treat each position as independent. I agree the human must string together combinations and hence each of their "solutions" are not independent.

I'm not sure quite what is being tested. Are you asserting that ten moves in a row by a human matching an engine's choice is less likely to be a matter of parallel judgement and more indicative of consulting the engine than one move?

Regarding whether an engine's choice is independent, wouldn't you have to clean the computer between moves? In order to find good moves quickly, programmers have devised all sorts of methods (hash tables?) to avoid analysing the same position more than once. So for a fixed time of search, an engine will be able to go deeper into the position if the opponent selects one of the most expected replies, because it remembers what it looked at on the previous move.

[Brian Valentine]

There are a number of tests that could be set up.

The one I was thinking about is the situation where there is no computer assistance. In that situation how likely was Ivanov to achieve this performance given his prior strength.

I don't know how these computer programs work. Implicitly I have assumed independence. If there is meaningful correlations in each step then I think it is much more difficult to assert outside assistance from these results (without other suspicions).

It might be interesting to hear from Steve at what threshold internet players are thrown off the system.

[Steve Collyer]

Top 1 65%
Top 2 80%
Top 3 90%
Every player who I've analysed/reported with these figures or higher for a large batch of games (usually 20, each with 20+ non-database moves) has either been banned for engine use shortly after reporting, or a little later for some reason or other. This would be over 100 online accounts. Mostly top 0.1% highest rated players on various sites.

As I said earlier, it is quite amazing how consistent the top unassisted players are in their engine match rate correlation if the methodology is applied to their games. Several qualified statisticians were highly skeptical - until they ran their own benchmark tests.

If you are going to attempt to ascertain if a player has used an engine to suggest moves on a frequent basis, then comparing the moves a player made with those an engine chooses will always be the single most important factor in determining this.
There are other factors, of course, but engine match rate correlation will always be the most important issue. If your suspect is found to have below benchmark threshold match rates, then the case is over as far as I can tell.

[Roger de Coverly]

In that situation how likely was Ivanov to achieve this performance given his prior strength.

But is that the right question? If you are going to assert that his performance is telephone number to 1, you also have to allow for the probability of intensive training out of the public eye improving his standard to that where finishing equal first against ageing GMs is plausible.

In server chess, particularly the turn based versions, there are no witnesses, so someone consulting an engine between moves is a plausible hypothesis. Over the board in a tournament, there are plenty of witnesses, so in principle it's a sterile environment with no external assistance possible. If there are devices out there that can receive a live game feed and covertly feed engine analysis back to players, then organisers will have to think again about covering events live without a delay.

Cheating is a moveable feast. Under ICCF rules, blunder checking using an engine is allowed. Even on sites which ban players for engine use, it would be allowed to consult a game database, book or published analysis in turn based play, if not in Blitz. None of these would be allowed in OTB chess.

[Matt Fletcher]

But is that the right question? If you are going to assert that his performance is telephone number to 1, you also have to allow for the probability of intensive training out of the public eye improving his standard to that where finishing equal first against ageing GMs is plausible.

With respect, Roger, I'm not convinced that this is quite the right question either because it's about the intrinsic quality of the moves rather than just the rating performance. One could just about imagine that a 2200 player could train himself up to (say) 2400 out of the public eye and then get a few results against underperforming GMs. But the quality of the moves seems to suggest that he was playing at (above) world champion strength and I'm not sure that's as plausible.

[Christopher Kreuzer]

On the general topic of cheating in chess, does anyone think that chess could eventually find itself in a situation like that in which cycling found itself? Or is it easier to keep chess 'clean'?

[Ray Sayers]

Well, to judge by the improvements in mobile communications in the last 10 - 20 years I would guess it will be pretty nigh impossible to prevent outside interference in games if people are determined to cheat in the next 10 - 20 years. So maybe the penalty for being caught needs to be much more severe.

[Roger de Coverly]

On the general topic of cheating in chess, does anyone think that chess could eventually find itself in a situation like that in which cycling found itself? Or is it easier to keep chess 'clean'?

As far as Over the Board chess is concerned, it's completely legal to "take computers" out of competition, so it's only "in competition" that you need to be concerned about. The traditional ruling that you don't seek external assistance continues to apply, but you need means of not putting temptation in the way of players. So measures like insisting that players remain at their boards, spectators aren't able to signal etc.

If I understood the cycling issues, it was that participants found ways to train which increased their physical endurance and these methods are regarded as "doping". In chess it's memory and mental endurance which you might try to enhance. I don't think there's really a "smart pill" and even if there was in the future, you might expect those in decision making positions to be required to take it.

[Mick Norris]

If I understood the cycling issues, it was that participants found ways to train which increased their physical endurance and these methods are regarded as "doping".

Yes, but they also took them during competitions - the Grand Tours last 3 weeks (France, Italy, Spain) and have a couple of rest days, recovery when cycling for hours and hours day after day is an issue - you might equate this to somewhere like Wijk, LCC, Linares, where you can be competing for up to 7 hours daily for days on end

[Roger de Coverly]

A couple of recent links.

http://pressadaily.bg/publication/9128- ... %82%D0%B0/

http://tv7.bg/news/9611422.html

You need Google translate to make any sense of them.

It appears he has become a minor celebrity in his home country.

The problem he will face is that he has to maintain his good form to see off the cheating accusations, which will only continue should he repeat his results from Croatia, in particular if he continues his success at reversing engineering Houdini's evaluation function.

The likes of Deep Blue and earlier models of Fritz acquired their strength by brute force search depth. There was no great sophistication and they just analysed almost blind to positional considerations to as much depth as time and hardware allowed. From the Fruit program onwards, the evaluations got more sophisticated, thus eliminating unpromising continuations at an earlier stage. Many of these programs are Open Source, so what if an aspiring player took the ideas encapsulated in the program code and applied them to his own method of thinking? I don't think the odds of this being possible are especially long.

[Leonard Barden]

Thanks to Roger for providing these very interesting links.

To me, the significant aspects of Ivanov's new statements are

1 As in his whychess interview, he has a paranoid tone and maintains his generalised claim that his 2700 performance was achieved by a mixture of natural improvement and poor play by his GM opponents. He does not provide any detailed technical information at all about any individual games.

2 He says he will play at Plovdiv next month. If so, that should be informative, particularly as he states that Bulgaria's No2 Ivan Cheparinov, who believes that Ivanov uses a program, is also competing at Plovdiv

3 The reader comments to the second link include one from a 2000-rated player who says he lives in Ivanov's home town Blagoevgrad and implies there is a conspiracy among Ivanov, his trainer, and the CEO of the Bulgarian federation.

There are actually several unresolved mysteries about the matter, some of which have not yet been published anywhere. I would just highlight the conflicting reports over whether or not Ivanov's round 9 win at Zahar was on the live transmission and also the question of his loss in round 8 where I wrote above


the suggestion is that the live feed stopped in round 8 at move 15, and before that Ivanov's moves correlated closely with Houdini, while from move 16 onwards they only correlated in the case of forced moves and captures.

I was hoping that Steve Collyer would answer that, but so far he hasn't.