4NCL Online

[Alex McFarlane]

The obvious answer to Matthew's question is zero.

Any system that works purely on the quality of moves will always run the risk of an innocent person being wrongly accused (and in the case of platforms, convicted!)

I think we need to establish what the 'compromise' position should be.

I don't think that the chess community really wants to apply the maxim "It is better that 100 guilty men go free than one innocent man is convicted".

But what figure would we be happy with? If that can be universally agreed then the maths can be applied to the current tests.

I am now looking at the sky for an aerial display of pachyderms.

[Roger de Coverly]

I think we need to establish what the 'compromise' position should be.

In the absence of external supervision, which is automatically present in over the board chess, you cannot play serious chess if external assistance is to be prevented. So how can eternal supervision be made to work? In the Carlsen events, they appear to have webcams in operation so they can see one another.

[Hok Yin Stephen Chiu]

I suppose it depends on perspectives, and how one views the application of British legal norms applied on this particular situation of online chess.

Clearly the only fool-proof method, is a full webcams' in operation showing the player in the room, to be absolutely sure no cheating is going on.

--

The question is whether that is achievable across a league like 4NCL, or indeed some of the regional online leagues that I've seen across the country.

if you played two hours of chess, AND you knew your opponent has cheated (and proven), that is clearly frustrating. In fact, if you believe your opponent has cheated, but you are unsure of how to prove it, the feeling is probably worse, more distracting, and more frustrating.. (At what point has your opponent played a couple of good moves, OR have you switched on an engine for a couple of moves? question for the philosophers!)

So unlike the long games at 4NCL, what I have seen in a local online league that I'm slightly involved with, is that being rapidplay (15+10'), there have been zero cases of cheating or accusations of cheating, across two divisions to my knowledge. Most likely, the incentive to cheat is a) very low, and b) the fear that your opponent is cheating is also low, namely because (1) it is rapidplay, and (2) you may feel less invested in a 15+10' game! E.g. it is more like that a player will be far angrier if they believe their opponent has cheated against them - wasting two hours of their life, than maybe 25 minutes of an evening..

But, what appears to be the case, is that clearly there is a huge demand for long-play online chess (even if I personally won't touch it with a barge pole, I am delighted at what 4NCL have achieved - creating an opportunity at Warwick, where both current players and alumni can play chess together like old times again), so I expect all this to rumble on!

--

Now, onto cheating algorithms and checks.

I suppose one either believes you are innocent until proven guilty in the strictest legal sense. Or, you withhold the principle over convenience. It may therefore be convenient of a league at any level to deploy a fairly accurate algorithm, but equally annoy teams who are convinced by their players, and even withdrawn their teams over their belief of injustice. But, be in no doubt that we could be in shakey grounds, for as long as people are unconvinced and have never seen the "science" behind a tarnish could in a very, very small number of cases could be libel.

It may interest some, that even in the field of DNA testing where there are far more academics and scientists working in that field, it has come under fire.. https://www.wired.co.uk/article/dna-tes ... ts-science

--

Perhaps, the unwritten social contract, in all this, is that players simply have to accept that (1) they participation will incur a small likelihood that they could be playing two hours of chess against a cheat, and (2) that one could be accused of cheating and personally find that libelous, but that is simply the risk. And that maybe that is just the status quo, we have to accept by entering? After all, leagues are private entities, which one has the freedom to participate or otherwise? I do not suggest I have the answer, namely because, no solution will satisfy everybody, not least myself!

"if you are not confused, you are not thinking", as greater men than myself have said..

[Alex McFarlane]

Clearly the only fool-proof method, is a full webcams' in operation showing the player in the room, to be absolutely sure no cheating is going on.

I'm afraid this, while offering a further degree of reassurance, is anything but a fool-proof method of ensuring that no cheating is going on. I can think of at least two ways of circumventing this.

[JustinHorton]

It may interest some, that even in the field of DNA testing where there are far more academics and scientists working in that field, it has come under fire.

I'm sure there's been some miscarriages of justice involving DNA evidence that turned out not to be as extremely reliable as the court was initially told.

[Hok Yin Stephen Chiu]

I'm afraid this, while offering a further degree of reassurance, is anything but a fool-proof method of ensuring that no cheating is going on. I can think of at least two ways of circumventing this.

I imagine it depends on where you put the webcam!

It may interest some, that even in the field of DNA testing where there are far more academics and scientists working in that field, it has come under fire.

I'm sure there's been some miscarriages of justice involving DNA evidence that turned out not to be as extremely reliable as the court was initially told.

Correct.

[DavidWalker]

So what would you be prepared to accept?

Maybe look at this from the perspective of an honest player over an entire playing career.

Suppose that such a player is willing to risk a 1% chance of being falsely accused of cheating during their career as the price for being able to compete in "serious" online competition. If that player plays 50 "serious" games per year over a 50 year period, that equates to 2,500 games. If their play is tested after each game, then they should be willing to tolerate a 1 in 2,500 * 100 = 250,000 "false positive" rate. This equates to a z-score slightly less than 4.5 which is at the lower end of the 4.5-5.0 range Dr. Regan has proposed in some of his posts.

[Roger de Coverly]

What hypothesis is being tested? Is it that the player used external assistance in every game, or is it that the player has used external assistance in at least one? If it's the latter, the test has to be reliable at a one game level, which by the usual reasoning must increase the odds of the engine matching having occurred by chance.

[Ian Thompson]

So what would you be prepared to accept?

Maybe look at this from the perspective of an honest player over an entire playing career.

Suppose that such a player is willing to risk a 1% chance of being falsely accused of cheating during their career as the price for being able to compete in "serious" online competition. If that player plays 50 "serious" games per year over a 50 year period, that equates to 2,500 games. If their play is tested after each game, then they should be willing to tolerate a 1 in 2,500 * 100 = 250,000 "false positive" rate. This equates to a z-score slightly less than 4.5 which is at the lower end of the 4.5-5.0 range Dr. Regan has proposed in some of his posts.

I do think the right way to go about this is to say "what's an acceptable false positive rate" and then accept whatever the resulting percentage is for cheats who get away with it.

That seems to be the opposite of what online platforms are doing, or at least claiming to do, though. Their priority is to catch all the cheats and not worry about how many innocent players they ban along the way.

[DavidWalker]

What hypothesis is being tested?

I hope that there would be some agreed minimum number of move events before any anti-cheating software would be used. For example, suppose that on average 5 games are required to give a reliable z-score. In a ten round event such as the 4NCL, this would mean that a player could be tested after each of rounds 5 - 10 inclusive, ie 6 games instead of the full ten.

I assumed a test after every game to include the possibility that a player's recent games from previous events (eg a previous 4NCL season) might be included in any analysis. However, even if the number of tests is halved, this would only change the required z-score from 4.47 to 4.31.

[Matthew Turner]

David,
It is an interesting approach, which seems perfectly reasonable to me. I don't think the number of tests would be halved because each test isn't independent. Clearly there is a much greater probability that you will pass a test in round 9 if you have already passed a test in round 8. So a season of 11 games in the 4NCL might be the equivalent of 2 independent tests.

[Matthew Turner]

Their priority is to catch all the cheats and not worry about how many innocent players they ban along the way.

Why would they do that though? Surely, it would just make no (commercial) sense.

[JustinHorton]

Why wouldn't it? The number of people who are able to do anything about it is very small, the number of cases which become known to the chess public likewise, and most of the players don't care because their priority is to get rid of the cheats.

[Matthew Turner]

So Chess.com banned 12,000+ accounts last month. Lets say 10% of those were innocent, in the space of a year you would have 15,000 people slagging chess.com off, isn't that going to do reputational damage?

[JustinHorton]

I do think the right way to go about this is to say "what's an acceptable false positive rate"

This may be quite a reasonable thing to ask, but perhaps rather than just pluck a figure from the sky, we try look at it and look at it from a more human angle, and then see how that might work out numerically.

Let us suppose that every so often, there is going to be an incident in which a player is disqualified for alleged cheating, and then subsequently it's found that this judgmen was wrong, or unclear, or what you will, and that they are reinstated, either after an appeal or perhaps even after an appeal is rejected but it is later felt that that judgement is unsustainable.

How often are people prepared to have such an event occur? How often do w think it could occur, before we decided this really wasn't working and the reputational damage to indivudals and the event was too costly?

That's a rhetorical way of putting the question, perhaps, but it's at root the same question. Perhaps the answer is "every five years", or every two years", in which case you can locate yourself a value of X in 1:X, depending on hw many games or events you think will take place in that time.