4NCL Online

[Rhys Cumming]

There are clearly edge cases such as Justin's

In my view "edge case" is not an appropriate term here

this case seems about as slam dunk as you can get.

Cool, can we see the games?

It's obviously a difficult area in terms of how publicly to identify who the player in question is so I don't wish to post the games directly: it should require at least a tiny bit of effort as a price of entry to see them! More than enough has been said however (including by Surbiton's statement) to make it not a particularly difficult deduction.

[JustinHorton]

I grasp that and it's fair enough as far as it goes: but you appreciate it does go to increase rather than decrease the degree to which this process is opaque rather than transparent.

[Ian Thompson]

Fair enough. But the analysis itself showed that it was very unlikely you were using a computer to cheat, so that would be in it's favour presumably?

That's probably fair to say. (The analysis was supplied confidentially, so I'm a bit reluctant to comment any further in public.)

I sent a PGN file of Justin's games to Ken Regan to pass through his anti-cheating software. Ken found equal parts corroboration and doubt. I have emailed a full copy of the report to Justin with Ken's permission.

[JustinHorton]

Yeah. I have some issues with that description but I can't outline them without, as I say, breaching confidentiality. I'll say it was an interesting report, and I understood more after reading it than I did before.

[Roger de Coverly]

I thought Regan's analysis was inconclusive, so he couldn't reliably say one way or the other.

Justin consulted recent books. It would be likely that the analysis therein had been prepared with engine assistance. In the absence of evidence to the contrary, it might be concluded that the chess.com detection process noticed this.

[JustinHorton]

Justin consulted recent books. It would be likely that the analysis therein had been prepared with engine assistance. In the absence of evidence to the contrary, it might be concluded that the chess.com detection process noticed this.

To be honest, we will never know until they say so, which in my view they should. The claim was that it was an experience they could learn from, which I might be prepared to believe were it not that subsequent statements from Danny Rensch (not to mention a general unwillingness to look too hard at their procedures at a time when online chess is all we have) gave me the impression that they were perfectly happy with the way they were doing things.

My general view is that may well have underestimated my ability to play chess at three days a move, and when I say "underestimated", I am not suggesting that I possess master strength in that field, just that I very likely got cut off well before I'd reached my rating peak. But I don't know this.

[Rhys Cumming]

I grasp that and it's fair enough as far as it goes: but you appreciate it does go to increase rather than decrease the degree to which this process is opaque rather than transparent.

Yes I agree, but I think that lack of transparency works both ways. There is an understandable degree of scepticism regarding anti-cheating measures given that lack of transparency which might lead a player to find a blanket denial easy to hide behind. As said in my first post, I do indeed hope the player is innocent, but if every cheating allegation results in a public withdrawal in protest and a long discussion about the possible issues with the anti-cheating system, then it poses only bad news for this and similar tournaments.

It is of course an important discussion to be had, there are clearly weaknesses in the process and the algorithms themselves, but I think the maths shows that the vast majority of cases are pretty cut and dry. What it certainly does call for is a robust appeals procedure, both by lichess but more importantly in this regard from the 4NCL. If both Surbiton and the player maintain his innocence and can provide sufficient evidence then hopefully the appeals procedure will come to an appropriate verdict (despite Surbiton's suggestion that they don't expect a fair procedure, I am yet to hear concrete issues with the appeals procedure).

[MartinCarpenter]

It is of course an important discussion to be had, there are clearly weaknesses in the process and the algorithms themselves, but I think the maths shows that the vast majority of cases are pretty cut and dry.

I'm rather less confident than that. Not that the maths is remotely nonsense of course, but it isn't perfect.

Most importantly, we're - given how the teams have reacted! - clearly a very long way from an agreed definition of cut and dry. That's definitely needed.

The levels of false positives suggested from the numbers discussed above are definitely a bit higher than I'd personally want to convict named, real people.

[Adam Raoof]

I know this is a daft question, but when I look at the pairings for the 4NCL, how do I find out the player's username on lichess?

Adam

[Rhys Cumming]

It is of course an important discussion to be had, there are clearly weaknesses in the process and the algorithms themselves, but I think the maths shows that the vast majority of cases are pretty cut and dry.

I'm rather less confident than that. Not that the maths is remotely nonsense of course, but it isn't perfect.

Most importantly, we're - given how the teams have reacted! - clearly a very long way from an agreed definition of cut and dry. That's definitely needed.

The levels of false positives suggested from the numbers discussed above are definitely a bit higher than I'd personally want to convict named, real people.

My point is we are discussing questions along the lines of 'how often will a 2100 player play at 2500?" and 'over how many games is the sample before it reaches a z value?'. In reality, we have a player who has played almost a whole season at over 600 points higher than his rating (which is certainly not at 1100) and there seems to be a great deal of circumstantial evidence on top of this. Why the team reacted in the way they did is certainly up for debate.

[MartinCarpenter]

We’re discussing the statistics because there should be really clear cut, widely agreed criteria to trigger those. Which would then hopefully cut down the dramatic team reactions!

If there’s lots of circumstantial evidence then that obviously has to be taken into account & makes quite a difference.

There might well be a lot of point in splitting the principled discussions of the statistics away from the 4NCL online thread itself.

[Matthew Turner]

I know this is a daft question, but when I look at the pairings for the 4NCL, how do I find out the player's username on lichess?

Adam

You can’t. If you post the name or pm me I’ll let you know the username if I know it.

[Adam Raoof]

I know this is a daft question, but when I look at the pairings for the 4NCL, how do I find out the player's username on lichess?

Adam

You can’t. If you post the name or pm me I’ll let you know the username if I know it.

Thanks Matthew, I will pm you.

This is fundamentally at the heart of the problem. Part of the solution is total transparency, at least if we are taking online chess seriously.

[Ian Thompson]

I know this is a daft question, but when I look at the pairings for the 4NCL, how do I find out the player's username on lichess?

Adam

You can’t. If you post the name or pm me I’ll let you know the username if I know it.

Thanks Matthew, I will pm you.

This is fundamentally at the heart of the problem. Part of the solution is total transparency, at least if we are taking online chess seriously.

You can see them on the 4NCL website pairings pages, but only while the individual games are in progress.

[Joseph Conlon]

I will note that once an account is closed on lichess it becomes very hard to look at the games played under that account. While the games still exist on lichess, it is not possible to click on the username and then bring up a list of all the games played (it seems to me the only way to find games is if you know the opponent, and then search the opponent's games).

This is relevant not just for cases of engine use but also for the training accounts that lurk at the top of the leaderboard. On one point there was (presumably) a super-GM training account (aqtobe) that sat clear at no.1 on the lichess blitz rankings with a rating of ~2950. What was interesting to me was that against e4 this account exclusively played the modern/Portugese Scandinavian, which is relatively uncommon but is what I play myself, and so the games provided a sort of mini-masterclass in how a strong GM handled the opening.

Then the account got closed, and it became almost impossible to find the games...