4NCL Online

Venues, fixtures, teams and related matters.
Joseph Conlon
Posts: 104
Joined: Thu Jun 06, 2019 4:18 pm

Re: 4NCL Online

Post by Joseph Conlon » Sun Aug 16, 2020 8:56 am

Roger:

To me I think arguments over whether the distribution of 'B' or 'C' are precisely normal are not relevant. The question is what is the chance of objective online performance lying in the tail of the distribution (z>3 here).

Unless one can think of a reason why objective online performance should be better than objective OTB performance (and all arguments I can think of go the other way, e.g. shorter time controls) then it doesn't matter about the nature of the distribution, given that 'A' is a data-driven distribution, and so the probabilities in B to fall in the extreme regions should all be strictly less than A.

Paul Cooksey
Posts: 733
Joined: Fri Oct 21, 2016 4:15 pm

Re: 4NCL Online

Post by Paul Cooksey » Sun Aug 16, 2020 11:36 am

Thanks Roger.

Joseph - I believe many psychologists say better performance under pressure is a myth. Maybe at home in familiar surroundings with creature comforts and without the minor stresses of a venue, I am under less pressure and my chess is better. Maybe, because I do most of my study on a screen, I play objectively better.

I am not sure how statistically relevant the advantages of playing from home are. But I can at least make an argument they exist. Which I think gets us back to how we interpret the data.

John McKenna
Posts: 4206
Joined: Tue May 17, 2011 2:02 pm

Re: 4NCL Online

Post by John McKenna » Sun Aug 16, 2020 12:51 pm

Roger Lancaster wrote:
Sun Aug 16, 2020 2:43 am
John McKenna wrote:
Sat Aug 15, 2020 10:59 pm

As I said, I find it hard to believe the people responsible for the underlying statistical treatments of the data available to them - both in the 4NCL and the wider online & otb chess world - have somehow got it basically wrong.
John, I don't want to dwell on this forever but my comments yesterday related to one specific graph which was the work of a single human or small group of humans rather than endorsed by "the wider online & otb chess world".
Roger, the author of the 4NCL Fair Play: Anti-Cheating Online article included this in it -
We use Professor Ken Regan’s model, which has existed in various forms since the bitter Topalov-Kramnik World Championship Match and has been used in FIDE and national cases since 2011...
That is what I say is the general acceptance of the methodology & s/w and is what you are ignoring when you say "was the work of a single human or small group of humans rather than endorsed by the wider online & otb chess world".

The article also contains -
If a website flags or bans a player for using computer engine assistance in 4NCL Online, I inform the captain of the player’s z score, and the MMP/ASD characteristics I described earlier. There have only been two confessions out of about 30 cases at the time of writing.

The usual response I receive is that they have independently reviewed them and said that there isn’t a shred of evidence to support the allegation. The resulting implication is that the website is wrong, and Professor Regan’s model (which seems to have otherwise worked well for 10 years) is wrong, and then a generally dissatisfied-with-everything article is published somewhere...
I agree with what Joseph Conlon posted, above, about your attempt to reinterpert the graph based on statistical assumptions of your own. It would be best if you could discuss that with the author of the article at some time and inform the forum of the outcome. (However, I believe the author is heavily engaged in the FIDE Online Olympiad at present and therefore unlikely to be available for some time.)

While you, and Paul Cooksey, are right to point out factors that will make the statistics of online performances differ from those of over-the-board ones I do not think they in any significant way account for the exceptional online engine-matching performances that the graph highlights and which rarely, if ever, exist over-the-board because the use of engine assistance is so much more difficult to get away with in the real - as opposed to the virtual - world.
To find a for(u)m that accommodates the mess, that is the task of the artist now. (Samuel Beckett)

Caoimhín de Búrca
Posts: 15
Joined: Wed Jun 17, 2020 12:21 am

Re: 4NCL Online

Post by Caoimhín de Búrca » Sun Aug 16, 2020 1:18 pm

On a slightly separate note, the Junior 4NCL gets underway on Thursday, with player lists to be submitted by (I presume) Wednesday.

Will the fixtures be published tomorrow so?

Roger Lancaster
Posts: 1047
Joined: Tue Mar 17, 2015 2:44 pm

Re: 4NCL Online

Post by Roger Lancaster » Sun Aug 16, 2020 1:25 pm

Thanks to Joseph and Paul for their comments earlier. As to the respective qualities of play [measuring online, sans cheating, against over-the-board] my gut feel tends to be with Joseph although I have no evidence to support this - except that there's a study by three Dutch academics [one an IM] of comparable academic seniority to Ken Regan suggesting that strong chess players play, on average, 200 rating points higher over the board. See "Cognitive Performance in the Home Office - Evidence from Professional Chess [2020]", Maastricht university.

I'll just quote from an abstract: "We use the Artificial Intelligence embodied in a powerful chess engine to assess the quality of chess moves and associated errors. Using within-player comparisons, we find a statistically and economically significant decrease in performance when competing online compared to competing offline". Perhaps someone here more familiar with Ken Regan's approach can tell me whether he agrees with these, presumably equally respected, professors?

Of course, I would point out that this study was conducted with strong chess players only. No inference is invited regarding weaker players.

The other point I should like to put gently to Joseph is this. The 4NCL graph sets out to show actual performance relative to expected performance. If, as Joseph and many others appear to believe, average online performance tends to be worse than over-the-board performance, why are both shown on the graph with the same mean performance?

Roger Lancaster
Posts: 1047
Joined: Tue Mar 17, 2015 2:44 pm

Re: 4NCL Online

Post by Roger Lancaster » Sun Aug 16, 2020 2:32 pm

Thanks also to John. My problem is the widespread belief that Ken Regan's methodology was handed down to Moses on Mount Sinai and must therefore - something that, in fairness, Ken is at pains to deny - be correct in every last detail. That's the same mindset as the Post Office directors who decided to prosecute entirely innocent postmasters, many of whom were convicted and imprisoned, on the basis of 'infallible' expert computer evidence which in hindsight proved only too fallible. There's no lack of other examples where experts, or 'experts', have been proved wrong.

That isn't intended to pass judgment on Ken Regan's methodology, which personally I regard as "best of type", but to draw attention to the fact that - in academic and scientific circles - acceptance of a finding or theory isn't widespread until it has been reviewed by the author's peers. As far as I am aware, that hasn't happened here. In fact, I'm not even sure that Ken's findings are consistent with those of the three academics mentioned earlier. As to FIDE's apparent endorsement, before commenting on this I rather think I'd like to know exactly who within FIDE has the technical/statistical expertise to pass judgment one way or the other.

John makes the entirely legitimate point that I might usefully have discussed the matter with Alex. In fact, we fleetingly discussed the subject several weeks ago, when I indicated to Alex I wasn't entirely happy, but subsequently life has intervened - you won't need me to tell you that Alex is busy while, for my part, when not running a small internet business, I'm regularly in the present situation of needing to get together six junior teams before Thursday. So the discussion never got going.

Matthew Turner
Posts: 3026
Joined: Fri May 16, 2008 11:54 am

Re: 4NCL Online

Post by Matthew Turner » Sun Aug 16, 2020 3:06 pm

Roger Lancaster wrote
" As to the respective qualities of play [measuring online, sans cheating, against over-the-board] my gut feel tends to be with Joseph"

So the evidence that you presented as showing the 4NCL exaggerated cheating, your gut feeling now suggests underestimates cheating.

In general, I think this is correct, however we need to address the point that Paul makes. There will certainly be some players who perform better at home, because they feel more comfortable etc. etc. However, I think, in general, it is unlikely that these players will massively outperform at home, because there will be compensating factors. For example, older players might benefit from not traveling, but then it will be less likely that they will be rapidly improving.
I suggest it will only be relatively extreme examples where the benefits of playing at home will have a significant impact on the Z score. For example, if a player had been involved in a car accident then the unease of getting to the 4NCL could have a significant impact on their standard of play. I am not sure that any automated fair play system can take account of all these disparate factors, which is why it is so important that an appeals system is in place to take account of individual circumstances.

MartinCarpenter
Posts: 2669
Joined: Tue May 24, 2011 10:58 am

Re: 4NCL Online

Post by MartinCarpenter » Sun Aug 16, 2020 3:46 pm

Roger Lancaster wrote:
Sun Aug 16, 2020 1:25 pm
The other point I should like to put gently to Joseph is this. The 4NCL graph sets out to show actual performance relative to expected performance. If, as Joseph and many others appear to believe, average online performance tends to be worse than over-the-board performance, why are both shown on the graph with the same mean performance?
The graphs are that way because they simply used over the board FIDE ratings as the expected underlying playing strength for both sets of results. Not a lot else they could have done to be honest.

If the 4NCL online runs for a few years they'll know what the right adjustment factor would be to use.

The 'true mean' of the online 4NCL results does seem to be a good 0.5-1 SD over to the left side, indicating that the population does play a bit worse. A few outliers might obviously still hold steady/improve online.

Roger de Coverly
Posts: 19086
Joined: Tue Apr 15, 2008 2:51 pm

Re: 4NCL Online

Post by Roger de Coverly » Sun Aug 16, 2020 3:47 pm

Matthew Turner wrote:
Sun Aug 16, 2020 3:06 pm
which is why it is so important that an appeals system is in place to take account of individual circumstances.
With the use of chess.com and lichess servers, that's not really happening. You have bans and flags imposed by the server using undisclosed methods and those imposed by the organisers such as the 4NCL or the ECF which utilise the Regan black box.

Roger Lancaster
Posts: 1047
Joined: Tue Mar 17, 2015 2:44 pm

Re: 4NCL Online

Post by Roger Lancaster » Sun Aug 16, 2020 3:53 pm

Matthew Turner wrote:
Sun Aug 16, 2020 3:06 pm
Roger Lancaster wrote
" As to the respective qualities of play [measuring online, sans cheating, against over-the-board] my gut feel tends to be with Joseph"

So the evidence that you presented as showing the 4NCL exaggerated cheating, your gut feeling now suggests underestimates cheating.
Matt, you've asked lots of questions so suppose I ask you a question for a change. Bearing in mind that Alex's graph appears to give identical means for online and OTB performance, do you believe that average online and OTB performance is indeed equal?

Matthew Turner
Posts: 3026
Joined: Fri May 16, 2008 11:54 am

Re: 4NCL Online

Post by Matthew Turner » Sun Aug 16, 2020 4:00 pm

Roger,
I believe the online performance will be lower, so the 4NCL underestimates cheating. I believe that is right and proper and gives more protection to players from wrongful accusations.

Roger Lancaster
Posts: 1047
Joined: Tue Mar 17, 2015 2:44 pm

Re: 4NCL Online

Post by Roger Lancaster » Sun Aug 16, 2020 9:35 pm

Matthew Turner wrote:
Sun Aug 16, 2020 4:00 pm
Roger,
I believe the online performance will be lower, so the 4NCL underestimates cheating. I believe that is right and proper and gives more protection to players from wrongful accusations.
Excellent, Matt, we're making progress. Now, as regards the OTB graph, I hope you'll agree that the data reflects the differences between actual OTB performances and expected [ratings-based] OTB performances - with wider differences resulting in higher z values. For the online graph, the data reflects the differences between actual online performances and exactly what, in your opinion?

Matthew Turner
Posts: 3026
Joined: Fri May 16, 2008 11:54 am

Re: 4NCL Online

Post by Matthew Turner » Sun Aug 16, 2020 10:26 pm

The graph reflects the difference between actual online performance and expected OTB performance. If you changed the graph to reflect the difference between actual online performance and expected online performance there would be more high Z scores and more players banned for cheating. Is that the change you are wanting to see enacted?

LawrenceCooper
Posts: 5330
Joined: Tue Dec 20, 2011 8:13 am

Re: 4NCL Online

Post by LawrenceCooper » Mon Aug 17, 2020 7:23 am

LawrenceCooper wrote:
Thu Aug 13, 2020 9:19 am
LawrenceCooper wrote:
Wed Aug 12, 2020 10:29 am
215 teams registered for season 2: http://www.4ncl.co.uk/data/online_teams_2.htm
Now up to 218 so entries still being accepted? http://www.4ncl.co.uk/data/online_teams_2.htm
A further increase to 221 teams.

Roger Lancaster
Posts: 1047
Joined: Tue Mar 17, 2015 2:44 pm

Re: 4NCL Online

Post by Roger Lancaster » Mon Aug 17, 2020 7:45 am

Matthew Turner wrote:
Sun Aug 16, 2020 10:26 pm
The graph reflects the difference between actual online performance and expected OTB performance.
Except, Matt, that you answered my earlier question, "Do you believe that average online and OTB performance is indeed equal?" by saying "I believe the online performance will be lower". Let's assume that you believe it to be, on average, X rating points lower [although the same point would apply if you believed it was Y points higher] then the first X points [or Y points] difference between actual online performance and expected OTB performance has nothing whatever to do with cheating. It's a direct consequence of overall online performance being lower/higher.

Since we already seem to have established that no-one knows, with any accuracy, what X or Y might be [and, since you say "I believe", I take it that you're not even sure which applies, never mind the actual figure] then we can't eliminate it to establish the residual effect of cheating. A statistically useful graph would reflect the difference between actual online performance and expected online performance rather than expected OTB performance. In practice, I understand why 'expected online performance' wasn't used but its omission undermines the statistics.

Post Reply