4NCL Online

[JustinHorton]

Yes, I'm talking in the scientific sense, can someone else run precisely what he runs.

[Joseph Conlon]

Joe,
It is important to note that we are talking about matching a computer's moves at the same rate as a 2300 as distinct from playing at 2300. So Magnus Carlsen is better than me, not only because he plays more computer moves, but also because he is better than me at knowing when it is important to play a computer move.

So, it is perfectly conceivable that a 1500 could 'perform' at 2300 by selecting the computer move half the time rather than all the time. In reality though, I am not sure that this would persist for very long, they would either get bored and stop or be drawn into playing more computer moves.

Thanks Matt. Once you said it, it made perfect sense - selecting computer moves at the same rate as a 2300 can have large statistical significance, even if the play is not remotely at the level of a 2300 (and wouldn't be if a 1500 was picking computer moves for some fraction of moves and for the rest playing whatever their first choice was).

This point also shows the deficiency of many arguments along the line of 'I looked at X's games and they didn't seem anything special; in fact in several he blundered and ended up losing.' - huge statistical outliers are still huge statistical outliers which require an explanation, whatever the actual result of the game.

BTW I found your earlier point compelling - if you have any measure of 'accuracy of play', and then find this measure produces far more extreme values in online play than in OTB play (or within two subsets of online play by the same player), that is strong evidence of assistance even if in any one case one could adduce arguments to explain this away. In this respect I note that lichess stopped women's prizes in its Titled Arenas, because it said they had detected too much account sharing.

Regarding levels of significance generally: in particle physics one reason 5-sigma is used is because even if you have lucky fluctuations, underestimate your errors etc, then you would have still have at least 3-4 sigma of significance. This is a similar idea to what someone [edit: actually you] mentioned earlier in the thread, namely take whatever significance you would like to have for a ban, and then add one sigma to it (to account for good days, freshness vs tiredness etc etc).

That said, there are still 5 sigmas and 5 sigmas (there is at least one dark matter experiment that claims > 10 sigma significance and no-one else believes them). Two mutually supporting 3.5 sigmas from different directions (which combine to 5 sigma) is perhaps much better than a single 5 sigma from one indicator.

Regarding the scientific method, peer review etc: this scientist would, perhaps cynically, regard it as having some analogue to sausage making: if you like the product, don't look too closely at what goes into it.

[JustinHorton]

Any chance of anybody explaining sigma to the non-scientists

[IM Jack Rudd]

Sigma is the standard deviation of a distribution. If the distribution is Normal, roughly two-thirds of all results will fall within one sigma of the mean, and nineteen-twentieths of all results will fall within two sigmas of it.

[JustinHorton]

That may possibly shift the locus of incomprehension rather than remove it

[Ian Thompson]

Sigma is the standard deviation of a distribution. If the distribution is Normal, roughly two-thirds of all results will fall within one sigma of the mean, and nineteen-twentieths of all results will fall within two sigmas of it.

as in this graph:

Standard_deviation_diagram.svg.png

[Matt Fletcher]

Any chance of anybody explaining sigma to the non-scientists

Don’t know if I’m going to help or not but I’ll give it a go.

As Jack mentioned, sigma refers to the standard deviation - this is basically just a measure of how much a given quantity varies from the average.

Taking male height as an example - the average is about 5’10” with standard deviation (sigma) of about 4 inches. So someone with a 1 sigma variation from the average would be 6’2”, quite tall but not particularly remarkable. A 4 sigma variation would be 16 inches above the average, or 7’2” - very much an extreme outlier.

[JustinHorton]

Thanks all

[NickFaulks]

.

[Joseph Conlon]

The graph above is the real deal; saying something is 2 sigma, or 3 sigma, or 4 sigma, is a measure of how often something is likely to happen.
Let me put another (identical, but differently phrased) scale to this:

Suppose chess player Knightie McKnightFace plays six tournaments a year and has a stable rating. Then

1 sigma outperformance = happens 1 time in 6, the best tournament performance they play that year

2 sigma outperformance = happens 1 time in 50, a once-in-a-decade performance, one of the few best tournaments of their life

3 sigma outperformance = happens 1 time in a 1000, the sort of performance a very active player going to tournaments every other weekend delivers once in their entire life

4 sigma outperformance = happens 1 time in 30,000, the best tournament performance that an active player has ever been present at - something roughly comparable to winning a 10-player all-play-all against equal opposition with 9/9.

5 sigma outperformance = happens 1 time in 3,000,000 - take all the GMs in the world there have ever been, take every tournament they have ever played in, from when they were U7 to when they died, and you expect that not one of these performances reach this standard - on a very rough estimate, this is something like a player ~190 - 200 (say you or me) playing in the main British championship and starting off with 6/6 or 7/7.

In a physics context, stuff under 3 sigma often goes away - in chess an active arbiter might see several 3 sigma performances a year. Once you get to 4 sigma (is this the same as Regan z=4?) or 5 sigma (or more) you are really in the 'what is going on?' category.

[Richard Bates]

So anyway, just to clarify re: Regan tool. If you are still within known theory you can cheat to your heart’s content?!

[Nick Burrows]

Suppose chess player Knightie McKnightFace plays six tournaments a year

My next online handle

[MartinCarpenter]

So anyway, just to clarify re: Regan tool. If you are still within known theory you can cheat to your heart’s content?!

Yes, so long as the tool knows that you're in known theory..... Which must be a genuinely very difficult task these days.

My a priori assumption was definitely that there'd be relatively much more cheating by looking up opening notes etc in the 4NCL online than by copying engine moves.

[John McKenna]

My take on it (based on A. Elo's original systen) -

In the Elo system the interval of a category is defined as one Standard Deviation (1 sigma) [corresponding to a band 200 Elo points wide with 2000 "as a reference point" both "already steeped in tradition" when Elo chose to use them. JM]

Rating categories (based on a 200 point interval and with 2000 as midpoint) -

3000+ Artificial Intelligences
2800+ Human World Champion & contenders
2600+ Strong GMs
2400+ IMs & weaker GMs
2200+ FMs & National Masters
2000+ Candidate Masters & other experts
1800+ Category 1 amateurs
1600+ Category 2 amateurs
1400+ Category 3 amateurs
1200+ Category 4 amateurs
1000+ Novices

Each step up/down from the 2000 midpoint is 200 points and thus 1 sigma (one Standard Deviation).

For a player rated exactly 2000 a -

2200+ performance is 1+ sigma
2400+ performance is 2+ sigma
2600+ performance is 3+ sigma
2800+ performance is 4+ sigma
3000+ performance is 5+ sigma
3200+ performance is 6+ sigma

And so on...

The theory underpinning it is given by A. Elo -

A player will perform around some average level... Deviations occur... (with) large deviations less frequently than small ones. These facts suggest the basic assumption of the Elo system -

The many performances of an individual will be normally distributed... [as shown by the well-known symmetrical bell curve, as given further above in this thread. JM]

"Extensive investigation (Elo 1965, McClintock 1977) bore out the validity of this assumption. Alternative assumptions are discussed..." [elsewhere - JM]

"Statistical and probability theory provides a widely used measure of these performance spreads [deviations from the average - JM] a measure which has worked quite well for many other natural phenomena... Standard Deviation." [1 SD is denoted by 1 sigma - JM]

"The central bulk [of the bell curve] - about two-thirds [68% JM] - of an individual's performances lie within 2 Standard Deviations" [i.e. minus 1 to plus 1 SD (-1 sigma to +1 sigma. And that leaves 32% of the player's performances outside of that central bulk. With 16% higher than +1 sigma and the remaining 16% being lower than -1 sigma].

Q.E.D. (At least I'd like to think so.)

[IM Jack Rudd]

Be careful with this analysis: five-round tournaments and nine-round tournaments will probably show significantly different numbers when it comes to variation around a TPR.