Improbability.

Discuss anything you like about chess related matters in this forum.
Keith Arkell
Posts: 928
Joined: Tue Dec 30, 2008 2:10 am

Improbability.

Post by Keith Arkell » Fri Nov 27, 2015 3:33 am

With insomnia my only excuse, I thought I'd share a curious statistic:

In the last 12 months I've played in 14 tournaments, and in every one of them had black in the last round.

I know that probability can be a tricky subject, and that any given uninteresting sequence is equally improbable before the outset, but it seems to me that the chances of this significant sequence occurring are 16,383 to 1. In other words a player participating in 14 tournaments every year would expect such a run approximately once every 16384 years!

Perhaps the odds are even longer than this, given that players on a plus score tend to get white slightly more than those on a minus score ( because in chess white scores approximately 54 %).

Here is the list, which, apart from none - FIDE rated Blackpool, is verifiable here:

https://ratings.fide.com/hist.phtml?event=400270

London Classic..Kerrigan.......(2277)........ B..1
Hastings..........La Garde......(GM 2576)....B..1/2
Malta.............Ahlander.......(IM 2406).....B..1/2
Blackpool........Allis.............(180 ECF)....B..1
West Eng.Ch....Brucey..........(2093)........B..1
Bristol............Meek...........(2093).........B..1
Euro Senior.....Bellia............(IM 2433).....B..1/2
South Wales....Chatalbashev...(GM 2516)....B..0
British Ch........Hawkins........(GM 2554)....B..0
Vienna............Gevorgyan.....(WIM 2206)..B...1
Coulsdon.........Cherniaev......(GM 2491)....B..1/2
Paignton.........Wheeler........(2117).........B..1
I. O. Man........ Tan..............(IM 2445).....B..1
World Senior.....Nikolic.........(GM 2603).....B..0
Last edited by Keith Arkell on Fri Nov 27, 2015 2:34 pm, edited 1 time in total.

User avatar
MJMcCready
Posts: 3178
Joined: Mon Jun 24, 2013 2:30 pm

Re: Improbability.

Post by MJMcCready » Fri Nov 27, 2015 5:13 am

Just think of all the money you could have won with foresight.

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: Improbability.

Post by Roger de Coverly » Fri Nov 27, 2015 9:15 am

Keith Arkell wrote:but it seems to me that the chances of this significant sequence occurring are 16,383 to 1.
It's probably lower than that, because given that tournaments will have similar numbers of rounds, similar fields and similar pairing rules, the assumption that events are independent you need to justify multiplying probabilities will break down. You might want to check what colour you had in the first round. In an event with an odd number of rounds, a player is likely to repeat his first round colour in the last round and reverse it in an event with an even number. That applies particularly to higher seeds given the colour preference rule of giving priority to the higher seed.

User avatar
Michael Farthing
Posts: 2069
Joined: Fri Apr 04, 2014 1:28 pm
Location: Morecambe, Europe

Re: Improbability.

Post by Michael Farthing » Fri Nov 27, 2015 9:44 am

There is also the fact that you have the benefit of hindsight. There are nine equally memorable and improbable similar occurrences that would have caused you the same surprise: White in all last rounds and all black or all white in rounds 1-4 (assuming each congress had at leats 5 rounds). The odds of one of these happening comes down to more like 1700:1 (I haven't done the maths, which is not as straightforward as just dividing by 10). Still significant, but if beforehand you have a sufficient list of individually unlikely events the chances that one of them will occur will not be that dramatic.

This, of course, is the trick regularly used by the BBC weather with comments like 'And November saw a new record set for the coldest month ever recorded in Market Snodsbury' and 'rainfall of only 10mm in Barchester is the driest since records were begun by Archdeacon Grantley in the 1860s'

Neill Cooper
Posts: 1298
Joined: Sun Feb 24, 2008 4:43 pm
Location: Cumbria

Re: Improbability.

Post by Neill Cooper » Fri Nov 27, 2015 10:08 am

David Spiegelhalter (Winton Professor of the Public Understanding of Risk at the University of Cambridge) would be interested in this. See his interesting website: http://understandinguncertainty.org/coincidences and the write up of his BBC TV programme on Coincidences "

Alistair Campbell
Posts: 379
Joined: Sat Mar 06, 2010 12:53 pm

Re: Improbability.

Post by Alistair Campbell » Fri Nov 27, 2015 1:07 pm

Keith Arkell wrote:In the last 12 months I've played in 14 tournaments, and in every one of them had black in the last round.
Perhaps you have offended the ne*us in some way? In which case you may wish to sacrifice a goat or bake a large pie as per the discussion here

(Incidentally, I can't trace the original post for the above discussion; I assume the thread was excised from the original in some way).

What colour did you have in the first game of each tournament?

Also, I don't understand this bit
Keith Arkell wrote:Perhaps the odds are even longer than this, given that players on a plus score tend to get white slightly more than those on a minus score ( because in chess white scores approximately 54 %).
I can see the opposite is true - that players who get white slightly more would tend to be on a plus score.

Also also, do you know the ratio of whites to blacks throughout your (or anyone's) career?

John McKenna

Re: Improbability.

Post by John McKenna » Fri Nov 27, 2015 1:35 pm

Alistair Campbell wrote:
Keith Arkell wrote:In the last 12 months I've played in 14 tournaments, and in every one of them had black in the last round.
Perhaps you have offended the ne*us in some way? In which case you may wish to sacrifice a goat or bake a large pie as per the discussion here

(Incidentally, I can't trace the original post for the above discussion; I assume the thread was excised from the original in some way)...
The answer is in the thread you linked to -
LawrenceCooper wrote:
Paul Georghiou wrote:
So am I just unfortunate? Or is something else going on? And should something be done about it?
Well you had white in your only game against me, not sure if you managed more whites or blacks in the tournament though :lol:
So, Paul G. was the originator, but he and his own posts seem to have been "excised" from the forum.

Keith Arkell
Posts: 928
Joined: Tue Dec 30, 2008 2:10 am

Re: Improbability.

Post by Keith Arkell » Fri Nov 27, 2015 3:59 pm

Error corrected, after I somehow combined Bristol and the Euro Senior into a single event ( well, it was at 3 in the morning).
MJMcCready wrote:Just think of all the money you could have won with foresight.
Soz but I'm missing the point here completely.
Roger de Coverly wrote:
Keith Arkell wrote:but it seems to me that the chances of this significant sequence occurring are 16,383 to 1.
It's probably lower than that, because given that tournaments will have similar numbers of rounds, similar fields and similar pairing rules, the assumption that events are independent you need to justify multiplying probabilities will break down. You might want to check what colour you had in the first round. In an event with an odd number of rounds, a player is likely to repeat his first round colour in the last round and reverse it in an event with an even number. That applies particularly to higher seeds given the colour preference rule of giving priority to the higher seed.
13 of the 14 tournaments had an odd number of rounds ( 2x5, 2x7, 6x9, 1x10, 2x11 and 1x 13), and I had white in round one in 7 out of the 14.

I would have thought that any similarities in types of events ought to be completely negated by the rule that the arbiter tosses a coin for the colour of the top seed in round one, after which the higher scorer, or higher rated player has priority over keeping sequence. I see no colour bias in any of this.

At the British Ch the top seed alternates round 1 colour each year, and Malta, uniquely, was a Round Robin, drawn randomly.
Michael Farthing wrote:There is also the fact that you have the benefit of hindsight. There are nine equally memorable and improbable similar occurrences that would have caused you the same surprise:..... Still significant, but if beforehand you have a sufficient list of individually unlikely events the chances that one of them will occur will not be that dramatic.

This, of course, is the trick regularly used by the BBC weather with comments like 'And November saw a new record set for the coldest month ever recorded in Market Snodsbury' and 'rainfall of only 10mm in Barchester is the driest since records were begun by Archdeacon Grantley in the 1860s'
Yes, I understand this and agree, Michael. That is why I began tentatively ( 'I know that probability can be a tricky subject, bla bla bla'), but it started to become interesting to me about 8 months ago, and I have watched the sequence continue event after event since then.


Alistair Campbell wrote: Also, I don't understand this bit
Keith Arkell wrote:Perhaps the odds are even longer than this, given that players on a plus score tend to get white slightly more than those on a minus score ( because in chess white scores approximately 54 %).
I can see the opposite is true - that players who get white slightly more would tend to be on a plus score.

Also also, do you know the ratio of whites to blacks throughout your (or anyone's) career?
Maybe I worded it badly, but my point was that because White scores about 54% in chess in general, therefore lower scorers in tournaments are more likely to have a long - term deficit of Whites and higher scorers vice=versa, and given that I am usually a higher scorer, therefore my number of whites in tournaments ought to be a long term plus ( including in the last rounds). This is interesting though, and I haven't checked it yet..

Neville Belinfante
Posts: 170
Joined: Sun Jan 24, 2010 10:23 pm

Re: Improbability.

Post by Neville Belinfante » Sat Nov 28, 2015 11:12 am

I am confused by this thread. There is a reference above to a link to a post by Paul Georgiou. I am not aware that he is or ever has been on this forum.

As regards its main point, odds of 16000 to 1 mean the probability of such a long sequence happening to Keith is extremely low. However considering that there are more than 16000 chess players in the country, means that the probability of such a sequence happening to someone is high.

E Michael White
Posts: 1420
Joined: Fri Jun 01, 2007 6:31 pm

Re: Improbability.

Post by E Michael White » Sat Nov 28, 2015 11:23 am

Neville Belinfante wrote:means that the probability of such a sequence happening to someone is high.
Oh no it doesn't ! Keith is estimating a conditional probability. Very few players play 14 tournaments in a year.

stevencarr

Re: Improbability.

Post by stevencarr » Sat Nov 28, 2015 1:23 pm

To change the subject a lot, but keep it on colours and probabiliites, I can guess the sequence of runs of Black and Red cards in a randomly shuffled pack of cards better than anybody else in the world.

To be more specific, if you name a 3 card run of colours in a shuffled pack (say BBB, or RRB, or BRB) I will predict a sequence that will come up more often than yours in the pack.

Not many people understand conditional probability....

Alistair Campbell
Posts: 379
Joined: Sat Mar 06, 2010 12:53 pm

Re: Improbability.

Post by Alistair Campbell » Sat Nov 28, 2015 3:55 pm

Keith Arkell wrote: and I had white in round one in 7 out of the 14.
Thanks. That rules out one tentative theory.
Keith Arkell wrote:Maybe I worded it badly, but my point was that because White scores about 54% in chess in general, therefore lower scorers in tournaments are more likely to have a long - term deficit of Whites and higher scorers vice=versa, and given that I am usually a higher scorer, therefore my number of whites in tournaments ought to be a long term plus ( including in the last rounds). This is interesting though, and I haven't checked it yet..
This sort of thing give me a headache. :cry:

I can believe that propensity to get white is correlated with the propensity to score relatively well. However, I can’t believe that scoring well causes the propensity to get white. I think low scoring is more likely to be better correlated with being a relatively worse player.

If people played in infinitely large swisses or leagues then would you expect performance to tend towards 50%? The more you won (lost) the more likely you are to be paired with better(worse) players or play a higher(lower) board or in a higher(lower) division. Only those at the top or bottom of the scales would tend to out- or under-perform. Of course, one can get into a rut where one is always a top player in a major or a lower player in an open and performance is biased appropriately. Similarly, colour bias could be introduced by a tendency only to play home games for example (if colours are predetermined) or choosing to take black on odds if the toss for colours is won.
Neville Belinfante wrote:I am confused by this thread. There is a reference above to a link to a post by Paul Georgiou. I am not aware that he is or ever has been on this forum.
I have a vague recollection that the complaint was over the number of tournaments in which the poster found himself with 3 blacks out of 5 (rather than a black in the last round). Why the post was deleted I’ve no idea.

I’ve checked my database of my own games – 555 Whites (scoring 56.2%) and a massive 621 Blacks (scoring 49.5%). Overall I think that means white scores 53.2% and I’ve had 47.2% whites. I guess neither is remarkable.

MartinCarpenter
Posts: 3044
Joined: Tue May 24, 2011 10:58 am

Re: Improbability.

Post by MartinCarpenter » Sat Nov 28, 2015 6:33 pm

Quickest way to get a propensity of whites is to score unusually badly with black ;) (Leagues only of course.).

The other complication for colours/expected results for someone like Keith in a weekender is that for a lot of the rounds there's a huge strength difference, when the colours won't affect the expected results that much.