Nice bit of maths on the BBC

[Paul McKeown]

Okay - for the brainiacs it will be elementary.

But the link gives the best explanation I have seen of the Monty Hall Problem.

Followed by a nice question with an unintuitive answer illustrating the dangers of not thinking through what medical diagnostic tests are telling us. Journalists almost always get this sort of thinking wrong, of course. An interesting question is why.

[Christopher Kreuzer]

Okay - for the brainiacs it will be elementary.

But the link gives the best explanation I have seen of the Monty Hall Problem.

Followed by a nice question with an unintuitive answer illustrating the dangers of not thinking through what medical diagnostic tests are telling us. Journalists almost always get this sort of thinking wrong, of course. An interesting question is why.

That medical diagnostic mathematical puzzler is:

"A diagnostic test has been developed which performs as follows - if you have the disease, the test has a 99% chance of giving the result "positive", while if you do not have the disease, the test has 2% chance of (falsely) giving the result "positive".

A randomly chosen person takes the test. If they get the result "positive", what is the probability that they actually have the disease? The answer, 1/3, is perhaps surprisingly low. "

So can anyone explain that? Is it to do with the 1% chance the positive test is incorrect and the 2% chance a positive test is incorrect sort of coming together to make the answer 1/3? I'm sure there is a simple way to work it out, but probability calculations aren't that natural to most people (or rather, the intuitive calculations humans use aren't based on probability but more on, well, intuition).

[Mike Gunn]

Isn't there a basic piece of information missing, i.e. what proportion of people in the population have the disease?

[Paul McKeown]

Absolutely right, Christopher omitted the necessary introduction:

Next year we will need something different, perhaps Simpson's Paradox. Imagine that 1% of people have a certain disease.

[Paul McKeown]

Christopher, given the information, ask yourself, if you applied the test to 100 people how many people would expect to have been diagnosed (correctly and incorrectly) as having the disease? After that it's all obvious.

[PeterFarr]

So if you imagine a population of 100, all of whom took the test, then 2 would get a false positive, 1 would get a real positive, and 97 would get a negative. So out of every 3 positives, only 1 is real.

I suppose the reason it's non-intuitive is that it's easy to forget that the very small chance of having the disease in the first place makes a big difference.

[MartinCarpenter]

It really is utterly amazing how bad we instincively are at the monty hall problem. I can still remember sundry insane conversations on rec.games.bridge about the principle of restricted choice....

That Simpsons thing surely doesn't surprise people though? Much easier to work out. I think the thing that surprises people there is when they here of a 99 per cent reliable test and then wonder why it isn't being used without knowing the proportions etc.

[Roger de Coverly]

i.e. what proportion of people in the population have the disease?

The general reasoning is that if you select 100 people and perform the test, you expect to detect the one person who has the disease (well 99% of one person, so not absolute) but also 2 people ( or 99% of 2 people) who don't. So taking three people who have tested positive, only one has the disease.

If the witch finders ever start screening all games for computer usage, they may well find a similar effect.

[Roger de Coverly]

It really is utterly amazing how bad we instincively are at the monty hall problem. I can still remember sundry insane conversations on rec.games.bridge about the principle of restricted choice....

The critical point is noting that the host is bending the rules by cheating, that is by making an initial selection to open that he knows doesn't contain the prize. If the initial selection is random, the intuitive 50% still applies.

[PeterFarr]

i.e. what proportion of people in the population have the disease?

The general reasoning is that if you select 100 people and perform the test, you expect to detect the one person who has the disease (well 99% of one person, so not absolute) but also 2 people ( or 99% of 2 people) who don't. So taking three people who have tested positive, only one has the disease.

If the witch finders ever start screening all games for computer usage, they may well find a similar effect.

There's something in that. Given a big enough sample of games, some players will presumably fluke a close approximation to an engine - monkeys typing Shakespeare etc. Guess you need a decent sample of games for one player - and preferably corroborative evidence, before proving anything.

[Christopher Kreuzer]

Absolutely right, Christopher omitted the necessary introduction:

Next year we will need something different, perhaps Simpson's Paradox. Imagine that 1% of people have a certain disease.

Oops! It does make more sense now...

[Christopher Kreuzer]

Isn't there a basic piece of information missing, i.e. what proportion of people in the population have the disease?

As Paul has pointed out, I missed that out. But surely working out what proportion of people in the population have the disease is dependent on the accuracy of tests that diagnose the disease. Does it not get a bit circular? Having said that, I have no idea how the incidence of false positives is worked out. Is a second test used to corroborate the results of the first one, or is it a case of wait and see if they develop symptoms?

[Simon Brown]

Um....I think it is hypothetical, and doesn't stand up to logical analysis. You might ask why someone hasn't developed a better test, given that if you test positive you are still more likely not to have the disease than have it.

Going back to Monty Hall, if you have any sort of aptitude for maths, it's pretty clear. My son when 11 (and a Deal or No Deal fan, incidentally) got it immediately, even when I used goats and Ferraris (he likes F1 as well).

[Paul McKeown]

Um....I think it is hypothetical, and doesn't stand up to logical analysis. You might ask why someone hasn't developed a better test, given that if you test positive you are still more likely not to have the disease than have it.

Because it ain't easy, p'raps?

[Paul McKeown]

But surely working out what proportion of people in the population have the disease is dependent on the accuracy of tests that diagnose the disease. Does it not get a bit circular? Having said that, I have no idea how the incidence of false positives is worked out. Is a second test used to corroborate the results of the first one, or is it a case of wait and see if they develop symptoms?

Presumably the disease has been characterised before anyone attempted to produce an automatic form of testing. So symptoms, prognosis and epidemiology will already have been well established. It's then possible to compare between the diagnoses provided by a specialist and those from the test.

What the example perhaps should illustrate is that the test should be another tool in the specialist's arsenal. It could help confirm his initial conclusions, Bayesian diagnostics if you like. However, the test, because it applies to a low prevalence disease, and because the rate of false positive is high, should never be used to screen a general population, whose members generally will not be expected to have the disease. That being said, there might be a case if the disease was very dangerous and the risks of treatment low. Journalists tend to approach these topics like Brut aftershave salesman chasing sales at a coal miners conference. [Was it Hartston who first used that metaphor?]