Posts Tagged ‘reason’
bayesian stuff again, I think
It’ll never catch on…
I’ve been reading and listening to stuff on Bayesian inference/probability for – a few years, on and off, and I’m not effectively wrapping my aged brain around it, so again I’ll embark on the almost forlorn hope that writing about it will not only help, but ultimately win the day.
Two recently read books (at least) have offered explanations to me – Steven Pinker’s Rationality (chapter 5 – ‘Beliefs and Evidence’) and Sean Carroll’s The Big Picture (chapters 9 and 10, ‘Learning about the world’ and ‘Updating our knowledge’), as well as the odd video and podcast, but I know that currently I couldn’t pass the basic test – if somebody asked me to explain Bayesian – let’s see – reasoning, statistics, inference, analysis, probability, optimisation… I’d just have to tell them something along the lines of ‘that’s for me to know and you to find out’.
But really, I know that putting this sort of stuff into my own words has worked for me in the past, and I do have some faith in my intelligence, so here goes. I won’t go into the identity of the reverend Mister Thomas Bayes, but I’ll note, as most commentators have, that Bayesian thinking has achieved massive prominence in recent decades. The reason is that not taking account of it is probably the major reason that people’s reasoning isn’t reasonable (this is me channelling Gilbert and Sullivan). So, okay, I’ll add another essential contributor, Daniel Kahneman (in Thinking fast and slow):
Bayes’ rule specifies how prior beliefs [or knowledge]… should be combined with the diagnosticity of the evidence, the degree to which it favours the hypothesis over the alternative.
which sounds a bit abstract, but it identifies a common failing in statistical analysis. One example would be that a certain suite of symptoms suggests that a patient is suffering from condition x, but the symptoms may also fit with condition y which is very prevalent in the community, whereas condition x is almost unheard of. This should not necessarily be enough to convince you, but it should affect the balance of probabilities. More data may be required. How much evidence do we need (and of what kind) to become convinced that we’re dealing with condition x rather than y?
Pinker gives the example of breast cancer. I’ll try not to quote – I want to describe it correctly in my own words! Suppose that 1% of women in a given community have breast cancer (that this is a clear, proven statistic, aka the base rate). And suppose that a test for this cancer is 90% sensitive or effective, and its false-positive rate is 9%. Woman x returns a positive test. What, as a percentage, is the chance she has breast cancer? Most people, including doctors, put it at over 80%, but using Bayes reasoning, 9% is more like it.
Of course, this is presented as an abstract case, so those tested easily forget the background statistic, the base rate (1%), and focus on the test statistic. If they were dealing with a real case they might observe other factors besides the diagnostic test, symptoms which might increase that 1% figure in their experience.
Can all this be quantified? In the case of a doctor experienced in diagnosing breast cancer, not so easy, but in the abstract case? And in the experienced case, how reliable is the experience? I’m probably setting myself too hard a task here (but to what degree of probability?). In any case, prob(hypothesis) is used to represent our degree of credence in the hypothesis (that x has breast cancer). And why such credence? The evidence, or the data supports it, or so we believe. So, prob(hypothesis)|data represents posterior probability, credence upon examination of said data.
One of the problems in our everyday reasoning is that we either ignore base rates or have little idea what they are, and having no clear idea might make that base rate diminish or rise in our estimation. Then again, there are many who believe in stuff, and fervently, when the base rate, or the prior occurrence, appears to be exactly nil. In fact, that is why they believe in them. Pinker quotes the most excellent Scottish philosopher:
No testimony is sufficient to establish a miracle, unless the testimony be of such a kind, that its falsehood would be more miraculous, than the fact, which it endeavours to establish.
If it wisnae incredibly rare and unbelievable an aw that, then it widnae be a miracle, ye ken? So it pays to be skeptical. Pinker presents an equation, of sorts, which I think is more confusing than it needs to be, considering that Bayesian thinking is more about likelihood of a hypothesis given background information or prior knowledge, and using quasi-equations doesn’t help with this (to my non-mathematical mind). A fine example is given by the vlogger Gutsick Gibbon (aka Erica) in analysing the interestingly large differences between the Y chromosomes in chimps and bonobos, and those in humans. Creationists (a semi-endangered species mostly found in the USA) have leapt upon these differences as evidence that chimps and bonobos are fundamentally different ‘created kinds’ from humans – created, that is, by their deity. In so doing, they’ve swept aside all the priors, such as our morphological and anatomical similarities vis-a-vis Pan paniscus and Pan troglodytes, and particularly the extreme closeness of our respective genomes in general. The Y chromosome contains the male sex-determining and developmental genes, and this chromosome, in both chimps and bonobos, does indeed differ substantially from the human version. Though this is curious, it hardly supports a creationist thesis (which takes us into miracle territory). Gutsick Gibbon, in a video linked below, takes us through a scientific paper which explains, based on the researchers’ hypothesis, why these two Pan species have Y chromosomes that differ markedly, not only from humans, but from other ‘great apes’, such as gorillas and orang-utans. As to the hypothesis, it’s about ‘elevated substitution rates’, about which I understand very little, but they have to do with ‘inversions’, ‘insertions’, ‘palindromic architecture’ and other technical genomic terminology, the overall point being that a lot of evolutionary development has occurred in the Y chromosomes of the Pan species in a very short amount of time, in evolutionary time scales. Here’s a couple of samples from the paper, highlighted by Erica:
The phylogenetic analysis of multi-species alignments for the X chromosome, and separately for the Y chromosome, revealed the expected species topology, but detected higher substitution rates on the Y chromosome than on the X chromosome for all the branches, consistent with male mutation bias.
These results indicate a stronger male mutation bias for the Pan lineage and a weaker bias for the Pongo [orangutan] lineage than for the human lineage. Strong male mutation bias in the Pan lineage is consistent with increased sperm production due to sperm competition.
Erica, and the paper, go into a lot more technical detail, of course, in their areas of expertise, but as she points out, this is a perfect example of the testing of reasonable hypotheses. As to the relationship with Bayesian probability or inference, it helps to inform you of what to look for. Obviously these researchers are not looking for evidence of ‘created kinds’, as that’s a dead-end ‘solution’, ultimately the product of a once-local religion made good, to the point that some of its adherents still wish to impose it on all humanity, and while my understanding of genomics is fairly limited, I can understand the importance of sperm competition in non-monogamous species well enough.
I’m not sure if all this has dealt with Bayesian stuff effectively, but I’m clearer about the general idea, I think, and that’s good enough, for now. There’s a whole area of expertise called Bayesian statistics, though, which I’m content to avoid. For now….?
References
Steven Pinker, Rationality, 2021
Sean Carroll, The Big Picture, 2016
Daniel Kahneman, Thinking fast and slow, 2011
The Dunning-Kruger effect – what does it really mean?
Canto: So we’re going to pick up on something else from the Skeptic’s Guide to the Universe (SGU), same episode in fact (931), because it’s interesting, and I’ve referred to it before, perhaps not accurately.
Jacinta: Yes, this is the thing, as autodidacts and dilettantes we learn, or try to learn, by putting things we read or hear into our own words, an attempt to own the knowledge, to make it ours – for as long as we remember or retain it. So, the Dunning-Kruger effect, which is generally, or at least by us, described as ‘we tend to think we’re smarter than we really are, because we’re not smart enough to recognise those that are smarter than we are’, or something like that.
Canto: So Dr Steven Novella, who may or may not be smarter than us, has looked into this effect for us, not for the first time, on the SGU. And we’re going to do our own job on his job.
Jacinta: So to paraphrase Novella’s account… Well, first, here’s a definition from the Decision Lab website:
The Dunning-Kruger effect occurs when a person’s lack of knowledge and skills in a certain area cause them to overestimate their own competence. By contrast, this effect also causes those who excel in a given area to think the task is simple for everyone, and underestimate their relative abilities as well.
Canto: Hmmm. I get the first part, but the second part, that’s interesting. A maths whizz – or should that be wiz, short for wizard? – is so good at simultaneously equationising, or so practised at it – that she underestimates others’ lack of knowledge/practise. We all think others are more or less like us, whether we’re dumb or smart?
Jacinta: Well here’s how Novella puts it. Dunning and Kruger observed that if you give people, say, a knowledge test, and then asked them two questions – first, how do you think you went? – say as a percentage – and second, how do you think you went, compared to others, or what percentage of others do you reckon you beat in the test? They found that mostly people were accurate predictors for the first question – that’s to say on how many questions they got right – but there was a tendency for those who did very well to underestimate their result, and for those who did very badly to overestimate. But when asked about how they did compared to others, everyone, whether they did well or badly, thought they were above average.
Canto: Right, so that means, for those who did badly, and even failed, they thought they were above average. And that’s the key finding – that’s the effect.
Jacinta: Yes, but Novella chose to talk about this because some mathematicians recently have questioned, or claimed they’ve debunked, Dunning-Kruger. And this seems to be a matter of interpretation. Dunning and Kruger claimed that the statistics show that ‘your relative lack of knowledge impairs your ability to assess your own knowledge’. The new analysis claims – and this might seem like hair-splitting – that this is an invalid inference. The alternative explanation is that everybody just thinks, or assumes, they’re above average. A kind of in-built cognitive bias that’s an evolutionary adaptation.
Canto: But of course, it can’t be true, statistically.
Jacinta: Well, of course. But it’s been a regular finding that most of us think we’re more physically attractive, better drivers, and generally smarter, which is statistically impossible.
Canto: So in the end, unsurprisingly, this effect is just part of the larger effect, that we have a higher opinion of ourselves or our various abilities or capabilities, if only slightly, than is justified by objective testing.
Jacinta: Yes, it’s like when people say ‘I can’t dance to save myself, I’ve got two left feet’, but they don’t quite believe it, they just think dancing is beneath them, and if they put the effort in…
Canto: Well maybe, but with Dunning-Kruger it’s like, you don’t know what you don’t know. And with smart people it might be not knowing quite how much smarter you are than others.
Jacinta: So, although there’s no cure, it’s always worth bearing in mind that you know less than you could know about any topic, and that there’s no skill you have that couldn’t be improved…
Canto: And then you get old and everything starts to fall apart….
References
https://thedecisionlab.com/biases/dunning-kruger-effect
https://www.theskepticsguide.org/podcasts (episode 931)
https://www.sgutranscripts.org/wiki/SGU_Episode_931
https://www.verywellmind.com/an-overview-of-the-dunning-kruger-effect-4160740
the evolution of reason: intellectualist v interactivist
In The Enigma of Reason, cognitive psychologists Hugo Mercier and Dan Sperber ask the question – What is reason for? I won’t go deeply into their own reasoning, I’m more interested in the implications of their conclusions, if correct – which I strongly suspect they are.
They looked at two claims about reason’s development, the intellectualist claim, which I might associate with Aristotelian and symbolic logic, premises and conclusions, and logical fallacies as pointed out by various sceptical podcasts and websites (and this can also be described as an individualist model of reasoning), and the interactionist model, in which reason is most effectively developed collectively.
In effect, the interactionist view is claiming that reason evolved in an interactionist environment. This suggests that it is language-dependent, or that it obviously couldn’t have its full flowering without language. Mercier and Sperber consider the use of reason in two forms – justificatory and argumentative. Justificatory reasoning tends to be lazy and easily satisfied, whereas it is in the realm of argument that reason comes into its own. We can see the flaws in the arguments of others much more readily than we can our own. This accords with the biblical saying about seeing motes in the eyes of others while being blind to the bricks in our own – or something like that. It also accords with our well-attested over-estimation of ourselves, in terms of our looks, our generosity, our physical abilities and so on.
I’m interested in this interactionist view because it also accords with my take on collaboration, participatory democracy and the bonobo way. Bonobos of course don’t have anything like human reason, not having language, but they do work together more collectively than chimps (and chimp-like humans) and show a feeling towards each other which some researchers have described as ‘spiritual’. For me, a better word would be ‘sympathetic’. Seeing the value in others’ arguments helps to take us outside of ourselves and to recognise the contribution others make to our thinking. We may even come to realise how much we rely on others for our personal development, and that we are, for better or worse, part of a larger, enriching whole. A kind of mildly antagonistic but ultimately fulfilling experience.
An important ingredient to the success of interactionist reasoning is the recognition of and respect for difference. That lazy kind of reasoning we engage in when left to ourselves can be exacerbated when our only interactions are with like-minded people. Nowadays we recognise this as a problem with social media and their algorithms. The feelings of solidarity we get with that kind of interaction can of course be very comforting but also stultifying, and they don’t generally lead to clear reasoning. For many, though, the comfort derived from solidarity outweighs the sense of clarity you might, hopefully, get from being made to recognise the flaws in your own arguments. This ghettoisation of reason, like other forms of ghettoisation, is by and large counter-productive. The problem is to prevent this from happening while reducing the ‘culture shock’ that this might entail. Within our own WEIRD (from Western Educated Industrial Rich Democratic countries) culture, where the differences aren’t so vast, being challenged by contrary arguments can be stimulating, even exhilarating. Here’s what the rich pre-industrialist Montaigne had to say on the matter:
The study of books is a languishing and feeble motion that heats not, whereas conversation teaches and exercises at once. If I converse with a strong mind and a rough disputant, he presses upon my flanks, and pricks me right and left; his imaginations stir up mine; jealousy, glory, and contention, stimulate and raise me up to something above myself; and acquiescence is a quality altogether tedious in discourse.
Nevertheless, I’ve met people who claim to hate arguments. They’re presumably not talking about philosophical discourse, but they tend to lump all forms of discord together in a negative basket. Mercier and Sperber, however, present a range of research to show that challenges to individual thinking have an improving effect – which is a good advert for diversity. But even the most basic interactions, for example between mother and child, show this effect. A young child might be asked why she took a toy from her sibling, and answer ‘because I want it’. Her mother will point out that the sibling wants it too, and/or had it first. The impact of this counter-argument may not be immediate, but given normal childhood development, it will be the beginning of the child’s road to developing more effective arguments through social interaction. In such an interactive world, reasons need to much more than purely selfish.
The authors give examples of how the the most celebrated intellects can go astray when insufficiently challenged, from dual Nobel prize-winner Linus Pauling’s overblown claims about vitamin C to Alphonse Bertillon’s ultra-convoluted testimony in favour of Albert Dreyfus’ guilt, to Thomas Jefferson’s absurdly tendentious arguments against emancipation. They also show how the standard fallacious arguments presented in logic classes can be valid under particular circumstances. Perhaps most convincingly they present evidence of how group work in which contentious topics were discussed resulted in improvements in individual essays. Those whose essay-writing was preceded by such group discussion produced more complex arguments for both sides than did those who simply read philosophical texts on the issues.
It might seem strange that a self-professed loner like me should be so drawn to an interactionist view of reason’s development. The fact is, I’ve always seen my ‘lonerdom’ as a failing, which I’ve never tried very hard to rectify. Instead, I’ve compensated by interacting with books and, more recently, podcasts, websites and videos. They’re my ‘people’, correcting and modifying my own views thorough presenting new information and perspectives (and yes, I do sometimes argue and discuss with flesh-and-blood entities). I’ve long argued that we’re the most socially constructed mammals on the planet, but Mercier and Sperber have introduced me to a new word – hypersocial – which packs more punch. This hypersocial quality of humans has undoubtedly made us, for better or worse, the dominant species on the planet. Other species can’t present us with their viewpoints, but we can at least learn from the co-operative behaviours of bonobos, cetaceans, elephants and corvids, to name a few. That’s interaction of a sort. And increased travel and globalisation of communications means we can learn about other cultures and how they manage their environments and how they have coped, or not, with the encroachments of the dominant WEIRD culture.
When I say ‘we’ I mean we, as individuals. The authors of The enigma of reason reject the idea of reason as a ‘group-level adaptation’. The benefits of interactive reason accrue to the individual, and of course this can be passed on to other receptive individuals, but the level of receptivity varies enormously. Myside bias, the default position from our solipsistic childhood, has the useful evolutionary function of self-promotion, even survival, against the world, but our hypersocial human world requires effective interaction. That’s how Australian Aboriginal culture managed to thrive in a set of sub-optimal environments for tens of thousands of years before the WEIRDs arrived, and that’s how WEIRDs have managed to transform those environments, creating a host of problems along with solutions, in a story that continues….
Reference
H Mercier & D Sperber, The enigma of reason, 2017
interactional reasoning: cognitive or myside bias?
In the previous post on this topic, I wrote of surprise as a motivator for questioning what we think we know about our world, a shaking of complacency. In fact we need to pay attention to the unexpected, because of its greater potential for harm (or benefit) than the expected. It follows that expecting the unexpected, or at least being on guard for it, is a reasonable approach. Something which disconfirms our expectations, can teach us a lot – it might be the ugly fact that undermines a beautiful theory. So, it’s in our interest to watch out for, and even seek out, information that undermines our current knowledge – though it might be pointed out that it’s rarely the person who puts forward a theory who discovers the inconvenient data that undermines it. The philosopher Karl Popper promoted ‘falsificationism’ as a way of testing and tightening our knowledge, and it’s interesting that the very title of his influential work Conjectures and refutations speaks to an interactive approach towards reasoning and evaluating ideas.
In The enigma of reason, Mercier and Sperber argue that confirmation bias can best be explained by the fact that, while most of our initial thinking about a topic is of the heuristic, fast-and-frugal kind, we then spend a great deal more time, when asked about our reasoning re a particular decision, developing post-hoc justifications. Psychological research has borne this out. The authors suggest that this is more a defence of the self, and of our reputation. They suggest that it’s more of a myside bias than a confirmation bias. Here’s an interesting example of the effect:
Deanna Kuhn, a pioneering scholar of argumentation and cognition, asked participants to take a stand on various social issues – unemployment, school failure and recidivism. Once the participants had given their opinion, they were asked to justify it. Nearly all participants obliged, readily producing reasons to support their point of view. But when they were asked to produce counterarguments to their own view, only 14 percent were consistently able to do so, most drawing a blank instead.
Mercier & Sperber, The enigma of reason, pp213-4
The authors give a number of other examples of research confirming this tendency, including one in which the participants were divided into two groups, one with high political knowledge and another with limited knowledge. The low-knowledge group were able to provide twice as many arguments for their view of an issue as arguments against, but the high-knowledge performed even more poorly, being unable to provide any arguments against. ‘Greater political knowledge only amplified their confirmation bias’. Again, the reason for this appears to be reputational. The more justifications you can find for your views and decisions, the more your reputation is enhanced, at least in your own mind. There seems no obvious benefit in finding arguments against yourself.
All of this seems very negative, and even disturbing. And it’s a problem that’s been known about for centuries. The authors quote a great passage from Francis Bacon’s Novum Organum:
The human understanding when it has once adopted an opinion… draws all things else to support and agree with it. And though there be a greater number and weight of instances to be found on the other side, yet these it either neglects and despises, or else by some distinction sets aside and rejects, in order that by this great and pernicious predetermination the authority of its former conclusions may remain inviolate.
Yet it isn’t all bad, as we shall see in future posts…
Reference
Hugo Mercier and Dan Sperber, The enigma of reason, 2017
preliminary thoughts on reasoning and reputation
In my youth I learned about syllogisms and modus ponens and modus tollens and the invalidity of arguments ad hominem and reductio ad absurdum, and valid but unsound arguments and deduction and induction and all the rest, and even wrote pages filled with ps and qs to get myself clear about it all, and then forgot about it. All that stuff was only rarely applied to everyday life, where, it seemed, our reasoning, though important, was more implicit and intuitive. What I did notice though – being a bit of a loner – was that when I did have a disagreement with someone which left a bitter taste in my mouth, I would afterwards go over the argument in my head to make it stronger, more comprehensive, more convincing and bullet-proof (and of course I would rarely get the chance to present this new and improved version). But interestingly, as part of this process, I would generally make my opponent’s argument stronger as well, even to the point of conceding some ground to her and coming to a reconciliation, out of which both of us would be reputationally enhanced.
In fact, I have to say I spend quite a bit of time having these imaginary to-and-fros, not only with ‘real people’, but often with TV pundits or politicians who’ll never know of my existence. To take another example, when many years ago I was accused of a heinous crime by a young lad to whom I was a foster-carer, I spent excessive amounts of time arguing my defence against imaginary prosecutors of fiendish trickiness, but the case was actually thrown out without my ever having, or being allowed, to say a word in a court-house, other than ‘not guilty’.
So, is all this just so much wasted energy? Well, of course not. For example, I’ve used all that reflection on the court case to give, from my perspective, a comprehensive account of what happened and why, of my view of the foster-care system and its deficiencies, of the failings of the police in the matter and so forth, to friends and interested parties, as well as in writing on my blog. And it’s the same with all the other conversations with myself – they’ve sharpened my view of the matter in hand, of people’s motivations for holding different views (or my view of their motivations), they’ve caused me to engage in research which has tightened or modified my position, and sometimes to change it altogether.
All of this is preliminary to my response to reading The enigma of reason, by Dan Sperber and Hugo Mercier, which I’m around halfway through. One of the factors they emphasise is this reputational aspect of reason. My work to justify myself in the face of a false allegation was all about restoring or shoring up my reputation, which involved not just explaining why I could not have done what I was accused of doing, but explaining why person x would accuse me of doing it, knowing I would have to contend with ‘where there’s smoke there’s fire’ views that could be put, even if nobody actually put them.
So because we’re concerned, as highly socialised creatures, with our reputations, we engage in a lot of post-hoc reasoning, which is not quite to say post-hoc rationalisation, which we tend to think of as making excuses after the fact (something we do a lot of as well). A major point that Sperber and Mercier are keen to emphasise is that we largely negotiate our way through life via pretty reliable unconscious inferences and intuitions, built up over years of experience, which we only give thought to when they’re challenged or when they fail us in some way. But of course there’s much more to their ‘new theory of human understanding’ than this. In any case much of what the book has to say makes very good sense to me, and I’ll explore this further in future posts.
What is inference?
What are you inferring?
So am I to infer from this you’re not interested?
What does inferring actually mean? What is it to ‘infer’? Does it require language? Can the birds and the bees do it? We traditionally associate inference with philosophy, which talks of deductive inference. For example, here’s a quote from Blackwell’s dictionary of cognitive science:
Inferences are made when a person (or machine) goes beyond available evidence to form a conclusion. With a deductive inference, this conclusion always follows the stated premises. In other words, if the premises are true, then the conclusion is valid. Studies of human efficiency in deductive inference involves conditional reasoning problems which follow the “if A, then B” format.
So according to this definition, only people, and machines constructed by people, can do it, deductively or otherwise. However, psychologists have pretty thoroughly demolished this view in recent years. In ‘Understanding Inference’, section 2 of their book The enigma of reason, cognitive psychologists Hugo Mercier and Dan Sperber explore our developing view of the concept.
Inference is largely based on experience. Think of Pavlov and his dogs. In his famous experiment he created an inferential association in the dogs’ minds between a bell and dinner. Hearing the bell thus set off salivation in expectation of food. The bell didn’t cause the salivation (or it wasn’t the ultimate cause), the connection was in the mind of the dog. The hearing of the bell set off a basic thought process which brought on the salivation. The dog inferred from experience, manipulated by the experimenter, that food was coming.
Mercier and Sperber approvingly quote David Hume’s common sense ideas about inference and its widespread application. Inference, he recognised, was a much more basic and universal tool than reason, and it was a necessary part of the toolkit of any sentient being. ‘Animals’, he wrote, ‘are not guided in these inferences by reasoning: Neither are children: Neither are the generality of mankind, in their ordinary actions and conclusions. Neither are philosophers themselves, who, in all the active parts of life, are, in the main, the same with the vulgar…. Nature must have provided some other principle, of more ready, and more general use and application; nor can an operation of such immense consequence in life, as that of inferring effects from causes, be trusted to the uncertain process of reasoning and argumentation’.
This is a lovely example of Humean skepticism, which flies in the face of arid logicalism, and recognises that the largely unconscious process of inference, which we would now recognise as a product of evolution, a basic survival mechanism, is more reliable in everyday life than the most brilliantly constructed logical systems.
The point is that we make inferences more or less constantly, and mostly unconsciously. The split-second decisions made in sport, for example, are all made, if not unconsciously, then with an automaticity not attributable to reason. And most of our life is lived with a similar lack of deep reflection, from inference to inference, like every other animal. Inference, then, to quote Mercier and Sperber’s gloss on Hume, is simply ‘the extraction of new information from information already available, whatever the process’. It’s what helps us slip the defender and score a goal in soccer, or prompts us to check the batteries when the remote stops working, or moves us to look forward to break-time when we smell coffee. It’s also what wags your dog’s tail when she hears familiar footsteps approaching the house.
There’s a lot more to be said, of course…
nothing so simple? the gambler’s fallacy
Humans are capable of reasoning, but not always or often very well. Daniel Kahneman’s famous book Thinking, fast and slow provides us with many examples, and not being much of a clear thinker myself, where probability and all that Bayesian stuff is concerned, I’ll start with something really simple before ascending, one day, to the simply simple. And not being much of a gambler, I’d never heard of the gambler’s fallacy before. It appears to be a simple and obvious fallacy, but I’m sure I can succeed in making it more confusing than it should be.
The fallacy involves believing that what has occurred before might dictate what happens in the future, in a particular context. It’s best explained by the tossing of a coin. With a fair coin, the probability of it landing tails up, on any toss, is .5, given that, in probability language, absolute certainty is given a value of 1, and no possibility at all is given 0. The key here is what I’ve italicised – the fallacy lies in believing that the coin, as if it’s a thinking being, has an interest in maintaining a result, over many tosses, of 50% tails – so that if results skew towards zero, say after 6 heads results in a row, the probability of the next toss being tails will rise above .5.
Put another way: assuming a fair coin, the probability of it landing heads on one toss is .5. That should mean that over time, with x number of tosses, assuming x to be a very large number, the result for a heads should approach 50%. So it would seem quite reasonable, if you were keeping count, to bet on a result that brings the average closer to 50%. That’s without imagining that the coin wants to get to 50%. It just should, shouldn’t it?
The clear answer is no. There can be no influence from the past on any new coin toss. How can there be? That would be truly weird if you think about it. The overall results may approach 50%, according to the law of large numbers, but that’s independent of particular tosses. If you look at it this way, creating a dependency, you decide to bet on a pair of tosses. It could be HH, TT, HT or TH. Those are the only four options and the probability of each of them is .25 (i.e .5 x .5). So you might think that, after two heads in a row, it would be wise to bet on tails. But this bet would still have a .5 probability of succeeding, and the result HHT, taken together, would be .5 x .5 x .5, which is .125 or one eighth, the same as all the other seven results of three coin tosses. The probability doesn’t change before each toss, no matter the result of the previous toss.
So far, so clear, but it would be hard not to be influenced into betting against a run continuing. That’s not irrational, is it? But nor is it rational, considering there’s alway a 50/50 chance with each toss. It’s just a bet. And yet… I’m reminded of Swann in a A la recherche du temps perdu, as my mind clouds over…