Monty Hall: Live from the !

Let’s envision the following fictitious scenario; in which after the success of his three door final showdown, Monty and his team have been gifted a bigger budget to make a more elaborate show, and a new, possibly infinitely big studio. Here, Monty utilises this increased budget to order his producers to purchase more doors. Now that he possesses a studio filled to the brim of disused doors, Monty again places one car behind a door and keeps count of where it is placed. While a flood of goats trundle in and hide behind the rest, he asks a contestant, now slightly more intimidated than their predecessors (see below gif), to pick a door. The contestant hesitantly chooses, giving way to our host opening every door but the contestants and one final door.

Now we again pose the question, stick or switch? Given the information we attained from the original Monty Hall situation, it makes sense to have an intuitive guess that switching will be in the contestants best interest. And we can test this again by using Bayes’ Theorem and some basic Probability theory.

Like in the original incarnation, we define events and variables for the new version. Instead of 3 doors, we now possess d doors. Each of these doors are assigned the event it may possess the prize behind it, we can define these events as D_1,...,D_d. While also allowing G to be the event we open all but doors 1 and d to reveal a goat. So with this in mind, we can formulate the following Bayes equation for any door in particular, say i:

The individual probabilities for the right hand side are calculated as:

Since D_1 to D_d cannot occur simultaneously, then we can use some more simple statistical properties, specifically , to express the probability of a goat behind all but doors 1 and d as follows in this slightly long, but hopefully intuitive derivation:

Remember we opened all but door 1 and d, so all doors in-between will have a probability 0 of having the car behind it. Thus, we substitute the above derivations back into Bayes Theorem equations, but only for doors 1 and d are,

While tedious, this derivation is pivotal in allowing a generalisation of the problem. Generalisations are crucial in mathematics, allowing us to expand our problem from an initial set of constrained numbers (like only 3 available doors) to as many doors as we want, and all we need to do is plug that number into d.

To test this – and provide a little sanity check – we can hark back to our original Monty Hall Problem and seeing that substituting 3 doors gives us probabilities of switching and staying as 2/3 and 1/3 respectively. So now we have a rather straightforward method to show, no matter how many doors we have, if we open all but one door and the original door, it is in our best interest to switch. In addition to this, although trivial to point out, we see that with more doors, our likelihood of winning when switching increases. For example, with d=7 doors, we should, by plugging our values in, attain the staying probability (i.e. door 1) as 1/7, and switching (to door d) as 6/7.

To see this in practice let’s run some simulations for 3, 5, 7 and 9 doors after carrying out Monty Hall’s deal 3000 times.

Winning isn’t everything

Our second generalisation into Monty Hall’s problem is one in which we are looking to try flip the odds back into the favour of the host. Given the generosity of winning when we expand the problem to d doors, it is now worth seeing if limiting the number of doors Monty opens can make it more – or less – likely for the contestant to win when switching. If we think about this in a logical manner, it should be the case that now we have the option of which door we can switch to, we are less likely to get the prize than in the previous scenario if we switch. But it is worth calculating how much of a detriment this new rule is to our contestant. And analysing how drastically our odds can change by opening less and less doors.

This time we will take a scheduled commercial break out of laziness to relieve ourselves of any more probability equations and focus solely on numerical computations. Consider an example where, given d doors, we open k of these. More specifically let’s look at the case of having 10 doors and we open a subset of these, analysing the number of times we win if we switch, we win if we stay, and the new third case that we don’t win if we stayed or switched. This is seen in the following graph.

As we can see in the choice between 10 doors, when opening 8 (which is the max number of doors we can open) and opening 6 (in which we have a choice as to what we can switch to), there exists a significant dip in the probability of winning when switching, with it then being more likely to not win the game whatever we do. This is due to the truly random choice we now have with the selection of the door we might want to switch to. Despite all these changes, the chance of switching consistently gives better odds than staying. We can see this in the generalisation of opening k from a set of d doors. This is given as the following equation:

For those in dire need of a Bayesian derivation (as I normally am), one can refer . As a little test, we can take d=10 and k=2 and see that the probability of winning when switching as \approx 0.129, which leaves our simulation pretty damn close to what we want.

And with this we can conclude our investigation into suspected goat farmer Monty Hall and his mystery doors. But was this investigation as concrete as the numbers suggest?

Statistical stage fright

In these two blogs we’ve seen how applying some mathematical rigour allows us to understand, dissect and create advantages in a game of seemingly random luck. With this being said, as often is the case of applying Mathematics to the real world, our logical reasoning may still not be perfect, nor reveal the true solution to the problem. since arguments could be made in that randomising the choices of contestants in the simulation and using conditional probability detracts from the human element in the game. That in which the host, the atmosphere and the audience play a crucial role in the dilemma posed to the contestant, perhaps leading to a bias in the options available. This is something that probability and random simulations simply cannot account for. Hence it could even be contested that in reality, based on the host’s hints and approach towards the contestant, the probability of finding the winning car can range from 1/2 to 1. A paper on this dilemma of the human element can be see .

Source

Pick a door, any door

In a 1975 letter to the American Statistician, Steve Selvin posed a problem loosely based off the 1960’s American TV show Let’s Make a Deal. The game consisted of three doors, two of which had goats behind them, and a third door containing a dream car. The host of the show, Monty Hall, asks the contestant to select a door. Monty then chooses a remaining door to reveal a goat. He then asks the contestant if they would like to switch to the one remaining unrevealed door, or stay with their initial choice.

Years later in 1990, Marilyn vos Savant, who rose to fame for her supposedly , was posed a similar question in her “Ask Marilyn” column for Parade magazine (!). The responses of both Selvin and vos Savant respectively, postulated that it will be in the best interest for
the contestant to switch doors.

This came as a rather counterintuitive conclusion to many as it may be initially thought that there is no difference in staying or switching, there is a still a 50% chance the car lies behind either remaining door. This idea was widely shared amongst the public, with Vos Savant. How could it be a better option to switch doors? The criticism ranged from soccer moms, to amateur Mathematicians, and even those possessing PhD’s in Maths based disciplines.

However the two savvy protagonists of this story were correct in their rational… and more importantly, had the maths to prove it.

The host knows all

To understand why it is in our best interest to switch, let’s pose the problem in a more formal sense by having a run through. We have three doors that can be labelled 1,2,3. Our esteemed host, Monty Hall, asks the contestant to pick a door. Say they pick door 1. Monty, who knows what’s behind each door, opens door 2 and reveals a goat. He then poses the question ”would you like to stick with your choice or switch to door three?”. To help quell the contestants dilemma, we can first think about the odds of initially picking the correct door. With three doors, the contestant has a 1/3 chance of picking correct straight away. Then if we remove a door, does that change our odds when switching?

One naive way to think about it is that, if each door has equal probability, surely switching makes no difference? Since removing a door will leave us with two options, we must have a 1/2 chance of winning either way? The problem with this approach is we do not account for the host knowing what is behind each door. Given that the first door possesses a probability of 1/3 of containing our Car. The other two doors must attain a 2/3 probability it lies behind either one; and behind one of these doors, must lie a
goat. Since Monty knows that, in this scenario, door 2 has a goat, this door is revealed and we are left with the option of switching to door 3. Yet we still have a 2/3 probability that the Car doesn’t lie behind the first door, so this probability carries over to represent just door 3. Hence we have probabilities of 1/ 3 for door 1, 0 for door 2, and 2/3 for door 3. Concluding that switching will be the correct decision to this dilemma.

Our newest contestant, Thomas Bayes

Wordy anecdotes are all well and good, but can we consolidate our understanding with mathematical rigour? Using some Bayesian statistics this is indeed possible (an excellent book on the topic for newcomers to the area can be ).

We start with the idea that, given two events (A and B), we know that if we are given information for one of these events, A say, then we can calculate the probability of event B happening given the information of A. We denote this \mathbb{P}[B|A] . A special formula, known as Bayes’ theorem, then states the following with this information:

We apply this to Monty Hall problem as follows. In this problem we have three doors 1,2,3 and we have the events that the prize lies between each respective door, denoted D_1, D_2, D_3. Lets suppose we select door 1 initially, and our host Monty opens door 3 to show a goat. So it follows that \mathbb{P}[D_3] =0.

Now let G be the information that there is a goat behind door 3. We can use Bayes’ Theorem to formulate the following probabilities for finding the Car behind doors 1 or 2, given the information of G, as:

We can calculate the probabilities of each quantity of the Right hand side intuitively as:

Since D_1, D_2, D_3 cannot occur simultaneously, then, we can use some more simple statistical properties, specifically mutually exclusivity to express the probability of a goat behind door 3 as:

Placing all these values back into our Bayes equations yields:

Hence, if we switch to door 2, it gives us a 2/3 chance of winning and staying with door 1 produces a 1/3 chance.

Numerical Simulation

Unfortunately I couldn’t think of a witty title for this section, but nevertheless it would still be interesting to explore a simulation and see if we can back up these theoretical results. To do this, we run the monty hall problem 1000 times, randomly generating where the car lies, which door we open and the probability of winning if we stay or switch. We should see that switching means we win about 66.6% of the time and the plot below, showing the aggregated probabilities as we run the simulation more times, support our Bayes based theory.

And with this, we close our first show. In the next blog post we explore some extensions to this problem, and answering the burning question: what if Monty Hall had access to more doors, more goats, and more contestants?

Further Reading

For another take on the Monty Hall problem, see this by a fellow STOR-i student.

Source

“It happened because during my first year at Berkeley I arrived late one day at one of Neyman’s classes.”

–

George B. Dantzig, a first year doctoral student at UC Berkeley, mistook a set of unsolved problems in statistics for a homework question. Scribbling them down and solving them over the next few days, Dantzig had found these problems slightly harder than a normal homework assignment. He threw his solutions onto the desk of his professor; expecting them to get lost within the clutter of Neyman’s office.

“About six weeks later, one Sunday morning about eight o’clock, [my wife] Anne and I were awakened by someone banging on our front door. It was Neyman.” . What the young student had done, initially unbeknownst to him, was solve these statistical problems and had a giddy professor already writing his papers introduction to be sent for publication. From this, Dantzig begun a journey into mathematical stardom.

Eight years after this tale, forever ingrained into the minds of wannabe mathematicians, Dantzig was working as a mathematical advisor for the pentagon. Tasked by his department to computationally speed up logistical issues faced by the US Air Force, he developed techniques stemming from the infant field of Linear Programming to optimise said issues. The method used was to be known as the Simplex method, but where does it come from and who are the significant players in Linear Programming?

Two more key figures

The ideas of Linear Programming in the history of mathematics often starts with Dantzig’s contributions, but its origins can be dated back to a few years earlier during World War II. Namely in the field of economics and with Soviet economist In 1939, he developed the first forms of the Linear Programming problem for organising and planning production. Cited as a founder of the field, Kantorovic’s method revolved around finding dual variables and corresponding primal solution, linking how the results from one directly impact the other. The ideas of Primal and Dual simplex are key components of linear programs, however they consist in a slightly adapted form than what Kantorovic designed. They are not explicitly covered in this post, but the eager can venture to see more on the topic. Kantorovic would later go on to win the for his work in resource allocation stemming from ideas he developed in the operations research field.

Another important character to this field, amongst many others, was . He was a proverbial rockstar in the mathematical world, dabbling in a variety of topics from quantum mechanics to game theory to the early days of computer science and most importantly to us, Linear Programming. He contributed an important aspect to this field, Duality Theory, involving the ideas of Primal and Dual Linear Programs recently touched upon. Without explicit knowledge of what this entails, it should be important to understand that this concept is pivotal to expanding and solving more complicated Linear Programs and highlighting a connection between optimisation by maximising or minimising.

Solving Linear Programs

While these aforementioned figures crafted a field in which decisions of optimality can be expressed simply as a set of linear inequalities, a key issue still remains… how do we solve these?

Recall Dantzig’s work with the Pentagon, his eventual solution to conundrums regarding optimal solutions for planning methods didn’t arise as quickly as his infamous Berkeley story (it might be of interest to know links between this story and inspiration for the movie Good Will Hunting is ). Instead, the acclaimed Simplex Algorithm was the result of an evolution from his PhD work six years prior. Here, Dantzig developed an algorithm that could solve sets of linear inequalities, with the aim of maximising or minimising some objective.

A quick example of what this objective could be is, for example, thinking of a fruit seller, figuring out how to maximise profit, with different fruits having certain purchase restraints, cost requirements, life expectancy etc.. While these problems may not have been at the upmost important to Dantzig at the time, these motivating ideas at least warrant some kind of solution that is optimal. His Simplex Method of 1947 did just that and, at the , had an incredible track record for being an effective method.

Decades later, where it is commonplace for new methods to come in and improve on the old, finding unique and novel ways to push the boundaries, the Simplex is still regarded as a strong contender for the best method at solving Linear Programs. With still employing the 75 year old method.

This blog covered a purely historical aspect of the method and some further reading can be found in these interviews and overviews of Dantzig:

A more wider history of linear programming, and its wider family of Operational Research can be uncovered in the following compendium of resources:

Source

Beyond consistent approaches

Now it may have dawned at some point that individuals, both of the human and bird-human monstrosity variety, could change their risk strategies depending on the progression of a situation. Two people that did such a thing were Milton Friedman and Leonard “Jimmy” Savage, formulating the idea that people could have multiple attitudes to risk and formulated a utility function that has both convex and concave periods. They began their argument through posing a question asking why people purchase both insurance (a risk-averse behaviour) and lottery tickets (risk-seeking behaviour), this goes against the consistent rational ideas originally found in VNM Theory. Although it should be stressed, this is simply an extension to VNM Theory and not a replacement to the ideas.

The function they derived presented a model to which it explained why at the extreme levels of wealth (both poor and super rich) risk averse patterns are observed, whereas in the situation when they are moderately wealthy, risk-seeking approaches are made. A short summary of these ideas can be found in, it is not necessary to fully understand why they came to the conclusion they did, just perhaps some self justification as to why shifting risk patterns can occur.

The overarching idea of shifting risk attitudes can now be applied to our bird example. Now, assume the bird is looking for a certain quantity of food to survive, except there is a threshold as to which the bird will turn from not having enough food (a scarcity) to having enough food (a surplus) to survive an arbitrary amount of time. Again these are not ravenous birds.

This culminates in a proposal utility function compromising of a piece-wise function (i.e. there are multiple sub-functions applied to different intervals). For when we are risk-seeking and are requiring food for survival, re-using U(x)=x^2 could be a reasonable model for this, whereas when we have enough food and purely here to maintain the quantity, U(x) = log(x) is proposed. We can see this in the below graph.

There are issues with this approach, predominantly that it assumes the bird isn’t as risk-averse as it is risk-seeking. In some sense it might make more sense for this to be the case. In this scenario we could perhaps use something akin to the the function to model this risk attitude.

With this, we have been able to construct a model for rational behaviour for this bird and that it can now go forth and implement these ideas into their food scavenging endeavours.

Conclusions and further reading

In the previous post it was noted that expected utility belonged to a cacophony of approaches, and hopefully as we progressed through these ideas on how we postulate our own rationality and the decisions we make, doubts and questions may have arisen. Namely on the mess that was the bird example, but also on is questioning if this the only way to measure rational thinking and the choices we make under uncertainty.

Prospect Theory provides another school of thought and has a prominent place within the economics field. Introduced by , they rely less on axiomatic predispositions that are assumed under VNM Theory, and instead assumes the decision maker may not have idealistic approaches to rational decision making. Therefore it aims to model what people actually behave like, with this area being more focused in the world of Economics and Psychology. For a view more closely aligned with VNM Theory, Jimmy Savage, referenced earlier with his special utility function, presented more thorough exposition of expected utility with a Bayesian approach of expected utility given by Savage’s representation theorem, contained within his work ““.

Source

As early motivation, consider a decision maker who is faced with with a set of outcomes, each with their own associated risk. Under expected utility, the main assumption is that the decision maker will aim to maximise the expected value over all possible outcomes. The function that contains all possible outcomes is referred to as a utility function and is the fundamental concept when trying to explore expected utility, especially from a mathematical perspective.

This blog aims to follow rational decision making from a mathematical viewpoint, starting from its formalised origins in the 20th century, although the ideas date back to the . With this, the four essential components to what makes a decision maker rational is presented; this is then followed by a primer on utility functions based on a school of thought known as Von Neumann-Morgernstern theory. To bring home understanding, we fly through an example that applies the theory to a problem involving the utility of food. Following this is an extension and consideration of shifting rational behaviours and a wrap up with some further theory and practical applications.

Rationality through a mathematical scope

In its mathematical sphere of interpretation, the theory stemmed from work by prominent economist Oskar Morgenstern (also attached to this project was some little known ). Explored in the opening of Section 3.2 in Ken Binmore’s fantastic book, ““, it was said Morgenstern had approached von Neumann to help formalise ideas in another field known as Game Theory. Together they developed and released the “Theory of Games and Economic Behaviour” in 1944 which packaged and cemented some neat initial results in game theory.

As a precursor to all the mathematics presented, Morgenstern was irrationally persistent in wanting the exhilaratingly high octane issue of cardinal utilities, which had been used fruitfully throughout the book, well defined and given as a basis for which their game theory ideas built upon. So as one does, Von Neumann invented a theory on the spot which measures a person’s preference to something based on the risk they are willing to take to attain it. Thus, we arrive at von Neumann-Morgenstern Theory (or VNM Theory from here forth).

Their ideas can be cemented through a set of four postulates. Under a mathematical context, postulates can be thought of as statements that are accepted as definitively true without having necessarily to prove it as such. Von Neumann and Morgenstern didn’t particularly provide the most approachable explanation of these postulates, however they can be summarised in laymen’s terms as follows.

These four postulates when satisfied constitute what is believed to be a rational decision maker. And from this, their preferences can be modelled by what is known as a utility function. But what is this?

Navigating the rabbit hole of utility functions

A key subset of the theory, utility functions, models a preference relation between utility and some
measurable quantity (wealth, food, etc.), the incorrect assumption of a utility function is something that
often occurs and leads many to come up with incorrect, and sometimes nonsensical musings on rational behaviour. A good example not explored here is the , which can help the reader understand what occurs when incorrect rational behaviour is assumed. Utility functions are essentially ways in which we can express preference as a set of ordered rankings in which we assign a utility metric known as utils to each. With this, we aim to find the expected value and to subsequently assess and explain risk preferences for individuals.

Utility functions are required to have satisfied the four axioms from VNM Theory, and can be
utilised to measure and model risk appetite. Under VNM theory, these attitudes to risk are consistent throughout, with three categories being risk-averse, risk-neutral, and risk-seeking behaviours. The first
of these is represented by a function that will slow down over time (formally defined as a concave function) and provides a preference for low variance outcomes. A risk-neutral individual is indifferent towards an outcome and can be represented by a function of just a straight line (affine). Finally, risk-seeking individuals will have more utility in achieving a higher level of outcome, therefore their function increases quicker and quicker over time (convex function). These functions are expressed in terms of U(x), which maps some quantity x such as food or wealth, to utils, an aforementioned unit for utility.

A very important addendum to all this is that these functions, and the values they produce, cannot be compared like regular numbers. Utility under this setting requires that if one value of utility is higher than another, it is just objectively preferred and not preferred so many times more than another. Take for example: if for events x_1 and x_2, U(x_1) = 42 and U(x_2) = 3, x_1 is not 14 times more preferred than x_2.

The Birds!

We can consider a quick example to see how different risk appetites arise, and how both a fixed utility function, or a variable utility function can be applied to the situation. To do this we can consider the choice a bird makes in the face of hunger.

To get rid of the trivial idea you may have that it’s not particularly feasible for a bird to be making rational human decisions. Consider some kafkaesque-light bird with some human psyche engrained within its internal cognitive function. I.e. This is not your senseless ; this is a rational bird, making rational decisions.

Now, this bird is fairly hungry, and needs some food to scavenge for, say x amount over an arbitrary amount of time. The bird has the choice as to whether go beyond what they usually are able to gather and risk dying in the process (obviously returning with nothing as they’re dead) or settle for an amount they know is safe and can be collected rather quickly.

The rational choices this bird can make, and the utility it attains from such, can be partitioned into a few scenarios:

• Scenario 1: (the bird is starving): In this scenario, a risk-seeking approach would need to be made to attain enough food for survival. To make sure this is reflected in their utility function, it would be required to see the bird value the expected utility from the amount of food it finds over the average amount of food it’s expected to find, and thus forage for food even if it’s more than should be expected.

• Scenario 2: (The bird likely has the means to attain ample amounts of food under both scenarios):
  Here, they are indifferent to the amount of food they’re expected to get, and the utility it attains from this. Therefore, it does not value taking the risk and potentially gaining a greater payoff, than it does going for a safe amount.

• Scenario 3: (The bird likely only needs a small amount of extra food): Finally, here the bird only requires a small amount of extra food and is willing to take a safe payoff for this, even if it’s less than the amount they usually expect to get.

Now, the first time I grappled with the avian madness and the overarching concepts as a whole, i found myself descending into a tangle to fully understand how this represents our decision making under risky situations. With scenes at the time of my brain trying to explain this to me roughly amounting to this:

Therefore, I often found it best to run through some numbers and to check the scenarios descriptions against the graphs of the different utility functions we saw earlier. So let us do just that:

Evaluating Scenario 3:
Here, the bird has enough food to survive, thus it is said to be risk-averse as explained in the scenario setting. From this, it’s utility function for the food quantity x could be something like ln(x) (), which is just the example function given in the earlier graphs of the previous section. From the worded description of the scenario, if we was to think of this in terms of expected value of the amount of food we expect to attain, what we are saying is that the utility of the expected value should be worth more than the expected utility. Specifically the amount of food we are expected to attain is a safe bet for us to take.

You can see how this is reflected in the chosen utility function for risk-averse behaviour by plugging in some values. Say for example we have an equal chance of attaining 2 amount or 7 amounts of food (whether that be kg, grams or some other metric is irrelevant here). Then the utility for the expected amount is just the mean of these numbers plugged into the utility function, or,

The expected value of the utility is analogously calculated as:

Using a calculator, it should be found that \ln(9/2)\geq \ln(14)/2, so taking the safe option of the expected amount has more utility to us in a risk-averse scenario, this is what was described earlier. Checks can also be done for the first two scenarios, but here we expect to value the expected value of the utility more and equally respectively.

Conclusion

This is all well and good modelling the Bird having a strict concept of rational behaviour and having a consistent approach to the matter. But what if the situation changes and evolves as we attain more food? What if the bird no longer the feels to value food as much as it once did? Or what if disaster strikes and food suddenly becomes a luxurious scarcity. The suspense is palpable!

We can answer such intriguing questions in a future blog post!

Source

“I simply wish that, in a matter which so closely concerns the well-being of the human race, no decision shall be made without all the knowledge which a little analysis and calculation can provide”

A brief history of Infectious Disease Modelling

In 1766, Swiss Mathematician Daniel Bernoulli published an article in the French literary magazine Mercure de France concerning the effect smallpox had on life expectancy and the improvements which could be made with the introduction of inoculation. Bernoulli’s concluding argument, from which the aforementioned quote is derived, led to the creation of some of the first epidemiological models.

This does not mean that the paper by Bernoulli was immediately lauded by his peers. Jean le Rond d’Alembert was a prominent mathematician and intellectual at the time; albeit slightly behind the curve in probability theory, publishing in “Croix ou Pile” his rationale behind . D’Alembert clashed with Bernoulli on the issue, authoring a rebuttal in a 1760 paper “On the application of probability theory to the inoculation of smallpox”.

Bernoulli had initially communicated his work in a 1760 presentation in Paris. However, issues caused the paper to not be released until 1766, giving d’Alembert a head start in his critique. He argued against some of the assumptions Bernoulli had made with respect to the probability of infections and the independence between age and dying of smallpox. D’Alembert’s alternative formulation also resembles modern modelling and, despite the differing opinions, both agreed that inoculation of the population was the way forward.

Bernoulli’s submission is often regarded as the foundation to what would eventually become , although the field as it is known today did not further develop until the early 20th century. This arrived in the form of work by the polymath , who wrote on malaria prevention by crafting a model of . More generalised models were also produced by William Kermack and Anderson McKendrick , their SIR model provided early forms of compartmental modelling; a class of models that form the cornerstone of much subsequent research. These models took a where no randomness is involved in the system and the output will always be replicated if given the same set of initial parameters.

What we are more interested in are another class of compartmental models that arrived not too long after deterministic models: stochastic models. These types of models included the effects of randomness commonly found in real-life scenarios. There has been many approaches to these, each applicable to certain areas. We consider a few of these models in this blog post, all of which revolve around compartmentalising the population into key components: Susceptible, Infected, and Recovered individuals. The idea being you leave one state and filter into the next. Thus, under these basic assumptions, recovery can also mean recovering 6 feet under in a casket. But let’s be positive and imagine all contractors of the hypothetical disease we discuss recover to live long and fruitful lives.

Chain Binomial Models

We can try garner understanding of stochastic models through the introduction of a simple, probability based method in chain binomials.

These models are discrete time (updates happen in an incremental step) and see where each fraction of the population is at the next time step. Some general yet subsequently quite restrictive assumptions are placed on the model. Namely:

1. The population is fixed.
2. Disease will always transfer whenever contact is made.
3. Contacts are independent (two people cannot infect one individual).
4. Infected people recover one time step after infection.

Obviously in reality this is not likely to hold on large scale populations, however in small, enclosed environments such as Hospitals, Schools and Households, the use for this model becomes more relevant.

So how does one model the movements between compartments? Say we infect I_t people at time t, then at the next time point we infect I_{t+1} people. These newly infected individuals will now leave the susceptible state by our assumption (2), and then by assumption (4), we must put the infectious people from the previous time step t into the recovered population. Mathematically we express this as:

Now the most pertinent question is how do we govern the number of infections? You can perform some pretty elementary maths to arrive at the binomial distribution (). From a pool of S_t susceptible individuals, we find the probability of infecting x people given the probability of not infecting anyone is q. You can see why we phrase this last part in such a counterintuitive way shortly.

Deterministic, or fixed updates, can be done by taking the expectation to find infections. Conversely, if you want to incorporate an air of chaos and randomness (and you actually read the title of this post), updates can also be done stochastically through a set of .

The last step of the model is how to determine the average number of infections one infectious person is expected to give in this set up, or more commonly referred to as the basic reproduction number (\mathcal{R}_{0}). Say we have a total population of N, then,

The value q, which is used as a parameter in the binomial distribution to inform on the number of infections, can essentially be recovered from this reproduction number. To see chain binomial in action we can simulate an epidemic of N=500 people and a \mathcal{R}_0=1.5.

Given the stochastic nature, it may be best to run multiple simulations as to not end up with potential anomalous results informing us incorrectly on what is likely to happen. Here, 7 iterations are chosen.

Here, given our large initial number of people, it can be seen that the chain binomial model dies out quickly with large populations and the discrete time steps lead to chunky graphs where conclusions are hard to be drawn. Therefore, this model is fairly weak and outdated to the task at hand, and it should come at no surprise that there are models that do perform better with larger populations, and a reaction based approach is considered next.

Gillespie’s Algorithm

Despite being initially formulated by the much cooler named Joseph Doob, the method was presented to the public forum by Dan Gillespie in 1976 an showcased a stochastic method to simulate the time evolution of a chemical system, through chemical reactions. This method can be stolen borrowed and applied to the epidemic setting through a re-evaluation of what these reactions can represent.

We can think of the compartments we have defined earlier and how the movement between them can be thought of as reactions between states. More specifically, two ‘reactions’ take place, an infection and a recovery. The former being a combination of a susceptible and infectious individual, and the latter being an infection contacting a recovery; which in this sense can be thought of as perhaps a healing ailment. For the algorithm itself, this offers a neat overview of what’s at play and includes an epidemiological example.

An interesting feature of this algorithm is its use of methods to dictate what reaction takes place, and its update states in a fraction of a time. This leads to both the stochastic nature we are looking for in these methods, as well as a more advantageous update strategy when compared to the often large discrete time updates with the previous chain binomial model.

It should be noted that with fixed fractional time updates, this method can be computationally expensive to implement, so a way to calculate more efficient time steps using is preferred. An example using the same set up of 500 infectious individuals, but with a slightly higher reproduction number of \mathcal{R}_0=3, to compare to the chain binomial is seen below:

Here, red, blue and green represent susceptible, infected and recovered individuals respectively, and the thick black lines give the trajectory of the epidemic if no randomness was involved.

You can see the different trajectories (included for the same reason as the chain binomial to accompany for the effects of randomness and give us a fuller picture) vary greatly around the time of most infections, but all balance out towards the end. This method of evaluation is fairly good at epidemics on large scales, and the randomness feeds into why we’d want to use stochastic models in the first place…

Takeaways and further investigations

The use of stochastic models might not be immediately obvious, from a naive point of view why would we want models that have the potential to deviate from what we expect to happen with fixed, deterministic models? In an idealised scenario where we play god, and have a clear view of how something will progress, then models that don’t factor in randomness is a clear and obvious avenue. However, this is never the case and has been seen extensively within the last few years that there is no way to truly model and predict the behaviour of humans and the rationale behind their decisions. So modelling contacts between groups and adding unpredictability in how it might happen show, even on a small simplified scale, the range of possibilities that could still potentially happen. Where deterministic models give an idea on what should happen given a set of assumptions, stochastic models provides what could if these fail or are effected by chance.

This blog just scratches the surface of possible avenues in stochastic modelling of epidemics. Another big idea in this field is Stochastic Differential Equations based approaches, which uses ideas from Stochastic calculus and financial mathematics, a potential future blog post in the making. More on this, simulation of methods, sensitivity analysis and a use case relating to malaria, can be found in my MSc dissertation .

References

Linda J. S Allen is a rockstar of stochastic mathematical epidemiology, . She has produced the following concise tutorials, as well as length books on areas of stochastic epidemic modelling:

• An Introduction to Stochastic Processes with Applications to Biology, CRC Press:
• An Introduction to Stochastic Epidemic Models, Springer Berlin Heidelberg, Berlin, Heidelberg, pp. 81–130.
• “A primer on stochastic epidemic models: Formulation, numerical simulation, and analysis”, Infect Dis Model 2(2), 128–142.

For more on the history of epidemic models, and directions to the plethora of literature on the matter, the review article : “How mathematical epidemiology became a field of biology: a commentary on anderson and may (1981) ’the population dynamics of microparasites and their invertebrate hosts” is an excellent read and is open access from here: .

Source