Field Notes from an Uncertain Science
By: Siddhartha Mukherjee
Published: 2015
Read: 2020
Summary:
Despite the increasing accuracy of medical tests and other advances, much of modern medicine still takes place under conditions of incomplete information and uncertainty, partly because more complex problems are investigated (using better machines to take care of sicker people).
The author formulates three laws aimed at improving (medical) decision making in the face of uncertain, inaccurate and imperfect information.
- Priors: use prior knowledge to make better predictions.
- Outliers: use new data that proves a prediction wrong to change your model.
- Biases: understand the limits of your model, when to generalize and when not to.
The three laws explore the limits and constraints of human decision making: how to reconcile prior knowledge (certain, fixed, perfect, concrete) with complex reality (uncertain, fluid, imperfect, abstract). As such, the laws apply more broadly to any discipline that involves the manipulation of knowledge under uncertainty.
Worth Reading:
Short and to the point. A book about “How to Practice Medicine” rather than “The Laws of Medicine”: general rules rather than specific laws; a “how to” rather than a “what is”.
The first rule somewhat aligns with examining a world where patterns repeat (Bayesian) and the second rule with one where interactions become complex (non-Bayesian). As knowledge and science advance, we are more likely to encounter less of the first and more of the second type of environment. Unfortunately, we seem to be better at using the first (interpreting patterns) rather than the second (handling outliers) rule. People are probably more strongly wired to form beliefs (survival) rather than to change them (adapt).
Convincingly makes the point that modern medicine is still a very young science and our understanding of the some of most basic diseases is still very limited, making evidence-based medical practice difficult. Perhaps at a more basic level than acknowledged by the author. For instance, we apparently don’t even know whether a common fever is “good” or “bad”, when or even whether we should “let it ride” (fever is a useful adaptive reaction) or treat it (it no longer is a useful adaptive reaction of the body).
Key Takeaways:
- The more complex the underlying process, the more difficult it is to find universal laws.
- Physics -> chemistry -> biology -> medicine (economics, etc.)
- Uncertainty increases, level of understanding decreases.
- Handling uncertainty is about manipulating beliefs.
- Predict: use and test your beliefs.
- Be open: adjust your beliefs.
- Be aware: beliefs have limits.
Key Concepts:
Development of medical science
- Early medicine involved largely ineffective therapeutic interventions.
- Placebos.
- Effect through psychosomatic reactions in patients.
- Elixirs for weakness, ageing, depression.
- Palliatives.
- Genuinely effective drugs to ameliorate symptoms, pains.
- Morphine, opium, alcohol, etc.
- Purging.
- Purge the stomach, intestines to disgorge poisons.
- Laxatives, purgatives, etc.
- Placebos.
- Innovation: do nothing and observe.
- Pathophysiology.
- Platform for modern medicine.
- New paradigm: make medicine more scientific.
- Use information from observation.
- Develop therapeutic interventions based on rational precepts.
- Problem of modern medicine: understanding is still incomplete.
- Models produce the illusion of a systematic understanding of a disease.
- Many rules to understand normalcy.
- Lack a deeper, more unified understanding of physiology and pathology.
Find rules to help reduce uncertainty
- Sciences have laws: rules that nature must live by.
- Statements of truth.
- Based on repeated experimental observations.
- Describe some universal or generalizeable attributes of nature.
- The more complicated the underlying process, the more difficult it is to find these laws.
- Physics: many.
- Chemistry: less.
- Biology: very few.
- Rules laws of medicine?
- Need to distill universal guiding principles of medicine into a statement of truth.
- Practical, guiding rules.
Law 1: A strong intuition is much more powerful than a weak test.
- Weak test = poor predictive power.
- Predictions lack accuracy and/or consistency.
- High levels of false positives.
- Especially when true positive incidence rate is low.
- Intuition = prior knowledge = assessing prior probabilities.
- Weigh prior evidence to assess prior probabilities.
- Prior probabilities can help overcome the weakness of a test.
- Playing with probabilities.
- By changing the structure of the tested population.
- In medical context: ask questions before doing a test.
- Before questions: chance of having a certain disease are 1 in x.
- After questions: chance of having the disease may change to 1 in y.
- May be difficult for technology to replicate.
- Applies to any discipline that relies on predicting.
- Bayes’ Theorem.
- There is no absolute knowledge, there is only conditional knowledge.
- History repeats itself, and so do statistical patterns.
- The past is the best guide to the future.
- A test is not a predictor of perfect truths.
- It is a machine that modifies probabilities.
- It takes information in and puts information out.
Law Two: “Normals” teach us rules; “outliers” teach us laws.
- Outliers: new data that does not fit with existing models.
- Indication that normal rules are flawed.
- Understanding is incomplete.
- Provides route to potentially new ways of organizing knowledge.
Law Three: For every perfect medical experiment, there is a perfect human bias.
- Every science suffers from human biases.
- Science relies on data.
- Humans are the final observers, interpreters, and arbiters of data.
- Worse in sciences where humans are the object of study or active participants.
- Not necessarily ruled out through controlled, randomized, double-blind studies.
- To what extent are results generalizable (different genders, geographies, etc.)
- Big data / new (medical) technologies may increase bias.
- Still need humans to interpret, make sense.
- “Doctors hunt bias, not diseases”.
Links: