Understanding False Positive Rate

Jump to the simulation

Throughout the Coronavirus pandemic, the most distressing thing has been the cherry picking of statistics.

Things have gotten better. Cases, deaths, and hospitalizations are often reported per 100k, rather than being without context. We're also getting information about the percentage of people that test positive out of all the people that were tested.

But if you test positive for an illness, what is the chance that you really have that illness?

Speaking with friends and family, the conversation around the chances of having COVID given a positive test generally goes:

• Me: What do you think the chances are of having COVID if you test positive for COVID?
• Friend/Family: What's the false positive rate of the test?
• Me: 0.5%
• F/F: Then there's a 99.5% chance that you do have COVID.
• Me: That's actually not how false positive rate works. You have to account for prevalence of the disease.

To cut to the chase, you can actually have a false positive rate of 0.5% but still have 50% of positive test results be incorrect! Or even 60, 70, 80, 90 all the way up to 100% incorrect positive results!

How is that even possible!?

Example:

Imagine you have a population of 1,000 people and we know for a fact that only one of them has COVID but we don't know which one. Someone devises a test to try to find the sick person. That test has a 10% false positive rate and 0% false negative rate.

We run that test against all 1,000 people. After testing the entire population we will have 100 people test positive for COVID. 1 person that actually has the disease and 99 (10% of 999) that are false positives.

What the above means is that given a positive test result, you still only have a 1/100 chance of having COVID.

In other words, a false positive rate of 10% under these conditions will still result in 99% of positive tests being incorrect.

Prior Probability

What is going on is that the prior probability of having a disease impacts the amount of positive test results that are in error. If 100% of the population has the disease then the number of tests in error would match the test's false positive rate. If 0% of the population has the disease then 100% of positive test results are incorrect.

To get a feel for how the different variables impact testing, you can modify the prevalence, false positive and false negative rates below to see the various outcomes.

All of this sheds a different light on case numbers. Are reported numbers only based on testing, or are they compensating for the number of incorrect test results?

To compensate for incorrect test results, you can calculate test positivity. Test positivity represents the fraction of people tested that had a positive result. If you have high test positivity then you will have many fewer positive results that are incorrect.

TODO:

• More stuff for me to write:
  • Explain Bayes theorem
  • Interactive plot of positivity vs false positives
  • On editorializing

References:
https://www.icd10monitor.com/false-positives-in-pcr-tests-for-covid-19