April 20, 2020

COVID-19

COVID-19: What’s wrong with the models?

Originally written as an internal memo on April 15, 2020; updated and shared publicly on April 20

Read Time 10 minutes

The COVID-19 pandemic is constantly evolving, but where we stand today looks a lot different than where we stood a month ago. The good news is that it doesn’t look nearly as catastrophic as it seemed in mid-March. The numbers of new cases and new deaths seem to be plateauing and even declining (slightly) in hotspots such as New York City. So now we are at a fork in the road, as the diagram above suggests. Do we continue the “lockdowns” in hard-hit parts of the country, to halt the further spread of the disease? Or do we begin to open up parts of the population (and economy), and inch back towards something resembling “normal?”

To contemplate, let alone answer, this question really digs into a much deeper question about the current state of affairs and how we got here. Are we in the somewhat favorable state we are in today because of how well we’ve contained the virus, how well we’ve “flattened the curve?” Or are we in this state because the SARS-CoV-2 virus is less deadly than we initially thought?

If possible, let’s try to have this discussion with as little emotion as possible. Instead, we should think about it through the lens of what we know about logic, supposition, and probabilities.

Let’s start with the early predictions that many people, myself included, found beyond frightening, but also at least somewhat plausible. Those predictions were produced by epidemiological models that used as inputs various properties assumed to be known about the virus, most importantly how readily it spread between people and how harmful it was to those who acquired it. And while these models varied wildly in their predicted outcomes—from 200,000 to more than 2 million deaths in the United States—they all have one thing in common, which is that the way things stand now, they appear wrong.

What follows is my current thinking about COVID-19 and the all-important models upon which we are basing our decisions, along with some suggestions of what we need to do to begin to break this logjam.

As I pointed out a few weeks ago, the models that predicted that ~60% of Americans would be infected and ~1.6 million of us would die in the coming 18 months were based on assumptions about SARS-CoV-2 and COVID-19 for which we had little to no data—specifically, the exact value of R_0 (i.e., how many new people each virus carrier infects, on average), the percentage of infected patients who will require hospitalization, and the fraction of infected patients who will die (i.e., infection fatality rate, or IFR). For the most part, we only know the case fatality rate (CFR) of COVID-19—that is, the number of confirmed positive patients who end up dying of the disease. This number is less helpful, because patients with the most severe symptoms (and probable bad outcomes) are more likely to be tested. So by definition the CFR must overestimate the IFR. This is a very important point, and it comes up again, so let’s be sure it’s clear before we proceed. The CFR is the ratio of deaths to known cases; the IFR is the ratio of deaths to total cases, known and unknown. If, as in the case of ebola, these are very similar, then using CFR for IFR will not take you too far off the mark. But what happens if the IFR is one-tenth of the CFR? In other words, what if the total number of unconfirmed infected persons is an order of magnitude larger than the number of confirmed cases?

We’ll come back to this. Let’s get back to the models.

The sensitivity of the models to one variable in particular is especially pronounced. If you want to experience this firsthand, play with the model described in this New York Times article, and see how even the smallest changes in the virus’s reproductive number, R_0, altered the outcome in seismic ways. For example, using the default parameters in place, simply changing the R_0 from 2.3 to 2.4 triples the projected number of infected people from 10 million to 30 million. Think about that for a second. A seemingly negligible increase in the per-person rate of transmission leads to a 3x difference in total infections! (According to the model, anyway.) And what if you assume R_0 is a “mere” 2.1 (still a very contagious virus, by the way)? Fewer than 1 million Americans could expect to be infected. Tiny changes in inputs make the difference between a catastrophe and a minor speed bump. As someone who used to make a living building models—and as someone who has been humbled by them (albeit for mortgage defaults, not pandemics)—I can tell you that when you have a model that behaves this way, you need to be even more cautious than you otherwise would, and should, be with any model.

Projections only matter if you can hold conditions constant from the moment of your prediction, and even then, it’s not clear if projections and models matter much at all if they are not based on actual, real-world data. In the case of this pandemic, conditions have changed dramatically (e.g., aggressive social distancing), while our data inputs remain guesswork at best.

So, absent actual data, assumptions about these parameters were made—guesses, actually—but these assumptions lacked the uncertainty that we would expect from actual epidemiological data. What do I mean by lacking in uncertainty? Imagine that you are trying to estimate the number of acorns in your neighborhood by the end of next year. You build a model that factors in many variables, such as the number of oak trees, the weather, and so on, but in the end you realize the model is most sensitive to the number of squirrels in your neighborhood and how much their weight changes over the winter. You could guess at those parameters. Or you could spend time measuring them and using actual data as the inputs to the model. If you choose the former, you are merely entering a value (or values) for the respective parameters. That’s your best guess. But if you choose the latter, you are probably not using a single, accurate number—it’s quite a project to count squirrels with any accuracy. Instead, you must use a probability distribution for the input.

Why? Because you are accepting the inherent uncertainty of the situation: Actually trying to count each and every squirrel, which would require an enormous effort and likely some draconian tactics, would still not yield a completely accurate number. It might be better to just count the squirrels on, say, one block, and multiply by the number of blocks, and adjust for other factors, and come up with a likely range of squirrel population numbers. This is not a pure guess, but neither is it an exact number, because when it comes to squirrels, and viruses, it is almost impossible to know their actual prevalence with total certainty.

As I learned when I was modeling mortgage credit risk, an A-plus model accounts for this inherent uncertainty by allowing you to use ranges of numbers (or better yet, a probability distribution curve) as inputs, instead of just static values. Instead of assuming every person who originated a mortgage in a particular tranche of risk has $3,000 in cash reserve for a rainy day, you might assume a probability distribution of cash reserve (and therefore financial runway prior to defaulting) that was normally distributed (i.e., shaped like a bell curve) around $3,000 or if you were really slick you’d get actual data from the Treasury or a consumer database that would give an even more nuanced probability function.1In reality, such a number would not be normally distributed because it is bounded below by zero, but technically has no upper bound, so the distribution of cash reserve would be skewed—and all of this could be approximated or measured. Obviously, knowing how much cash a person has in reserve is a very important factor in determining how long they will pay their mortgage in the event of an economic shock. (And rest assured that the major banks are furiously adjusting their own models in this regard right at this very moment.)

Back to our squirrels. If we choose to do the work and use actual data to inform our model, rather than our best point estimate, the input would be accompanied by a confidence level, or a measure of how certain you are that the correct answer lies in your range. Again, an A-plus model would have the ability to process the “number of squirrels” as 5,634 to 8,251 with 95% confidence. (For a quick primer on what it means to be “95% confident” in your guess, please take a few minutes to do this exercise). A B-minus model (or worse) would take one single number in for the number of squirrels and, worse yet, it would assume you have 100% certainty in that number. When a B-minus model gives you an “answer,” it has no range. It communicates no uncertainty. You have no ability to assign confidence to it, statistical or otherwise.

Unfortunately, most of the models used to make COVID-19 projections were not built to incorporate uncertain data, nor were they capable of spitting out answers with varying degrees of uncertainty. And while I suspect the people building said models realized this shortcoming, the majority of the press is not really mathematically or scientifically literate enough to point this out in their reporting. The result was a false sense of certainty, based on the models. I should emphasize that the models were off target not because the people who made them are ignorant or incompetent, but because we had little to no viable data to put into the models to begin with. We didn’t have several months to painstakingly count the squirrels. We didn’t even have a method for counting them. The best we could do was make guesses about squirrels, which we had never seen before, based on our understanding of bunnies and mice.

So, what does the future look like from where we stand today, versus a month ago? Do we have the same dire view of the future? Or has it changed?

Mine has changed. Quite a bit, actually. Today I suspect American fatalities from COVID-19 will be more in line with a very bad, perhaps the worst, season of influenza (The last decade saw flu deaths in the U.S. range from 12,000 to 61,000, so you can imagine how much variability exists). This suggests COVID-19 will kill tens of thousands in the U.S. this year, but likely not hundreds of thousands, and definitely not millions, as previously predicted.

What accounts for my different outlook today? There are really only two first-order explanations for why I can say the early projections were incorrect:

  1. Either the models were wrong because they incorrectly assigned properties about the biology of the virus pertaining to its lethality and/or ability to spread, or
  2. The models were correct, but as a society we changed our behavior enough in response to the threat of the virus to alter the outcome. In other words, the models were correct in assuming R_0 was high (north of 2.25 and in some estimates as high as 3), but aggressive measures of social distancing reduced R_0 to <1, thereby stopping the spread of the virus, despite its lethal nature.

It is, of course, most likely to be a combination of these two conditions; call them Case I and Case II, respectively. They are not mutually exclusive, either. In fact the jugular question today is how much of each? Is it 90/10, 10/90, or 50/50? If the predictions were wrong because we misunderstood the biology of the virus (overstating its risk significantly)—that is, we’re in a mostly Case I scenario—then we may start the process of thoughtful reintegration. If the predictions were wrong because we understood the biology, modeled it correctly, and appropriately put into place extreme social distancing measures—that is, we’re mostly in a Case II scenario—then we need to continue strict social distancing until we have effective treatments. Otherwise we risk a resurgence of disease that could dwarf what we are currently experiencing.

I have thought very long and hard about how to differentiate between these two scenarios—Case I vs Case II—and in my opinion the most effective and expeditious way to do so is to determine the seroprevalence of asymptomatic people in the major cities in the U.S., starting with the epicenter, NYC. In other words, find out (via blood testing for antibodies) how many people were already infected that weren’t captured as “confirmed cases.” Ideally, we would be able to do that by testing every single person in the city (that is, counting all the squirrels). But because that is infeasible, we should test as large a cross-section of the asymptomatic NYC population as possible, and extrapolate from the results. Either way, we need to broadly test people with no symptoms, which is something we have not done so far in an area hit as hard as NYC.

These data are enormously important. If the asymptomatic prevalence in NYC is 5%, meaning 5% of asymptomatic persons in NYC have been infected, while 95% have not, it would imply the IFR for COVID-19 is approximately 2.4%. This is a deadly virus, approximately 25x more deadly than seasonal influenza in NYC.2Bear in mind that the widely reported CFR of 0.1% for influenza runs into the same problem we have with COVID-19: the IFR of seasonal influenza may be a fraction of its CFR. It would also imply that efforts to contain the spread have been effective and/or the R_0 of the virus (the reproduction number) is much lower than has been estimated (2.2 to 2.6).

Conversely, if the asymptomatic prevalence in NYC is, say, 30%, it would imply that the IFR for COVID-19 is approximately 0.4%. This is a far less deadly virus than previously suggested, although still approximately 4x more deadly than influenza in NYC.3And it’s not an apples-to-apples comparison because healthcare workers are immunized, albeit modestly, to influenza, as are many high risk people, while no one is immunized to SARS-CoV-2. When the dust settles, I suspect much of the spread of this virus will likely trace to nosocomial sources that would normally be less in the case of influenza. It also implies that the disease is far more widespread than previously suggested. If 30 percent of New Yorkers have been infected, then efforts to prevent its spread have not been very successful, but NYC is approaching herd immunity (see figure and table, below, which show the relationship between R_0 and herd immunity).

Figure. Measles basic reproduction number, herd immunity, and coverage. As R_0 increases, higher immunisation coverage is needed to achieve herd immunity. Blue zone indicates the R_0 estimate for measles of 12–18. In the context of COVID-19, notice the higher the R_0, the higher the threshold to reach herd immunity. Image credit: Guerra et al., 2017

§

Table. Estimated R_0 and herd immunity thresholds for different infectious diseases.

The sooner we know how the virus behaved in the most hard-hit city in the country (and likely the world), the sooner we can start making better decisions with at least some modicum of confidence, versus blind certainty in models that don’t have the humility to incorporate a margin of error or degree of uncertainty. And of course, the models should also be to take into account regional and demographic variation. It seems likely that in some areas we will need to remain cautious, while in others less so; with some people we will need to remain cautious, while in others less so. For example, the virus clearly seems to spread more rapidly (meaning, the R_0 is higher) in NYC than in, say, Utah. And clearly some people are much more susceptible to major illness and death than others.

Testing broadly, especially asymptomatic people, to better estimate the true fatality rate is an essential part of any strategy to move forward. Doing so, especially if we can add more elaborate tools for contact tracing, can give us real data on the most important properties of the virus: how rapidly it spreads and how harmful it is to all people, not just the ones we already know about. And that data, in turn, will help us build better and more accurate models.

But we shouldn’t look at models to give us the “answers.” How many people will be hospitalized, how many people will die, and so on. That’s our natural, lazy inclination. Instead we should look to the models to show us how to change the answers. That’s why they are important, and why it is so important that those models a) accept uncertainty, and b) are based on the best data we can obtain. The model is not a prophet. It is simply a tool to help us understand the biology of what is happening, and to help us figure out what we have to do next.

Go back in time to March 1: Knowing what we knew then, quarantine and extreme social distancing was absolutely the right thing to do because we didn’t even know what we didn’t know, and we needed to slow the clock down. It was like taking a timeout early in the first quarter after your opponent has just scored two lightning touchdowns in a rapid succession. It may have seemed unnecessary to some, but we needed to figure out what was going on.

The mistake was not taking the timeout. The mistake was not using our timeout to better understand our opponent. We failed to scale up testing and gather the essential information outlined here that would have helped us create better, more nuanced and hopefully more accurate models, rather than having to essentially guess at our data inputs (and hence at the outcomes). Now, six weeks later, we are still in the dark because we didn’t do the broad testing that we should have done back then. We still don’t know fully how many people contract this virus and come out relatively unscathed.

We still have time to reduce the health and economic damage done by this virus and our response to it, but we can’t waste another timeout sitting around looking at each other and guessing.



– Peter

Disclaimer: This blog is for general informational purposes only and does not constitute the practice of medicine, nursing or other professional health care services, including the giving of medical advice, and no doctor/patient relationship is formed. The use of information on this blog or materials linked from this blog is at the user's own risk. The content of this blog is not intended to be a substitute for professional medical advice, diagnosis, or treatment. Users should not disregard, or delay in obtaining, medical advice for any medical condition they may have, and should seek the assistance of their health care professionals for any such conditions.

83 Comments

  1. Thank you for your informative article. However, the severe lockdown that has been implemented never made much sense even if the IFR or Ro proves to be in the high range say 1%. The problem is that there was never, unless the virus is mild, an acceptable exit strategy to the lockdown. The main problem with the lockdown is the massive economic and resulting social damage it causes while only deferring, but probably not preventing, the health impacts. If the lockdown is continued, the economic damage will greatly outweigh the direct loss of life. If the lockdown is lifted, and the virus is in the bad range, then illnesses will just come back and we would be forced into periodic lockdowns until vaccines are available. This is not a viable approach considering the economic and social disruption. We were/are never going to be able to scale virus/antibody testing or implement draconian social policies ala China to contain this virus. Herd immunity is the only way this virus was/is going to be effectively mitigated in democratic countries. This was the case intially, based on what was known before the lockdown, and it is still the case today. This is why Sweden’s and Japan’s mild, relatively non-economically disruptive social distancing policies, were the only policies that ever made sense.

  2. Obviously numbers are down because we’re all home. What exactly has changed between then and now exactly? Not sure how letting up is the answer

    • Because it isn’t reality to be in a lock down bubble. Reality is moving freely in society (now that we know a few things) and letting people develop immunity (which has happened). The recent Stanford Study did just what this article suggested. The results are astonishing. Many many many tested positive for COVID 19. The mortality rate was adjusted to that of the flu for that county. First study that has tested everyone in a geographic area.
      Texas Lt. Gov. said that he would rather open the country and take the risk of death in order that the country moves ahead in prosperity for the future of his children and grandchildren.

      • Joan L Hand is spot-on. Destroying the world economy is not a solution! If this is a “war”, then we must expect casualties. From the moment we are born, we are dead men walking. I’m in the high-risk group and ONLY I am responsible for my safety. The virus will spread regardless of how locked-down we are; the longer we stay shut down, the longer the economic devastation persists – and the virus will STILL BE THERE. We can’t stay locked down for 18 months until a vaccine is available. Never forget – flattening the curve was never meant to reduce the total infections and deaths, only spread them out over a longer time scale.
        As more and more serology tests results come in, we will learn that the IFR is actually only slightly more than the seasonal flu.
        I fear that people who want the economy to stay crippled until the November elections are driving the shutdown.

  3. The first step is determine the underlying distribution. Lots of problems if the underlying distribution is a power law. Take the unknown data, bootstrap it by random drawing and replace. Then do statistical analysis on both. If they are wildly different then you can only use the bootstrapped data. In extreme rate problems I have found that ONLY the median is defined.

  4. Excellent analysis, thank you. I agree that in the US we lost a great opportunity to gather data during the lockdown. But other countries are doing it. Could we apply that information to the US? Since Germany (or Luxembourg) is testing samples of the population, can we assume that the same finding would be true in the US?

  5. The models were wrong. Models are always wrong, but these models were skewed so far to the extreme that they painted a dire picture that never conformed to reality. Those who were duped should admit their mistake, learn from their error, and correct their error immediately by abandoning the adopted flawed protocols. Some may claim that the mitigation efforts are the reason for the drastic variance between the theoretical model and the reality. While that hypothesis is possible there is no way to prove it scientifically. It is also just as likely that we would have had the same outcome if we had done nothing more than taking the normal societal precautions as we do during the seasonal flu. Social distancing is an overreaction that may have flattened the curve but not without consequences. It will likely elongate the curve as well. The models were wrong. Models are always wrong.

    • If social distancing is an overreaction and the curve will elongate. How come in China’s case the curve did not elongate and extreme social distancing did much better for them than the USA?

      • In China, a communist country, there was massive policing and enforcement so that distancing was obeyed. The police state in PRC is not how it is in North America. Here, infected people were out shopping. People who were to ‘self quarantine’ didn’t. The order to stay home was ignored. In China they fear being shot if they are caught being disobedient. Here we can get away with being out and if caught, face a 750 or 1000 dollar fine. I might add that the authorities here are totally inconsistent in their approach and direction.

    • But weren’t Italy and Spain reasonable blueprints for how the disease might spread w/o strict social distancing? Italy waited a very long time before they closed down their factories, and their hospitals were overwhelmed to the point where they had to choose who to try to save. It was only after they went to very extreme social distancing that they were able to bring the spread under control. Model or no model, we could look at how they proceeded and guess at what our future might be with similar behavior.

    • Indeed, “may” have flattened the curve.
      Sweden is the fly in the ointment of any argument proclaiming that lockdown helped to flatten the curve.
      Sweden did not impose lockdown yet their curve looks like, well, everybody else’s.

      Washing hands is the only proven action to reduce the chances of infection. Everything else is pure hypothesis.

  6. Thank you for the article! But…there’s another equation that tends to get ignored in solely probabilistic models; especially when those models are based upon exponential growth. That equation involves seriousness of outcomes.

    Simply calculating the probability that some will die is quite short sighted if not downright poor judgement. This is a serious result which is easy to write about, but how many here have experienced within their family?

    You, as an MD, and others in the health professions deal with death on a routine basis and have built mechanisms to help, though not alleviate, their effect on your psyche.

    But, as a retiree (with an engineering and statistics background), the seriousness of the data was, and is, still quite daunting. It’s daunting because the results are terminal at worst or potentially long term health effects due to treatments. Though, admittedly, the better case scenarios are good.

    Therefore, leaving out a seriousness part of the models, only looks at part of the data, and therefore, the probabilistic models are more wrong than you have allowed.

  7. It seems to me the CDC numbers have changed so often they are to be regarded as worthless.
    Questions:
    -what about herd immunity? The lockdown contradicts basic common sense if we need herd immunity.
    -why do we have to hide in our homes and lose our jobs, but we’re allowed to go to the post office, grocery store, hardware store?
    -where are all the 5 minute tests?
    -what to make of all the recent accounts of testing certain groups that are 85% to 100% asymptomatic? Two I can recall. First, in NYC, mothers giving birth were tested upon entering the hospital, and 85% of those testing positive were found to be asymptomatic. Second, in Boston, 450 homeless were tested. 150 tested positive and 100% were asymptomatic.
    -where’s all the hydroxychloroquine?
    -what to make of antibody testing that shows 25 to 55 times the probable cases exist. That would make this much less lethal than a common cold.

    I’m not a doctor or scientist, just someone who’s lucky enough to still be working. But we (the hoi poloi) have questions, lots of them. None of this adds up.
    If we don’t get this country back to normal, if that’s ever possible, I fear the second and third tier effects of this virus will be much much worse than the virus itself.

    • Unless the numbers are entirely cooked (which you might be arguing next, I don’t know), I think it’s safe to say the confirmed deaths above historical averages definitively render your proposition that it’s possible Covid-19 is “much less lethal than a common cold” utterly ludicrous.

      https://www.economist.com/graphic-detail/2020/04/16/tracking-covid-19-excess-deaths-across-countries

      Or perhaps tens of thousands of people are simply dying of boredom?

    • Just because someone presents as asymptomatic at a point in time, does not mean they won’t become symptomatic later. It takes 2 to 14 days for symptoms to show up in those that could become symptomatic. Then it takes a variable amount of symptomatic time before each person is either over it or dead which also has a range of a few days to over a month.
      In some, they remain contagious for up to 8 days after their symptoms have disappeared according to scientists and medical doctors in other countries.
      In S Korea they are finding that some who were over the initial infection and supposedly virus-free, have had the virus re-emerge, or they have become re-infected. Since there are currently 20 strains of this virus running around right now, and having had one strain doesn’t automatically make you immune to one of the other strains.
      I suggest looking at the history of the Spanish flu where the first wave wasn’t nearly as deadly as the second and third waves of it. History tends to repeat and letting up now would defeat the purpose of trying to eliminate this virus from the population. The big re-emergence in 1918 came in the fall when Philadelphia decided to have a parade(the same plan NYC mayor Deblasio is planning right now) The first wave killed less than 5 million people while the second wave killed up to an additional 30 million people.

      If we can’t slow this down our people on the front lines will continue to become infected and die from this. If too many do so, our hospital systems will collapse. In return, even more, will die due to a lack of supports to help some breathe while the virus is doing its best to kill them.

      As far as a lack of testing goes, that mishandling of the situation can be laid at the feet of one Donald J Trump, who even to this day is not doing what is needed to both order tests and require businesses to make the items needed and pay them for those efforts.

      His administration didn’t even put in an order for 30,000 ventilators until about the 20th of March. Far too late to help most states.

      Even now the states of Florida and Georgia are planning on opening up their economies after only 2 weeks of social distancing. This is taking place in their states even while their infection rates are continuing to surge. They will be a wide-scale experiment with the lives of their residents laid on the line. Since Florida is a huge retiree destination, this could potentially kill off many of their residents.

      I’m glad the governor in my state isn’t being so reckless with human lives.

      • Linda, we mean asymptomatic people who test positive for the antibodies, not for the active virus. Those people have already had COVID-19; they will not become infectious; they have survived the disease. They had it and never experienced symptoms. Three recent studies show that the numbers of people with COVID-19 have been underestimated by anywhere from 20 to 55 times. That means the IFR has been overestimated by the same factor.

      • First of all, the incubation and serial periods have also been coming down based on historical data, by roughly 50%. (Prior estimates assumed the numbers from the first SARS virus.) That puts the R0 value down to 1.3 or so. Secondly, the antibody testing (prior to Abbott’s tests) had some accuracy issues, but more often than not were false negatives. This is good news, yet so many are unreceptive to the findings. (Perhaps they fear having to admit an over-reaction, or have an emotional bond to the dire predictions, or know someone personally that died from this. It’s getting harder for people to trust objective data in such a polarizing atmosphere!)
        At any rate, the fog of war is lifting on the virus front, but it will take a long time to calculate the losses due to friendly-fire on our mental health, delays in “non-essential” screenings for other diseases, and indirect deaths from poverty, etc.

    • Aside from the virus behavior (serology, IFR/CFR, etc.), there’s the political side that should be examined. It’s so easy to lose track of what we knew and when. Maybe it should be said, “Hindsight is 20/40” in this case? I found this helpful: https://www.usatoday.com/in-depth/news/nation/2020/04/21/coronavirus-updates-how-covid-19-unfolded-u-s-timeline/2990956001/
      How many people would have supported a lock-down with less than 100 cases at the beginning of March? Yet the mantra of acting sooner is repeated ad nauseum. Yes, the CDC & FDA had bureaucratic procedural issues, reliability issues in early tests, and qualification hurdles (who should be tested). But I see many of those obstacles having been eliminated at a rapid pace (by D.C. standards). Generals should only be judged by the decisions they made with the resources and information they had at the time, right? Some fault leaders for trying to control panic with confident words when they should have been more realistic. So should the SOTU address included “Covid-19 will certainly kill 40,000 by the mid-April, and possibly hundreds of thousands if drastic measures aren’t taken?” Nobody wants their leadership to be such a defeatist, unless you’re their opposition. Did we have very little to worry about in February from a daily-living standpoint? Perhaps. The narrative changed preceding the bad news, though, so I always felt like I knew what was coming. The people that surprised me most are the ones that made proclamations one day and did the complete opposite the next, such as our Wisconsin governor. That inconsistency did not instill confidence whatsoever. It began to look like a game of one-up-manship from one state to the next. At this point, we’re at loggerheads because 50 of 72 counties have no new cases this week and those folks are feeling punished for the failures of the others. That kind of division is not helping anyone, but there’s no representation, debate, or compromise taking place either. I’m hoping the serology will show herd immunity closer than expected. We even have a difficult time discussing Sweden! Furthermore, even if we agree that some measure of precaution is warranted, on what basis do we decide what commerce is essential? Instead of controlling the economy at the supply side, why not enact common sense procedural changes at those places of business and leave it up to the patrons to decide? For example, exercise gyms and pools have always been a source of exposure to other contagions, so protocols were already in place to limit the spread of such things and the health benefits are perceived to outweigh the risks. Why should they be shuttered? Why should the guidelines encourage outdoor activity while closing lakes and parks from public access? The duplicity is frustrating to say the least. I’m all for managing the curve, but we’re now stuck in an analysis paralysis in terms of public policy! (And certainly the shouting match and name calling aren’t helping – Not speaking for anyone here specifically.)

  8. Day 36 of total lockdown in the French Alps – it helps to be in the alps.
    Thank you for your thought-provoking article of which I think I understood about 30%, having been blessed with no formal education. I left school @ 15 – and then went on to be an adman, a professional that requires absolutely no education, talent or morality.
    I do think our leaders over-reacted, and we would have benefited enormously from some form of testing. The French claim they have not tested because they don’t believe in the efficacy – the reality is: they don’t have any tests.
    As a 73 year old heart patient – a double-whammy of vulnerabilities – who is still breathing, I have to thank the French Government for the steps it took, but when George Bush II claimed he has kept America safe since 9/11 how do you prove that what they did is the cause of our safety?
    Are we safe because Macron locked us up, or would we have been just as safe if we would have been allowed to carry on playing mountain rugby?
    On May 11th – 56 TOTAL DAYS – the situation will be relaxed a little in France, and fortunately due to the age of Mrs Macron he can’t keep as old buggers locked up much longer that the rest of the population.
    The Swiss are relaxing earlier – difficult for the Swiss – and they will start by reopening hairdressers, which as a bald man I find discriminatory, and then tattoo parlours and massage parlours – three artisanal skills I would have thought were difficult to practise at arms length?
    So on May 11th do we trust the politicians who locked us up erroneously – and are opening tattoos parlours before churches – to tell us it is safe to go back in the water – cue JAWS theme song.

    • I wish I knew you! You gave me many a chuckle!

      Praying for you right now. That you will continue to be (relatively) healthy and that many more humans on this big earth will have the opportunity to enjoy your wit!

      • As a Scot, I.E. a man who wears a skirt, I consider humour essential to sanity – not that I am claiming to be sane.
        I might have been before the launch of Coronervirus (sic) but 37 DAYS in confinement with a Belgian woman who spells whisky with an “e” and I fear the wurst (sick).
        As you know all forms of pleasure are banned in France for at least another 19 DAYS – bringing the total to 56 DAYS for those of us who made it to the end still breathing.

        Biking for pleasure IS BANNED!

        Hiking for pleasure IS BANNED!

        Striking – the French national sport – for pleasure IS BANNED!

        SEX IS BANNED – but I can’t blame the French for that because that is a Belgian law!

        But you can “use your bike” for essential outings such as going to adopt an abandoned pet so that thousands of animals in France can be saved from the guillotine.

        Members of the public are permitted to leave their shelter if the purpose of their journey is to pick up a pet from a local pet shelter.

        There are certain conditions:

        The animal must have been chosen and confirmed online ahead of time and must be called Chris.

        The adopters must not have more than three convictions for bestiality: over three and the only animal you can adopt is an aardvark .

        The adopters must come to the refuge dressed as circus performer – preferably clowns – and both future “pet-parents” not just one – must arrive with a special attestation signed in triplicate and bearing the official wax seal of the French government before they can adopt a seal.

        By ticking the box reading “imperative family reason” (“motif familial impérieux”) the adopters agree to be filmed with the animal during all times of the day and night.

        As I said I am not claiming to be sane….

    • Just a thought. Maybe they are opening those first because those folks are considered to be less desirable members of society, and therefore they will see if there is a bad outcome with them before expanding it to the rest of the populace?

      I guess we will find out eventually.

  9. I could be wrong, but I think the author said this:

    “But what happens if the IFR is one-tenth of the CFR?”

    and meant this:

    “But what happens if the CFR is one-tenth of the IFR?”

  10. I think one glaring omission of the models is the apparent lack of a baseline (perhaps the media perpetuates this). Since it has become apparent that casualties are overwhelmingly among the aged, infirmed, and immunocompromised, how many of these deaths would have taken place anyway in the baseline scenario without COVID-19 due to some other infection or natural disease progression? It’s sensationalism for the media to constantly being reporting that X number of people died after testing positive (or in NYC presumed positive) when clearly some significant percentage would have died anyway.

    I would think very differently if those dying of COVID-19 were sometimes in relatively good health. However, it is suspicious when the media jumps up and reports on any single case of a young person dying only for us to find out that person was morbidly obese, had cancer, diabetes, HIV, etc. In terms of policy, it seems quarantine should be for the aged and infirmed instead of a blanket lockdown for that half of the workforce that “society doesn’t need”.

  11. No denominator, No numerator , models are all wrong one way too high. Keeping people inside is only delaying what could happen. 1/2 of deaths in NY. If the regular flu was advertised like this no one would go out. Where are the deaths now of the flu and pneumonia? Nursing homes and compromised people should be isolated.

  12. The author implies we should have done massive testing in the 1st half of March. Was that possible?

    The US has now tested about 1.25% of the total population after a massive effort by state and private sector actors. However, most Western European nations have tested a higher % of their population with mixed results. Germany has a death rate < half of US; has tested ~80% more people. OTOH Spain has tested ~60% more people but has a death rate about 3 times the US.

    Do we know how effective all these tests are?

  13. I’d bet a lot of $$ that the % of NY’ers with antibodies is low, much much lower than 30%, indicating that it was social distancing that made the models “wrong” (in which case the models were actually right in predicting what spread will look like if we end, or had never started, social distancing). Of course your sampling approach is the right way to answer the question but the reason I would be surprised is that otherwise it indicates that social distancing actually had *no or little effect* on the actual spread which would be very very odd. Just look at cases of influenza this year, they are way way down and will probably stay down for *years* before recovering – because of social distancing. How on earth would Covid have magically spread while influenza couldn’t?

    There is one possibility which is that it was spreading in the US long before we realized it which has been proposed but again, I’d be surprised. The 100% telltale residue just isn’t there – people weren’t turning up dead from a mysterious disease in Feb. So why would people suddenly start doing so in March?

    Honestly I think a lot of the angry “models were wrong”/conspiracy posts here are way off base. However, someone made a point on that side I 100% agree with: what is the exit strategy? Social isolation for 12 mo will mean our economy will be utterly destroyed, forget about “worse than the great depression”, it will be much much worse than that.

    • I agree with Steve S. In addition to the timing of the onset of hospitalizations and deaths, the trends were extremely exponential prior to shutting down. Log plots were as straight as arrows. One signature of approaching herd immunity is an inflection in those daily trend curves. That inflection did happen, of course, but only after the shutdown. When the “R0” falls below 1, then herd immunity takes hold with anything over 0% fraction of the population infected. After shutdown, the data will begin to look like herd immunity has taken over. This is the expected result. That’s why we did it. With mitigation in place, death estimates were 100K. The 2M estimate was without mitigation We mitigated. Epidiemilogosits quickly adjusted the 100K down to 60K based on incoming data. The 100K to 60K adjustment given the nature of exponential growth and all those uncertainties you mentioned is not that far off. We are already over 47K deaths on 4/22/20. I think it will be over 60K when all is said and done. Maybe 70-80K which would be even closer to those initial 100K estimates. But overall this is not a bad job of prediction by our epidemiologists.

      Yes, these are flu-like numbers, I agree. But, this is only flu-like because we have been locked in our houses for the past 5-6 weeks. Look at NYC, Bergamo, Wuhan where things slipped away just a little–they all locked down, too…but, just a tad late. Right now there are more people dying per day in NYC than all other causes of death combined. ALL OTHER causes. I’m sure the same was true in Bergamo and Wuhan. The flu NEVER has done that in our lifetime. Three cities with the same results all paying a heavy price just for being a tad late in their shutdown. How many times do we have to hit our head against the same wall before we grok what this disease does when it’s let loose in the wild? The sheer number of people dying every hour says this is in a different league than anything we have ever seen in our lifetime.

      Yes, there is huge uncertainty in the denominator. It will be years before we really know what the final denominator is. We don’t have years to figure out the denominator. But the number of people dying per hour in NYC has already effectively put the denominator into the equation. Whatever the denominator is, the number of people dying per hour is way too high.

      30-50% of the population being infected already is wonderful to hope for (we all hope for it), but we should not use hope to make public health decisions. We have to go with a reasonable worst-case scenario in making public health decisions. With steel buildings, we make sure any loading stays away from the yield strength limit of the steel, even though we have documented data that says we can approach much closer to that limit and probably be OK. When it comes to public safety, we don’t play with best-case scenarios even when we know the numbers. Why would we ever want to play with best-case numbers when numbers are so unknown?

      I know the answer to that question is the economy. But, where is the all the equivalent detailed analysis of the supposed long term economic fallout? I don’t expect that analysis from you, Peter.. but where is it from anybody? Yes, of course, it’s intuitively obvious all this is not good for the economy, and the longer the lockdown last the worse it will be. But, where is the quantitative analysis of the number of shutdown weeks vs economic damage? How long can we be in this partial shutdown mode? And we are in a partial shutdown. Supply chains remain intact. I can buy almost anything I want, except toilet paper and hand sanitizer. How long can we operate in this mode before we cause irreversible damage to the economy? Where is that analysis? Why are the epidemiologists scrutinized so heavily for uncertainties in their work, based on the mere concept of “the cure can’t be worse than the disease” We are analyzing the disease, but who is analyzing the cure? Exactly how and when does the elasticity of the economy break? In the language of steel when does the economy hit its yield strength limit?

      If we back off what our expert epidemiologists tell us we should be doing, then I want to back off based on equivalently well thought out economic projections that show we are just as damned or more damned than the epidemiological projections. I don’t want to back off from expert epidemiological recommendations based on the pressure of Fox News talking points.

      • Someone earlier described the “we need to open things up because the economy will be destroyed” vs “we need to stay locked down or the outcome will be Bergamo” is like two parties playing tug-of-war on two different ropes. Both are decent points but neither is really connected to the opposing concerns and both are just playing tug-of-war, pulling for their team but not constructively looking at real solutions.

        Bottom line, we can’t juts open things up, that would clearly be disastrous (I have to disagree with Peter’s uncertainty on this question) and if we just keep sheltering in place the economy and the economic welfare of millions (billions?) will be damaged to the point where many lives will be lost either way. But the idea that we have to choose between those extremes is a false, polarized dichotomy all too common in US society and politics. We need aggressive, well coordinated, intelligent ongoing mitigation plan that *also* allows us to get back to life. There are options. Sadly our government hasn’t shown a great history of being able to work in an “aggressive, coordinated, intelligent” way, heh, not by a long shot. So maybe we are screwed long term one way or the other.

  14. I find it difficult to argue with your notion that the 2 week shutdown (and Trump’s early ban on Chinese travel) bought time which was then squandered. In looking at how it was squandered, two factors stand out. First, the utter incompetence of the FDA/CDC. The FDA gave the CDC a monopoly on the test kits, going so far as to shut down an alternative developed at the University of Washington. The CDC then didn’t ramp up production and what production of test kits they did was contaminated. This crippled the early testing process. The other factor was the outsourcing of PPE, medicine, and medical technology to China. Both of these have roots that go back far beyond the current crisis.

  15. Peter you need to get Michael Osterholm back on your podcast. This is NOT the flu. This is many times more deadly than the flu. Here’s a link to Osterholm’s appearance this morning on Morning Joe, which you really need to watch: https://www.msnbc.com/morning-joe/watch/dr-michael-osterholm-says-high-amount-of-transmission-to-come-82441285976

    The numbers out of NY this morning were illuminating. About 14% of all New Yorkers appear to have the antibodies. Only about 1.4% of all New Yorkers have been confirmed to have the virus according to the latest data. So the number of actual cases is about 10 times the confirmed cases.

    Based on the latest NY confirmed case count, the death rate is 7.74%. Divide this by 10 to account for the correct denominator and you get 0.774%. This would mean Covid-19 is around 7 times deadlier than the flu (the flu death rate is about 0.1%). But I’m not even sure this is comparing apples to apples. I’ve heard that the flu number does NOT include asymptomatic cases in the denominator. Flu estimates each year are only for ill people according to one expert I’ve heard.

    It’s very likely that this will turn out to be roughly 5-10 times deadlier than the flu. Just as Dr. Osterholm predicted on your podcast I believe. Given the over-run hospitals and piled up body bags, all while much of the world was under lockdown and in only a couple months mind you, this factor of 5-10 is certainly in the ballpark. The level of carnage does not make sense under even a bad flu season.

    Love the blog. Keep up the good work.

  16. I don`t have the time to evaluate based on “flawed models”.What has been reported by local media here in Arizona is that there are enough ICU`s,ventilators and hospital rooms if a surge occurs.So,come on and visit there is room for you.Hotel and golf course rates are at the lowest.

  17. Anyone in favor of “opening up” our economy ought to read/listen to any first-hand account by a medical professional who has been working in a NYC hospital during the past month. It seems most like being on the front line in a military combat zone, with no breaks in battle, without adequate ammunition, reconnaissance, back-up, or air cover.
    It seems irresponsible to suggest we invite more waves of infection until and unless our medical facilities are properly equipped to deal with them and their personnel adequately protected so they can do their jobs.
    This country cannot have a functioning economy without a functioning health care system.

Facebook icon Twitter icon Instagram icon Pinterest icon Google+ icon YouTube icon LinkedIn icon Contact icon