Seemingly Popular Covid-19 Model is Obvious Nonsense

Previous Covid-19 thoughts: On R0, Taking Initial Viral Load Seriously

Epistemic Status: Something Is Wrong On The Internet. Which should almost always be ignored even when you are an expert, and I am nothing of the kind. However, this was a necessary exception. My expectation that I would regret writing this has proven incorrect.

People are taking the projection of 60,000 American deaths from Covid-19 as if it were a real prediction. This number is being used to make policy, to deny states medical equipment and to make plans that spend trillions of dollars and when to plan to reopen entire economies.

Ignoring this in the hopes it will go away does not seem reasonable.

My suspicions that this was necessary were more than confirmed when, failing to realize just how obvious the nonsense in question was and thinking I needed to justify labeling it nonsense, I wrote a reference post called The One Mistake Rule. 

The second comment on that post was to argue that we should indeed use exactly the model that motivated me to write the post. The comment is here in full:

>> If a model gives a definitely wrong answer anywhere, it is useless everywhere.

Except if it needs to be used right now to make important decisions and it’s the best model we have. See: https://covid19.healthdata.org/united-states-of-america

We could plausibly think this is the best model we have? Oh my are we screwed.

The Baseline Scenario That Makes No Sense

There seems to be a developing consensus on many fronts, for now, that the model linked above represents our reality. The model says it is ‘designed to be a planning tool’ and that is exactly what is happening here. 

What is this model doing? Time to look at the pdf. 

Here’s the money quote that describes the core of what they are actually doing.

A covariate of days with expected exponential growth in the cumulative death rate was created using information on the number of days after the death rate exceeded 0.31 per million to the day when 4 different social distancing measures were mandated by local and national government:

School closures, non-essential business closures including bars and restaurants, stay-at-home recommendations, and travel restrictions including public transport closures. Days with 1 measure were counted as 0.67 equivalents, days with 2 measures as 0.334 equivalents and with 3 or 4 measures as 0. For states that have not yet implemented all of the closure measures, we assumed that the remaining measures will be put in place within 1 week. This lag between reaching a threshold death rate and implementing more aggressive social distancing was combined with the observed period of exponential growth in the cumulative death rate seen in Wuhan after Level 4 social distancing was implemented, adjusted for the median time from incidence to death. For ease of interpretation of statistical coefficients, this covariate was normalized so the value for Wuhan was 1.

In other words, this model assumes that social distancing measures work really, really well. Absurdly well. All you have to do to stop Covid-19 is any three of: Close schools, close non-essential businesses, tell people to stay at home, impose travel restrictions.

If you do that and maintain it, people stop dying. Entirely.

Look at the graph they have up as of this writing (updated on 4/10). By June 20, they predict actual zero deaths that day and every future day. They have us under 100 deaths per day by the end of May.

The peak in hospital use? Today, April 11.

The peak in deaths? Yesterday, April 10. For New York, several days ago, with our last death on May 20. 

In other words, considering the delay in deaths is about three weeks, they predict that no one in New York State will be infected after April. No one! We’ll all be safe in only three weeks! 

This is despite us not yet seeing any evidence of a major decline in positive test rates in New York. Deaths lag positive tests by weeks.

Hard to be more maximally optimistic than that. One could call this the ‘theoretical beyond best case scenario.’ 

(The statement is actually even more absurd than that, considering variation in time to case progression, but I’m going to let that one go.)

(Exercise for the reader, you have five seconds: What is the implied R0?)

(Second exercise for the reader: If there are four things that reduce the spread of infection some amount, and R0 is about 4 initially, and you implement three of them, what is the new R0?) 

They Account for Uncertainty, Right?

They generously account for uncertainty with the following ‘confidence interval’:

Figure 9 shows the expected cumulative death numbers with 95% uncertainty intervals. The average forecast suggests 81,114 deaths, but the range is large, from 38,242 to 162,106 deaths.

(Note: this was as of paper publishing, numbers are now lower.)

That is not how this works. That is not how any of this works.

The way this works once we correct for all the obvious absurdities is that this is a lower bound on how good things could possibly go.

If I am incorrect, and that is how any of this works I have some very, very large bets I would like to place.

A Simpler Version of the Same Model

The model seems functionally the same as this:

Assume all reported numbers are accurate, and assume that no one gets infected once you nominally implement three of the four social distancing measures. Which you assume every US state will do within a week from the model starting.

Let’s simplify that again. 

Assume that no one under an even half-serious (three quarters serious?) lock down ever gets infected out-of-household.

We still see deaths for a few weeks, because there is a lag, but then it’s all over.

What the Model Outputs

As of when I wrote this line, this more-than-maximally-optimistic model projects 61,545 deaths in the United States.

People with power, people with influence, what some might call our “best people,” are on television and in the media predicting around 60,000 total American deaths.

I will say that again.

We are telling the public a death count that effectively implies that by about a month from now, and in many places earlier than that, no new American ever gets infected with Covid-19.

The model assumes that our half measures towards social distancing will have the same impact as was reported in Wuhan. In Wuhan, they blockaded apartment buildings, took anyone suspected of being positive away for isolation, and still, months after this model says there are no infections or even deaths, have severe movement restrictions and blockades up all over the place.

Whereas the New York City subways continue to run, and California thinks weed sales are an essential business.

I hope that my perception of this is wrong. Perhaps everyone knows this model is nonsense. Perhaps there are better ones out there – if you know of one you respect, please let me know about it!

But again, this is a maximally optimistic model on every front. I keep seeing people whose voice matters share this same final answer of predicting 60,000 deaths. If it’s not from a model doing more or less this, I don’t know how you get an answer in that ballpark.

Unless of course answers are being chosen without regard to reality.