A pdf copy of this article is available here.
There has been much media coverage about the danger to life posed by the COVID-19 coronavirus pandemic. While it is clearly a serious threat, one should consider whether the best evidence supports the current degree of panic and hence government policy. Much of the concern resulted from a non-peer reviewed study published by the COVID-19 Response Team from Imperial College (Ferguson et al 2020[i]). In this article, I examine whether data from the Diamond Princess cruise ship – arguably the most useful data set available – support the fatality rate assumptions underlying the Imperial study. I find that it does not do so. The likely fatality rates for age groups from 60 upwards, which account for the vast bulk of projected deaths, appear to be much lower than those in the Ferguson et al. study.
Metrics for COVID-19’s fatality rate and their estimation
The fatality rate from infection (IFR), by age group, is a key parameter in determining how serious a threat the COVID-19 pandemic represents. Unfortunately, the IFR is difficult to determine. It is more practical to estimate the fatality rate for cases where the COVID-19 virus can be shown, by a standard test, to be present, whether or not there are any symptoms. This is referred to as the true case fatality rate (tCFR). The tCFR will overestimate the IFR, since a proportion of people who actually have been infected may show no viral presence when tested, either because they have already fought off and cleared an infection without any noticeable symptoms, or perhaps because they have pre-existing immunity. Nevertheless, where testing has been applied to a sample of people without regard to whether they show symptoms, the tCFR may provide a reasonable, albeit somewhat biased high, estimate of the IFR.
However, determining tCFR is not simple either, since in most cases infected people with no or mild symptoms will not be tested for COVID-19. Attempts have nevertheless been made to estimate tCFR by adjusting estimates of the CFR based on symptomatic cases only (sCFR), by adjusting for the non-random nature of testing, and also for the outcome of positive test result cases not being known for some time.
The Imperial studies
The Ferguson et al. study used estimates of the IFR[ii] from another paper from the same team, Verity et al. (2020)[iii], which had been published a few days earlier on 13 March. Very helpfully, Verity et al., unlike Ferguson et al., published the computer code and data that they used.
The Verity et al. CFR estimates were derived primarily from Chinese data, which reflected non-random testing. The authors obtained age-stratified IFR estimates (in reality, tCFR estimates) by adjusting their CFR estimates using infection prevalence data for expatriates evacuated from Wuhan, all of whom were tested for COVID-19 infection. This approach involves very large uncertainties.
An alternative approach to estimating the tCFR, as a proxy for the IFR, is to use data from a large sample of people, all of whom were tested for the presence of the virus without regard to whether they showed any symptoms, with all who tested positive subsequently being isolated and the case outcome recorded. I use that approach. While the sample of expatriates evacuated from Wuhan is too small for this purpose,[iv] occupants of the Diamond Princess cruise ship do provide a suitable such sample.[v] Moreover, the Diamond Princess sample has the advantage that it consists mainly of people from high income countries, and those requiring hospitalisation were treated in such countries.
The Diamond Princess sample may well represent the best available evidence regarding tCFR for older age groups, who are most at risk. Verity et al (2020) did analyse data from the Diamond Princess, but did not use sCFR or tCFR estimates from them for their main CFR and IFR estimates.[vi]
The Diamond Princess death toll
When Verity et al. was prepared, the final death toll was not known. The data available only ran to 5 March 2020, at which point 7 passengers had died. The authors therefore used a fitted probability distribution for the delay from testing positive to dying to estimate that those deaths would represent 56% of the eventual death toll. They accordingly therefore estimated the tCFR using a scaled figure of 12.5 deaths.
Here, I adopt the same death rate model and use the same data set, but brought up to date. By 26 March the number of deaths had increased from 7 to 10. Of those 10 deaths, 3 are reported to have been in their 70s and 4 in their 80s. I allocate the remaining, unknown age, persons pro rata between those two age groups. As at 25 March the Verity et al. model estimates that 96–98% of the eventual deaths should have occurred. I scale up to 100% using the 96% estimate, giving an estimated ultimate death toll of 10.43, allocated as to 4.47 to the 70-79 age group and 5.96 to the 80+ age group.
Accordingly, the Verity et al central estimate for the Diamond Princess death toll, of 12.5 eventual deaths, appears to be 20% too high. This necessarily means that the estimates of tCFR and sCFR they derived from it are too high by the same proportion.
Numbers testing positive
The Diamond Princess dataset was published by the Japan National Institute of Infectious Diseases (NIID). I use the second version published on 21 February[vii], which gives detailed data for 619 confirmed cases, updating it for subsequent test results.[viii] Verity used the original 19 February version of NIID, which gave data for 531 confirmed cases, although they did update it for subsequent test results.
The entire set of passengers and crew, totalling 3711 individuals, was tested for COVID-19. Some 712 (19.2%) ultimately had positive test results, of whom (based on the NIID data for 619 of them) 51% were asymptomatic. The infection rate varied between 10.1% for ages under 30 years to 24.7% for ages 60+ years. The age-distribution was only known for cases included in the NIID data. Verity et al. assumed that the age distribution for the overall total of confirmed cases was the same as for the 531 NIID reported cases that they used. I do the same, but using the later NIID data, with 619 reported cases. On that basis, 203.6, 269.2 and 62.1 people in respectively the 60–69, 70–79 and 80+ key age groups had positive test results.
Recall that tCFR is the eventual death toll divided by the total numbers testing positive.
My overall tCFR central estimates from the Diamond Princess 70+ age groups, where all the deaths are taken to have occurred, are 3.15% overall (10.43/331.3),[ix] with a breakdown of 1.66% for ages 70-79 (4.47/269.2) and 9.59% (5.96/62.1) for ages 80+. For the 60–69 age group, there are sufficient test-positive occupants to make a crude median estimate of the tCFR, by calculating what it would need to be for there to be a 50% probability that no 60-69 year-old has died, as appears to have been the case. The thus-implied tCFR is 0.35%. There were too few Diamond Princess occupants in age groups below 60 with positive test results to provide any useful information about the COVID-19 tCFR for those groups.
Adjustments for false negatives and underlying death rates
It appears that in about 30% of symptomatic cases the standard RT-PCR test for COVID-19 infection gives a negative result when the patient is in fact infected.[x] There is no evidence of any COVID-19 related deaths among Diamond Princess occupants who tested negative, which would be consistent with a lower viral load being associated with a lower probability both of a positive RT-PCR test result and of eventual death. The false-negative rate may be slightly lower for Diamond Princess occupants, a few of whom may have been retested or tested by a more reliable method where they had typical COVID-19 symptoms but an initially negative RT-PCR test result. However, it seems likely that the proportion of asymptomatic infected cases that are not detected by a RT-PCR test will be somewhat higher than the 30% estimated for symptomatic cases. We accordingly adjust all the tCFR ratios estimated from Diamond Princess case data down by 30% on account of false-negative test results.
The observed deaths of Diamond Princess occupants occurred over a 45 day period, during which a non-negligible percentage of old people would be expected to die from non-COVID-19 related causes. I have accordingly deducted from the adjusted tCFR ratios an allowance for non-COVID-19 deaths for 70+ age groups, based on UK age-stratified 2018 death rates,[xi] to arrive at estimates of deaths caused by COVID-19. There are arguments for the non-COVID death rates being either higher or lower than those for the UK population of the same age, but using those death statistics appears to be a reasonable first approximation.
Comparing the Ferguson et al. UK and Diamond Princess based fatality rate estimates
The results of the foregoing analysis are set out in Table 1. The key finding is that the estimated tCFRs for Diamond Princess 60+ age groups, which must if anything overestimate their IFRs, are far lower than the corresponding IFR estimates used by Ferguson et al. in the study adopted by the UK government.[xii] Those age groups account for the vast bulk of projected deaths. For people aged 60–69, the Ferguson et al IFR estimate is 18.1 times as high as the best tCFR estimate based on Diamond Princess data, for the 70–79 age group it is 6.1 times as high, and for the 80+ age group it is 1.65 times as high.[xiii]
Table 1: True Case Fatality Rates estimated from the latest Diamond Princess data compared with Infection Fatality Rates per Ferguson et al. 2020, used by the UK government
Note: An all-causes tCFR of 0.35% (and hence 0.72 notional ultimate fatalities) is assumed for age-group 60-69 despite there being no actual fatalities in that age group (see text). Expected non-COVID-19 fatalities are based on UK 2018 death rates by age group applied to the DP positive test cases, scaled by the 45 day period over which COVID-19 deaths were recorded and divided by the same 0.96 factor used to scale up the 10 actual deaths. DP= Diamond Princess.
Based on the Diamond Princess data, the COVID-19 fatality rates by age-group assumed by Ferguson et al. appear to be far too pessimistic for all 60+ age groups, where the vast bulk of fatalities are projected to occur. It is quite possible that they are also too pessimistic for younger age groups as well, but unfortunately the Diamond Princess data are uninformative about death rates below age 60.
It is notable that for all the 60+ age groups the projected excess death rates, based on Diamond Princess case data, caused by COVID-19 is substantially lower than the underlying non-COVID-19 annual death rate. Even assuming, very pessimistically, that there is no overlap between the two, and that the same proportion of each age group becomes infected, projected COVID-19 related deaths from an epidemic in which the vast bulk of the population became infected with COVID-19 are only 9% of expected annual non-COVID deaths for the 60–69 age group.[xiv] For the 70–79 age group, the proportion is 20%, and for the 80+ age group it is 26%. Relative to the expected non-COVID deaths over two years, the approximate period during which very onerous restrictions are projected to be in force in the UK, these COVID-19 excess death proportions would each be reduced by almost half. In practice, a high proportion of people killed by COVID-19 will have serious underlying health conditions, and would be much more likely than average to die from non-COVID-19 causes.
Nicholas Lewis 27 March 2020
Note: this is an updated version of the original published on 25 March 2020, and takes into account further information on deaths, and positive test results, that were reported (in Japanese) on 24 March 2020 by the Japanese Ministry of Health, Labor and Welfare,[xv] which had not been found in previous searches.
i] Neil M Ferguson et al., Impact of non-pharmaceutical interventions (NPIs) to reduce COVID-19 mortality and healthcare demand, Imperial College COVID-19 Response Team Report 9, 16 March 2020, https://spiral.imperial.ac.uk:8443/handle/10044/1/77482
[ii] Ferguson et al. adjusted the Verity et al. IFR estimates “to account for a non-uniform attack rate”, without giving further information about the assumed attack rates. They appear to have increased the Verity et al. IFR estimates for all 60+ age groups by approximately 19%, while making little or no changes to those for younger age groups. It is unclear whether doing so was justified.
[iii] Verity R, Okell LC, Dorigatti I, et al. Estimates of the severity of COVID-19 disease. medRxiv 13 March 2020; https://www.medrxiv.org/content/10.1101/2020.03.09.20033357v1.
[iv] Their sample of evacuated expatriates is 689 people, of whom on 6 tested positive for COVID-19, none of whom died.
[v]Russell et al., “Estimating the infection and case fatality ratio for COVID-19 using age-adjusted data from the outbreak on the Diamond Princess cruise ship”, medRxiv preprint dated 9 March 2020, did use exclusively Diamond Princess data. However, the early data that they used was incomplete and their IFR (actually tCFR) estimates appear to be based on assuming 26 eventual deaths, and hence are far too high.
[vi] Verity et al. simply noted that that figures derived from the Diamond Princess data set were “consistent” with their main estimates, meaning that they fell within their very wide main estimate uncertainty ranges.
[vii] Field Briefing: Diamond Princess COVID-19 Cases, 20 Feb Update. National Institute of Infectious Diseases, Japan https://www.niid.go.jp/niid/en/2019-ncov-e/9417-covid-dp-fe-02.html
[viii] Using the same sources as given in Verity et al., for dates from 20 February on. Doing so yields a total of 704 positive test results, which I adjust to equal the cumulative total of 706 results stated in the final, 2 March 2020, update.
[ix] A simple estimate based on the binomial distribution suggests that the 97.5% upper uncertainty bound is approximately double this figure.
[xi] https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/datasets/deathregistrationssummarytablesenglandandwalesdeathsbysingleyearofagetables and https://www.ons.gov.uk/file?uri=%2fpeoplepopulationandcommunity%2fpopulationandmigration%2fpopulationestimates%2fdatasets%2fpopulationestimatesforukenglandandwalesscotlandandnorthernireland%2fmid20182019laboundaries/ukmidyearestimates20182018ladcodes.xls
[xii] Moreover, I am unable to reproduce the Ferguson et al. estimate of 510,000 deaths in the ‘Do nothing’ case, based on their estimate of 81% of the population being infected and using their IFRs. Note that the Ferguson et al. IFR estimates assume that, as was the case for the infected Diamond Princess occupants, health systems have not been overwhelmed by COVID-19 cases.
[xiii] Part of the difference is due to Ferguson et al. adjusting up the Verity et al. IFR estimate to reflect assumed non-uniform attack rates (see endnote 2).
[xiv] Based on the Ferguson et al. assumption that 81% of the population eventually becomes infected.
There’s another modelling study from Oxford University that says that COVID-19 probably arrived in the U.K. in January long before anyone realised and 50% of the population now have it or have had it and are now immune, therefore we are almost at 60% “herd immunity” where an epidemic can’t continue to spread. (This story was published in the Financial Times 25 March and is still on their website but is paywalled so I can’t provide a link). If this is true then the draconian shutdown is unescesary as the disease will fizzle out pretty quickly what ever happens.
The obvious point, and the obvious parallel with climate change is that if you’re going to make major decisions based on computer models you need to make damn sure the models you’re using are correct.
Unfortunately, in the absence of random testing of complete samples from the population for antibodies to COVID-19 it is impossible to tell whether the Oxford study is correct, or whether (as the Ferguson et al study assumes) a far lower proportion of the population (<20%) is either immune or has already been infected by COVID-19. The UK government is flying almost blind due to their failure to organise random testing, although at present the available tests would only reveal the proportion of people who have some viral load at the time of testing. I am somewhat doubtful that anything like 50% of the UK population have yet been exposed to the COVID-19 virus, however there is some evidence tentatively suggesting that a significant proportion of people may be immune to it, presumably through having been exposed to other similar viruses earlier in their lives.
I have been researching data, and have many data points to share. please contact me with your email address so we can further discuss.
Thanks; will do.
People aged 60+ who take a cruise are unlikely to be a random or representative sample of people aged 60+ in the general; population. They will definitely have above average wealth and probably have above average health. Really, ill people won’t travel, but they are the ones most likely to succumb to the virus.
That is true. On the other hand they are unlikely IMO to be the most active and fit people of their age, and they are probably more likely be overeating during the cruise than for their age-goup as a whole. Moreover, the comorbidities that are associated with higher risk of death if infected with COVID-19 appear to be hypertension, diabetes, coronary heart disease, atrial fibrillation, chronic renal failure, active cancer within the last 5 years and chronic obstructive pulmonary disease, and many of the passengers may well have suffered from one or more of these. I have found it impossible to obtain proper statistics for death rates on cruises, but it seems that deaths are by no means uncommon.
Even if the expected death rate among Diamond Princess passengers in the absence of COVID-19 would have been lower than in the same age-groups of the general UK population, it is also the case that those passengers who died of COVID-19 were very likely in relatively poor health and hence considerably more likely to have died over the next year or two than people of the same age in the UK population as a whole.
I found your article quite interesting. There are many references to which I am unfamiliar but for the sake of argument will accept their validity. Your arguments for using The Diamond Princess as a case study would appear sound at first glance, although under closer scrutiny one has to ask if they offer a true representation of the COVID-19 virus and therefore as an accurate tool for modelling.
My first observation would be that at the time of writing you suggest there have been 8 deaths from those aboard. Whilst allegedly not being the most accurate of sites at times, Wikipedia suggests there have been 10 deaths associated with passengers/crew, with 712 of 3,111 being infected. REF: https://en.wikipedia.org/wiki/Diamond_Princess_(ship) this would immediately put your entire estimates 40-50% closer to the governments, which is itself based upon the vast cross section analysed by Imperial College from across the world.
Your report makes conclusions upon the death toll by age figures, again using The Diamond Princess Data – suggesting that elderly – over 60’s – are more vulnerable to COVID-19. I would like to ask if you have considered the cross section of ages aboard The Diamond Princess at the time of the outbreak? Perhaps I am not looking closely enough, but travel/cruise industry reports suggest average passenger ages on cruise ships to be made up of in the region of 60% being above 50 years old (Source: CLIA Global Passenger Report 2018) with the Median age being seriously dragged down due largely to couples with young children travelling.
I too have reservations regarding the figures published by others and do not wish to belittle your findings whatsoever, simply ask you to challenge yourself on some of the figures in this ever so quickly changing environment.
For example, a figure of great interest, which I believe will begin to filter through within the next few weeks, is at what stage; who and how many people have actually been tested. My best guess at this stage (before/during the roll-out to keyworkers) is that only those admitted to hospital and some key figures within government circles have probably been tested. Therefore how many people have recovered – a question you do refer to in your report.
Another anomaly, if you like, is the recovery rate of those put onto ventilators or rather the lack or success of taking those individuals successfully ‘off’ ventilators. Pneumonia is a known side effect of extended time spent attached to such apparatus and also a symptom of the virus itself. Indeed, albeit best efforts are being made can the manufacture of ventilators be expedited to an extent that there are enough to go around at all as the numbers of those requiring them spike?
I do not wish to scaremonger, I am not an academic. I do however believe that preparing for the worst whilst hoping for the best is an entirely more sensible course of action than that of its polar opposite and I salute those in government, our keyworkers and essential staff whom are very much on the frontline, in defending our nations against what might otherwise be somewhat catastrophic to many nations and in so many ways.
Kindest regards and stay safe.
Simon, Thank you for your comment. The Worldometer figure of 10 deaths given by Wikipedia as the source of that figure does not itself give any source, and the World Health Authority situaion reports still show only 7 deaths (excluding one outside Japan). However, as per the updated version of the article that I published on 27 March, it does indeed appear that a further two deaths of Diamond Princess occupants have occurred since I originally wrote the article. Unfortunately, the source documents for recent deaths were only published in Japanese, and searching for them is a bit tricky.
The cross-section of ages aboard the Diamond Princess was fully taken into account in my analysis. The average age was indeed high, at 58 years, despite over 25% of them being crew.
The problem that you refer to of most people not being tested is precisely why the Diamond Princess dataset, where all people were tested and the health of those infected was thereafter monitored, is so useful.
Hi … very interesting analysis. However the policy actions taken by the UK are really driven by the need to “protect the NHS”. That is, the Imperial team’s model suggests that ICU capacity would be totally overwhelmed. This is based on their assumptions of 4.4% infections are hospitalized and of those, 30% require ICU. Were you able to conclude similar numbers for the cruise ship?
Jason, I’m afraid that the Diamond Princess data isn’t directly informative about those assumptions. However, the Imperial team’s hospitalisation rate is based on 2/3 of infections being symptomatic, whereas for 50+ year olds (which account for the vast bulk of their assumed hospitalisations) the Diamond Princess data suggests that the symptomatic proportion will be slightly under 50%.
Nicholas, thank you for your response. There’s an interesting report showing only 8 deaths on The Diamond Princess, yet it does suggest as of 26th March; “11 U.S. passengers remain hospitalized in Japan (median age = 75 years), including seven in serious condition (median age = 76 years).” REF: https://www.cdc.gov/mmwr/volumes/69/wr/mm6912e3.htm?s_cid=mm6912e3_w so The Diamond Princess scenario remains unresolved.
It must be very difficult as a scientist to produce such evidence relying solely on verified information, previous reports and yet omitting what I would call ‘The human intelligence factor”. The Worldometer reference you appear not to accept as you say it comes from Wikipedia directly – I wasn’t aware of that – and no reliable source can be found. This must make very difficult work for modelling purposes. After all what, especially in this day and age, is considered to be a trusted source?
When it comes to the majority of analysis it would appear, on face value, that the world is looking to China who have not only flattened the curve but rather stopped it completely in its tracks. Once that information is partnered with the expulsion of US foreign journalists and furthermore a discussion between the Chinese President Xi Jinping and President Trump that allegedly took place very close to the timing of the announcement that he had ordered US manufacturers to manufacture 100,000 or more new ventilators post haste, one has to ask whether or not the Chinese figures are reliable?
There are also anomalies in our own reporting methods in the UK when it comes to ‘closed cases’. Whilst most other European countries are reporting in the region of 18% fatality and 92% recovery on closed cases. The UK returns have been consistently higher, indeed almost the polar opposite at 90% fatality vs 10% recovery on closed cases.
I can appreciate that each country will have, and wish to retain, its own standards, although one would surely, in such an unprecedented situation, expect the WHO to set out very clear global guidelines/instruction as to what data is reported and when, if data modelling is to be used accurately in key procedural decisions within governments?
I think the argument stands that The Diamond Princess was probably a very useful ‘early’ analysis; yet, as the cross section of passengers compared to the current number of cases is so very low and cross referencing the current situation could contain vastly higher amounts of data regarding; age ranges, ethnicities, associated health issues relating to potential fatalities, gender, incubation periods, transmission periods, treatment periods etc etc, I think I would opt for the later… that is, if everybody was singing off the same hymn sheet and indeed the virus itself doesn’t ‘morph’ along the way?
Kindest regards and stay safe.
I cannot help but be somewhat concerned by a link between an increase in flu jabs in recent years and this outbreak. I am NOT suggesting a direct causal link but more a statistical one. When you add in an exceptionally mild winter the result was that we just didn’t suffer the usual population correction this winter. The result is a sudden correction now by the first thing that comes along. If it wasn’t this it would have been something else, people have to die of something. I would suggest that flu jabs have contributed to this compression of deaths into a small time frame.
I’m not aware of any evidence linking the COVID-19 outbreak with flu vaccinations, nor would I expect that to be the case. However, I agree that it is almost certainly true that had this winter’s death rate from flu been higher, the number of deaths ascribed to COVID-19 would be lower.
Have you asked others to peer review your article? When they do is this something that should be passed on to Boris.??
Whats the number of deaths you predict for a do nothing scenario? Why have so many people Fergusen et al got it wrong.?
Ben, I don’t think trying to turn this article into a peer reviewed paper makes sense, certainly at this time.
I think one needs data, from antibody-based testing, on what proportion of the population is not susceptible to COVID-19, due to having been infected but recovered or already being immune, in order to make a good estimate of the number of deaths in a do-nothing-scenario. But bear in mind also that the vast majority of deaths form COVID-19 are among old people with serious existing medical conditions. A non-negligible proportion of such individuals would probably have died anyway over the one to two year period for which anti-COVID-19 policies are likely to be in force.
A friend from Japan passed on this link to me.
As of May 8, 13 deaths, with 8 still in hospital. It may not include the Perth death. Deaths in the US have in some cases been labeled related to international travel.
Under updates on Covid 19.
Matt, that’s quite correct; now 14 deaths, including that in Australia. I’m afraid I stopped updating this post, but I’ve kept updating my model. As of 20 May the number of people in intensive hospital care is down to 4. The increase in fatalities is largely in the 70-79 age group, although one person aged 60-69 died. Of course, the number of natural deaths that would be expected to have occurred by now has also increased with the passage of time, although to a much lesser extent.
Hello, I stumbled upon this page looking for a record-level Diamond Princess dataset. Does one exist? If not, I was thinking about creating one. While I would have to impute a lot of the information, it would still have teaching value, I think.
Incidentally, early on I applied the Diamond Princess age-specific death rates to the US and came up with 150,000 deaths. At the time that seemed like a wildly optimistic number, then for a while it seemed wildly pessimistic, and now it seems like the number we are moving toward.
There is fair amount of Diamond Princess data, but it is not very well organised. Most of the information is from releases by the Japanese National Institute of Infectious Diseases. I give one or two links to it here. There is also a page at Wikipedia.