Unreliability in Simulations of COVID-19 Cases and Deaths Based on Transmission Models
Keywords:
SIR model, COVID-19, Cross-validation, Sensitivity analysis, Confidence intervalAbstract
Predictions on the number of COVID-19 infections by researchers have failed repeatedly in Japan. In this review we discuss the mathematical models that led to these failures in predictions. Specifically, we focus on two papers published in October 2023, both of which simulate counterfactual COVID-19 cases and deaths using transmission models. One paper estimates that the COVID-19 cases and deaths from February 17 to November 30, 2021 in Japan would have been as many as 63.3 million and 364 thousand respectively had the vaccination not been implemented, where the 95% confidence interval is claimed to be less than 1% of the estimated value. It also claims that the cases and deaths could have been reduced by 54% and 48% respectively had the vaccination been implemented 14 days earlier. The other paper estimates that the number of cases in early 2022, in Tokyo would have been larger than the population in the age group under 49 in the absence of the vaccination program. We reexamine the results given by these papers to find that the simulation results do not explain the real-world data in Japan including prefectures with early/late vaccination schedules. The cause of the discrepancy is identified as the low reliability of model parameters that immensely affect the simulation results of case and death counts. Finally, we review a series of failed predictions on the number of infections during the COVID-19 pandemic in Japan and discuss the implications for future public health.
References
N. Ferguson, D. Laydon, G. Nedjati-Gilani, et al., Report 9: Impact of non-pharmaceutical interventions (NPIs) to
reduce COVID19 mortality and healthcare demand, Imperial College London, 20 (2020).
O.J. Watson, G. Barnsley, J. Toor, et al., Global impact of the first year of COVID-19 vaccination: a mathematical
modelling study, Lancet Infect. Dis., 22(9) (2022), 1293-302.
M. C. Fitzpatrick, S. M. Moghadas, A. Pandey, et al., Two years of U.S. COVID-19 vaccines have prevented millions
of hospitalizations and deaths, (2022).
G. A. Quinn, R. Connolly, C. ´OhAiseadha, et al., What lessons can be learned from the management of the COVID-19
pandemic?, Int. J. Public Health, 70 (2025), 1607727.
T. Kayano, Y. Ko, K. Otani, et al., Evaluating the COVID-19 vaccination program in Japan, 2021 using the counterfactual
reproduction number, Sci. Rep., 13 (2023), 17762.
T. Kayano and H. Nishiura, Assessing the COVID-19 vaccination program during the Omicron variant (B.1.1.529)
epidemic in early 2022, Tokyo, BMC Infect. Dis., 23 (2023), 748.
Kyodo News, COVID vaccine reduced cases by more than 90%, Kyoto Univ. estimates (in Japanese), Nov. 16, 2023.
Nikkei, More than 90% COVID cases reduced, Kyoto Univ. estimates (in Japanese), Nov. 16, 2023.
NHK, COVID vaccine drastically reduced infections and deaths, Kyoto Univ. estimates (in Japanese), Dec. 3, 2023.
PubPeer, Comments on publications.
T. Nakamura, Transmission model and arithmetic of dynamical system, ResearchGate (2024).
Our World in Data, Coronavirus pandemic country profile. Available at:
Scientific Advisory Group for Emergencies (UK), SPI-M-O: Summary of further modelling of easing restrictions?
Roadmap Step 4, June 9, 2021.
F. Campbell, B. Archer, H. Laurenson-Schafer, et al., Increased transmissibility and global spread of SARS-CoV-2
variants of concern as at June 2021, Euro Surveill., 26 (2021), 2100509.
R. Earnest, R. Uddin, N. Matluk, et al., Comparative transmissibility of SARS-CoV-2 variants Delta and Alpha in
New England, USA, Cell Rep. Med., 3(4) (2022), 100583.
N. Gavish, R. Yaari, A. Huppert, G. Katriel, Population-level implications of the Israeli booster campaign to curtail
COVID-19 resurgence, Sci. Transl. Med., 14 (2022).
COVID-19 Forecasting Team, Past SARS-CoV-2 infection protection against re-infection: a systematic review and
meta-analysis, Lancet, 401 (2023), 833-842.
C. M. Bishop, Pattern recognition and machine learning, Springer, (2006).
Sapporo Medical University, Transition of new coronavirus COVID-19 cases per population by prefectures (in
Japanese), Available at:
Tokyo Metropolitan Government, Populations in Tokyo (in Japanese). Available at:
K. Abbasi, Covid-19: politicisation,”corruption,” and suppression of science, BMJ, 371 (2020), m4425.
Ministry of Health, Labour andWelfare. Results of deliberations by the Disease and Disability Certification Review
Board, May 30, 2025 (in Japanese)
Y. Suzumura, Analysis of the association between BNT162b2 mRNA COVID-19 vaccination and deaths within 10
days after vaccination using the sex ratio in Japan, Cureus, 15(12) (2023), e50144.
The Japan Times, Excess deaths doubled in Japan in 2022. Available at:
BuzzFeed News, “Watching the Olympics at home” does not reduce infection: appeal for suspension of the Tokyo
Olympics again (in Japanese), Jul. 29, 2021. Available at:
BuzzFeed News, Delta strain, Olympics, summer holidaysc four hypotheses for decreasing infections (in
Japanese), Sep. 2, 2021. Available at:
Mainichi Shimbun, COVID goes “endemic” with repeated epidemics: comments by Hiroshi Nishiura (in
Japanese), May 4, 2023. Available at:
H. Kitano, COVID-19 AI & Simulation Project (in Japanese), Dec. 7, 2021. Available at:
Yomiuri Shimbun, Infection in Tokyo metropolitan area likely to peak in May (in Japanese), Apr. 3, 2023. Available
at:
H. Kakeya, Speech deposit: systematic approach to free speech and responsibility, Forum on Public Policy, 2008(1)
(2008).
P. D. Thacker, Covid-19: Researcher blows the whistle on data integrity issues in Pfizerfs vaccine trial, BMJ, 375
(2021), n2635.
J. Jureidini and L. B. McHenry, The illusion of evidence-based medicine, BMJ, 376 (2022), o702.
S. Hatfill, The COVID debacle: merging criminal law and medical science for accountability, J. Am. Phys. Surg.,
(4) (2023), 129-135.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Hideki Kakeya, Makoto Itoh, Yukari Kamijima, Takeshi Nitta, Yoshitaka Umeno
This work is licensed under a Creative Commons Attribution 4.0 International License.
