Why comparing coronavirus outbreaks in different countries can be misleading – and even dangerous

WASHINGTON, DC - NOVEMBER 15:  U.S. Surgeon General Jerome Adams testifies before the Senate Health, Education, Labor and Pensions Committee in the Dirksen Senate Office Building on Capitol Hill November 15, 2017 in Washington, DC. Adams testified about community-level health promotion programs and businesses that offer incentives to employees that practice healthy lifestyles.  (Photo by Chip Somodevilla/Getty Images)
Surgeon general's stark warning: It's going to get bad
01:34 - Source: CNN
CNN  — 

Why is the death rate so low in Germany? How come the UK has comparatively fewer cases than elsewhere? And what’s going on in Russia?

An online map of global coronavirus cases, compiled by Johns Hopkins University (JHU), is, according to the university, being viewed more than 1 billion times every day by people who want to see the figures for themselves. Questions once reserved for epidemiologists are now being discussed over dinner tables and around (virtual) water coolers across the world.

But what if the numbers aren’t painting the full picture?

For people who are comparing the data and attempting to divine how different countries’ epidemics stack up against each other, experts have a warning: nations have different reporting standards, different approaches to testing, and different approaches to tracing cases, all of which makes such comparisons dangerously misleading.

Sheila Bird, a renowned British biostatistician and professor at Cambridge University, says testing and reporting have been inconsistent even within individual countries, let alone internationally. Rules on who gets tested – and where and when – can evolve, and these changes and time lapses are not necessarily captured in the data published by different governments.

Bird is one of a number of high-profile scientists who have been calling on the UK government to run more coronavirus tests and provide more nuanced data. The UK government says the health system doesn’t currently have the capacity to test everyone who has Covid-19 symptoms. Only patients sick enough to require hospital treatment get tested, according to the government. The country had more than 9,962 cases and 467 deaths as of Thursday, according to JHU.

Meanwhile in Germany, anyone who shows flu-like symptoms and has in the past 14 days come into contact with a confirmed case, or traveled to a high-risk region, gets tested, according to official guidelines published by the health ministry. According to JHU, more than 41,519 people there have tested positive as of Thursday, with just 239 deaths.

And in South Korea, there is free and easy access to testing for anyone whom a doctor deems needs it, and authorities have been tracing the contacts of infected patients. As of Wednesday, the country had 9,241 cases and 131 deaths, according to JHU.

The relatively low number of tests done in the UK helps explain why the case numbers appear so much lower in the JHU map compared to other European countries. It doesn’t mean fewer people are sick, necessarily – just that fewer people are being tested.

“The danger is relying on the figures and assuming that they give us an accurate count of the total number of people who have been infected,” said Mike Tildesley, an associate professor at Warwick University.

The lack of testing could also make the situation look worse than it is. Based on the number of confirmed cases and reported deaths, the UK appears to have a relatively high Covid-19 mortality rate. But that’s not necessarily the case, because in the UK, only those who are very ill get tested. People with mild symptoms, who are very likely to recover, are being told to stay home without being tested – meaning they’re not being captured by the data.

Internationally, the comparisons rely on a piecemeal of different sources. The JHU map, which has become the go-to resource for many media organizations and experts, pulls in data from the World Health Organization, the US Centers for Disease Control and Prevention, the European Center for Disease Prevention and Control and China’s national health commission, in addition to local media reports, health departments, and the online community for health care professionals known as DXY.

Bird said comparing such numbers could be dangerous. “Lacking assured like-with-like comparators in other countries, we risk making heroic, and perhaps heroically wrong, inferences country-to-country,” she told CNN in an email.

Tildesley, who specializes in developing mathematical models to simulate the spread of diseases, said people need to be mindful of the bigger picture. “The fear is that people may look at their own risk, consider it’s low and then not take the measures that are being advised,” he said. His advice: Even though the likelihood of any one individual in the UK being infected is very low at the moment, everyone should behave as if they were.

Rather than focus too much on the actual numbers, scientists spend a lot of time looking at the epidemic curve – the shape of the graph that captures the total number of new cases in each country day by day.

“If different nations have different standards and conditions, they at least generate a consistent curve if those standards and conditions are stable across time,” said Lisa Gitelman, an NYU professor and the editor of a book called ‘“Raw Data’ Is an Oxymoron.”

So, if Italy keeps testing people at similar rate and its daily rate of increase in new cases starts going down – the curve starts flattening – that’s good news. But the testing rate needs to stay the same. If the UK suddenly starts testing many more people and sees a big jump in new cases, it doesn’t necessarily mean the epidemic is spreading faster. Similarly, if a country runs out of tests, it may suddenly report a misleading drop in new cases.