The comments pending before release analyzed work on the effectiveness of public health measures, including family isolation, social distancing and the use of masks. His conclusion is that these tools lack the quality to draw conclusions, but that doesn’t mean they don’t work. In this case, the vision of epidemiologists is more important than ever.
Since the beginning of the pandemic, we have tried to fight against covid-19 through the following measures: Family imprisonment, mask with School closed. But do we know how helpful they are in fighting the coronavirus?recent Before publication The conclusion is that the literature devoted to analyzing the impact of such policies so far cannot provide a reliable answer to this question. The author believes that we need better review, but also accept that sometimes the evidence we want cannot be obtained.
Stanford University researchers and co-authors of the study explained to SINC: “We found that published literature that tried to assess the impact of specific policies on covid-19 could not do this.” Noah Haber. He said: “This does not mean that these measures are ineffective, but the situation does not allow us to measure them correctly.”
Unfortunately proceed Well-designed research The method of assessing the impact of masks and social distance is not as simple as knowing whether a vaccine is effective. According to Haber, the difficulty lies in “too many things happening at the same time” and prevents “isolation of the effects of specific policies.” In addition, the “complexity” dynamics of infectious diseases must be increased.
The authors of this work analyzed 36 studies and natural, Jama, British Medical Journal, Number one with NASA, Whose purpose is to evaluate different Health policy. After selecting them, send them to three independent reviewers, who must identify the problems in the design and determine the usefulness of the conclusions.
The reviewers concluded that only four studies were “probably appropriate” in determining the impact of health policies on COVID-19. The most common reason for failure is failure to take into account the changes that have occurred at the same time as political and social measures.
Experts to manage the lack of evidence
Haber believes that the problem is beyond the 36 studies analyzed, but it can prove the implications of the results. “We can say that the literature surrounding this issue is flawed, but this does not necessarily mean that our understanding of these policies is poor.”
This is because knowledge about the usefulness of these measures is beyond their direct quantification range. Hubble said: “Some studies that we have not reviewed may provide a lot of information.” “In addition, policy makers have a lot of supporting evidence and experience with other epidemics and response policies.”
Hubble said: “We may never know for sure how effective each policy will be in the spring of 2020.” However, this does not mean that epidemiologists will turn a blind eye.
Professional skills are the ability to collect and order a lot of messy information.Therefore, experts such as epidemiologists and health policy experts are critical to making decisions, rather than relying on the title of the study
Haber said: “Professional skills are the ability to collect and order a large amount of disorganized information.” “This includes understanding and updating what we know and what we don’t know. In this case, We don’t know. This is one of the reasons why experts such as epidemiologists and health policy experts are critical to making decisions rather than relying on the title of the research.”
Epidemiologist at Johns Hopkins University Brooke Jellet It clarifies that “policy investigations are not necessary to know that the intervention mechanism will work.” For example, “from a biological point of view, we can be sure that quarantining someone during the infection stage will prevent further infection.” Citing laboratory experiments to show mask Even if they are not made under real conditions, they will slow down the transmission speed.
The pandemic has raised many questions, but when we seek scientific help, not everyone can find the answer. Haber said: “In some cases, it is impossible to design and conduct any form of information research, and we must learn to be satisfied with it.” Randomized controlled trial They are the holy grail of scientific evidence; but the researcher Pointed out last year One of these experiments has a huge design error, which aims to eliminate doubts about the effectiveness of masks once and for all.
“Sometimes, we don’t know much about understanding.” Haber said: “Randomized controlled trials may be incredible, almost miraculous evidence, or they may be completely useless and misleading. “The same thing happens with other types of research: it all depends, and it’s difficult to distinguish what’s useful.” “
Hubble is optimistic about the future. “I hope we can obtain better evidence in the following ways: Reopening policy “Compared to the initial response,” he said. The researchers recommend designing them to be “easier” to evaluate, but acknowledge that this will require “investment in collaboration, infrastructure and other resources.”
Covid-19 reproducibility crisis?
Hubble’s work is Before publication Ø PreprintThis work has not yet been recognized by the research community, so its results are preliminary and should be treated with caution. However, their conclusions remind us that it should not be considered that the research being reviewed has a reliable mark of quality is absolutely correct.
“We should not consider Peer review As a high-strength or high-quality proof,” Hubble said. “This is a reasonable indicator that the editor and two or three reviewers of the journal think [el trabajo] It is worthy of discussion among other researchers, but it does not necessarily indicate that it is worthy of influencing decision-making.”
The problems in the coronavirus literature are the same problems we have faced for decades.The difference is that now they have become more serious, more open, and more influential
In a year of continuous scientific development, the quality of published research on covid-19 has worried some scientists.Researcher, Aarhus University (Denmark) Serge Horbach This year published a report on How has the quality of peer review changed due to the pandemic. He concluded that the reviewers used different scales in the coronavirus research.
Hobach explained: “I think we have to be very critical of research that presents novel research.” “This applies to everyone, but the speed at which content related to covid-19 is produced and disseminated requires more scrutiny.”
Hubble argues that the problems experienced by the coronavirus literature are “same problems we have faced for decades.” He said the main difference is that “now they are becoming more serious, having a greater impact and being more open.”
“In order to use research results for policy or other purposes, it is vital that scientists must recognize the limitations of their research methods, and sometimes To thisHobach said. “It also enables them to explain the context in which their findings can be used.”
Horbach believes that reviewers are increasingly taking this into consideration. “Every researcher and political actor is responsible for rigorously evaluating new results when using them, rather than blindly choosing them.” In this sense, he believes that scientific articles should be updated through social networks and platforms (such as Pubpeer). Extensive public scrutiny is “promising.”
Does all of this mean that the scientific reproducibility crisis of COVID-19 is imminent? Horbach believes that it is too early to judge. Research quality. Some articles and pre-publications found to be defective will be withdrawn in the future, but not all. “He insisted, “This emphasizes the need to maintain a critical and skeptical attitude.”
Hubble hopes that the “new light” created by the pandemic on “old problems” will be enough to reform those fields that need it, from researchers and journalists to funding agencies and magazines.