Research Signal – Source date 2022-07-07
A trial report can give readers a precise number of side effects while leaving important questions about that number difficult to answer. In a review of cancer immunotherapy trials, almost every publication reported absolute adverse-event counts. Yet only a small fraction adequately described how those events were collected or how the harms were analyzed.
That gap matters when people try to interpret published results. A count tells readers how many events the report recorded; methods help explain how researchers gathered and assessed that information. The review measured how completely trial publications described harms. It did not determine whether the treatments were safe or unsafe, or whether events had been missed.
A review of what trial papers disclosed
The authors searched PubMed, Embase, the Cochrane Library, and Web of Science for eligible publications dated January 1, 2010, through December 31, 2021. They focused on head-to-head phase II and III randomized trials involving people with cancer and comparing treatments that included immune checkpoint inhibitor therapy. These drugs are a form of immunotherapy that can affect how the immune system responds to cancer.
The review screened 2,608 records and included 123 trial publications in its assessment. The unit being scored was the publication: the authors examined what each report said about harms in its main text or appendix. They did not pool patient outcomes to compare the safety of different treatments.
To assess reporting, the reviewers adapted recommendations from the 2004 CONSORT extension on harms into a 16-element checklist. Each item received one point if the publication adequately reported it, or zero if it did not. The items covered different aspects of reporting, including whether harms appeared in the title, how adverse events were defined and collected, and how results were presented. Every item counted equally in the total.
The resulting harm-reporting quality score measured completeness against that checklist. It was not a score for treatment safety, the seriousness of a therapy’s side effects, or the quality of the researchers’ underlying data collection.
Counts were common; collection methods were not
Absolute numbers of adverse events appeared in 99% of the publications. Most also gave readers other useful pieces of the results: 99% reported harms in the abstract, 99% gave absolute event numbers, and 99% presented harm results separately for each study group. Nearly all reported severe events separately by event type.
But a reported total does not, on its own, show how researchers gathered the information behind it. Only 3% of publications adequately described how harm information was collected. Only 11% adequately described the methods used to analyze harms. The review also found that 60% clearly defined adverse events, and 47% stated when harm information was collected.
These are gaps in the published descriptions. They do not show that researchers collected events poorly, or that the counts were wrong. They do mean that a reader may have limited information for understanding how the trial moved from gathering reports of harms to the numbers printed in the paper. For a fair reading of a result, both the count and the process used to produce it can matter.
The omissions were visible in how the papers framed harms, too. Only two publications addressed harms in their titles. The review authors noted that titles may be constrained and that trial papers often emphasized efficacy. Still, a title can influence what readers expect to find in a report, while the word “safety” may sound more reassuring than a detailed account of potential harms.
Immune-related reactions need a timeline
The review also examined immune-related adverse events, or irAEs: adverse events associated with immune checkpoint inhibitor therapy. Reporting that such reactions occurred is one part of the picture. Their timing, duration, and management can also help readers understand what the reports describe.
In 89% of publications, immune-related adverse events were reported separately. But descriptions of when they began and how long they lasted were adequately reported in just 14% of publications when the main text and appendices were considered. Management was adequately reported in 33% across those materials. In the main text alone, the corresponding figures were 10% for onset and duration and 24% for management.
The figures point to a difference between listing reactions and describing their course. The review authors noted that immune-related events can have distinct onset patterns and management compared with toxicities from conventional treatments. Without adequate information about timing and clinical interventions, a reader gets a less complete account of what happened during the trial. This audit does not establish how often such events occur outside the publications it assessed.
Reporting scores varied across publications
Across the 123 publications, the mean score was 11.1 out of 16. Scores ranged from 5 to 14, and no publication received all 16 points. Most reports fell between 10 and 13. The average captures a mix: some reporting elements were common, while some methods and details about immune-related events were much less often described.
The authors also tested whether publication and trial characteristics were associated with scores. In their multivariable analysis, phase III publications had scores an average of 1.03 points higher than phase II publications. Publications in journals with an impact factor above 30 and below 60 scored an average of 1.192 points higher than those in journals below 30. These were associations among the reports reviewed; they do not explain why the scores differed, and they do not show that one trial phase or journal category produced better harm data.
The score is useful as a broad summary, but its construction shapes what it can tell us. Each checklist item received equal weight, even though readers might consider some details more consequential than others. The authors also restricted the review to randomized phase II and III trials, excluding observational studies. Such studies may be relevant to understanding harms over longer periods, so this audit cannot describe the completeness of that broader evidence or establish the incidence of medium- and long-term harms.
How this could shape the future
A patient and oncologist comparing existing cancer treatment options could have a more informative benefit-and-harm discussion if trial reports explained not only how many immune-related reactions occurred, but how the events were sought, when they appeared, and how they were managed. That kind of account would give clinical interpretation more context than a count alone.
This review identifies reporting gaps that stand between current publications and that possibility. Closing them would require consistent approaches to collecting harms, clearer reporting in trial papers, and clinical interpretation of the resulting evidence. A nearer sign of progress would be for future phase II and III trial publications to routinely describe collection and analysis methods alongside immune-related event timing and management. Better disclosure would mark progress in reporting; by itself, it would not demonstrate improved patient outcomes.
Key Sources
- Primary source: Adverse Event Reporting Quality in Cancer Clinical Trials Evaluating Immune Checkpoint Inhibitor Therapy: A Systematic Review.
DOI or Journal Page · Review or Meta-Analysis · 2022-07-07
Keep exploring