Medical Grand Rounds featuring Dr. Tony Bai
Pierce Colpman, MSc Candidate (Translational Medicine)
Attendees to last week’s Medical Grand Rounds had the privilege of hearing from Dr. Tony Bai surrounding non-inferiority trials of new antibiotics. Dr. Bai firstly described the context and rationale for non-inferiority trials in approving antibiotics. It was explained that between 1990 to 2010, there was a void of new discoveries. Dr. Bai attributes this to the incentive for big pharma companies to invest their money elsewhere. The average cost of bringing a drug to market is similar across all diseases, approximately $500,000,000. However, there are vast differences in the return that companies will see. A cardiology drug can be broadly prescribed, and its use continued for weeks to years after it is developed. In contrast, an antibiotic drug will be restricted to one to two cases a year for a resistant organism, drastically limiting potential profits and deterring investors from seeing antibiotic drugs as a worthwhile financial investment (1).
Dr. Bai highlighted how this is changing with IDSAs commitment in 2010 to discover 10 new antibiotics prioritized against the ESKAPE pathogens that are effective and safe by 2020, along with regulatory and policy incentives associated with increasing antibiotic development. In 2010 The FDA published guidelines on how to approve new antibiotics for new common infectious diseases, and since then, there has been an explosion of non-inferior antibiotic trials. Infectious disease researchers believe that the efficacy of current antibiotics is adequate, and they are focusing their efforts largely on antibiotic conveniences, such as getting drugs into oral forms or creating drugs with a shorter duration. With this as the aim, non-inferiority trials are the perfect design, as the goal of infectious disease researchers is to prove that they are not compromising efficacy while increasing convenience.
A standard non-inferiority trial has two arms, both a treatment and comparison arm, to establish that the treatment arm is not inferior to the comparison arm. Once data is collected, a confidence interval is constructed around the point estimate. If this interval lies wholly to the right of the non-inferiority margin, then the trial has been deemed a success. When critically appraising non-inferiority trials, Dr. Bai suggests watching out for ways in which trialists cheat to obtain significance. Examples of this include using an inferior comparison arm, a larger non-inferiority margin, or a shorter confidence interval. Dr. Bai studied 227 non-inferiority trials for new antibiotics and measured them against quality markers of what a methodologically sound trial should contain. He concluded that there is much room for improvement and that many things we do to treat infections are propagated by tradition and not always supported by evidence (2). For example, Dr. Bai was taught that the intention to treat model (ITT) does not work for non-inferiority trials and instead per-protocol analysis (PP) should be used. However, in 2021 he investigated the empirical evidence and found that the point estimate of ITT and PP are the same. ITT yields a more conservative estimate based on the confidence interval in the majority of non-inferiority trials (3). Dr. Bai was also interested in the statistical methods used to determine the confidence interval, as the entire theory of the trial design is based on this calculation. He compared the 5 most common statistical methods used to calculate the confidence interval and concluded that the most conservative method is the Miettinen-Nurminen method, with the Wald method being the worst (4).
Dr. Bai then spoke to us about bio creep, the phenomenon where a small amount of absolute risk reduction is introduced that lies within the confidence interval. The treatment arm becomes the comparison arm through this mechanism, and a slow erosion of effectiveness is thought to be seen. Dr. Bai published a paper that tested to see if this was the case for antibiotics in non-inferiority trials. They found that in 227 non-inferiority trials for new antibiotics, bio creep was rare and is not occurring yet. Finally, Dr. Bai summarized the most useful new antibiotics that we can use in KGH and implored us to challenge traditional practices as sometimes they aren’t necessarily supported by recent evidence.
The TMED graduate students were lucky enough to continue talking to Dr. Bai about his research following his talk. We discussed how Dr. Bai’s research on non-inferiority trials could benefit patients as well as how antibiotics and their associated trials have been represented in the lay press. Finally, Dr. Bai outlined his education and career path, including how his clinical experience influenced his research passions, and ended with some wise advice for us surrounding skills for a great career, the forefront of which was an ability to be a nice person! On behalf of myself and my peers, I would like to thank Dr. Bai for presenting at Grand Rounds and for taking the extra time to discuss with myself and my peers.
- Silver, L. L. (2011). Challenges of antibacterial discovery. Clinical Microbiology Reviews, 24(1), 71–109. https://doi.org/10.1128/cmr.00030-10
- Bai, A. D., Komorowski, A. S., Lo, C. K., Tandon, P., Li, X. X., Mokashi, V., Cvetkovic, A., Kay, V. R., Findlater, A., Liang, L., Loeb, M., & Mertz, D. (2020). Methodological and reporting quality of noninferiority randomized controlled trials comparing antibiotic therapies: A systematic review. Clinical Infectious Diseases, 73(7). https://doi.org/10.1093/cid/ciaa1353
- Bai, A. D., Komorowski, A. S., Lo, C. K., Tandon, P., Li, X. X., Mokashi, V., Cvetkovic, A., Findlater, A., Liang, L., Tomlinson, G., Loeb, M., & Mertz, D. (2021). Intention-to-treat analysis may be more conservative than per protocol analysis in antibiotic non-inferiority trials: A systematic review. BMC Medical Research Methodology, 21(1). https://doi.org/10.1186/s12874-021-01260-7
- Bai, A. D., Komorowski, A. S., Lo, C. K., Tandon, P., Li, X. X., Mokashi, V., Cvetkovic, A., Findlater, A., Liang, L., Tomlinson, G., Loeb, M., & Mertz, D. (2021). Confidence interval of risk difference by different statistical methods and its impact on the study conclusion in antibiotic non-inferiority trials. Trials, 22(1). https://doi.org/10.1186/s13063-021-05686-8
