Are non-constant rates and non-proportional treatment effects accounted for in the design and analysis of randomised controlled trials? A review of current practice

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Background: Most clinical trials with time-To-event primary outcomes are designed assuming constant event rates and proportional hazards over time. Non-constant event rates and non-proportional hazards are seen increasingly frequently in trials. The objectives of this review were firstly to identify whether non-constant event rates and time-dependent treatment effects were allowed for in sample size calculations of trials, and secondly to assess the methods used for the analysis and reporting of time-To-event outcomes including how researchers accounted for non-proportional treatment effects. Methods: We reviewed all original reports published between January and June 2017 in four high impact medical journals for trials for which the primary outcome involved time-To-event analysis. We recorded the methods used to analyse and present the main outcomes of the trial and assessed the reporting of assumptions underlying these methods. The sample size calculation was reviewed to see if the effect of either non-constant hazard rates or anticipated non-proportionality of the treatment effect was allowed for during the trial design. Results: From 446 original reports we identified 66 trials with a time-To-event primary outcome encompassing trial start dates from July 1995 to November 2014. The majority of these trials (73%) had sample size calculations that used standard formulae with a minority of trials (11%) using simulation for anticipated changing event rates and/or non-proportional hazards. Well-established analytical methods, Kaplan-Meier curves (98%), the log rank test (88%) and the Cox proportional hazards model (97%), were used almost exclusively for the main outcome. Parametric regression models were considered in 11% of the reports. Of the trials reporting inference from the Cox model, only 11% reported any results of testing the assumption of proportional hazards. Conclusions: Our review confirmed that when designing trials with time-To-event primary outcomes, methodologies assuming constant event rates and proportional hazards were predominantly used despite potential efficiencies in sample size needed or power achieved using alternative methods. The Cox proportional hazards model was used almost exclusively to present inferential results, yet testing and reporting of the pivotal assumption underpinning this estimation method was lacking.

Original languageEnglish
Article number103
Number of pages9
JournalBMC Medical Research Methodology
Volume19
Issue number1
DOIs
Publication statusPublished - 16 May 2019

Keywords

  • Event rates
  • Proportional hazards
  • Randomised controlled trial
  • Sample size calculation
  • Time-To-event outcome
  • Trial reporting

Cite this

@article{87eb5efc9cc64392ac6538e26604bdaa,
title = "Are non-constant rates and non-proportional treatment effects accounted for in the design and analysis of randomised controlled trials? A review of current practice",
abstract = "Background: Most clinical trials with time-To-event primary outcomes are designed assuming constant event rates and proportional hazards over time. Non-constant event rates and non-proportional hazards are seen increasingly frequently in trials. The objectives of this review were firstly to identify whether non-constant event rates and time-dependent treatment effects were allowed for in sample size calculations of trials, and secondly to assess the methods used for the analysis and reporting of time-To-event outcomes including how researchers accounted for non-proportional treatment effects. Methods: We reviewed all original reports published between January and June 2017 in four high impact medical journals for trials for which the primary outcome involved time-To-event analysis. We recorded the methods used to analyse and present the main outcomes of the trial and assessed the reporting of assumptions underlying these methods. The sample size calculation was reviewed to see if the effect of either non-constant hazard rates or anticipated non-proportionality of the treatment effect was allowed for during the trial design. Results: From 446 original reports we identified 66 trials with a time-To-event primary outcome encompassing trial start dates from July 1995 to November 2014. The majority of these trials (73{\%}) had sample size calculations that used standard formulae with a minority of trials (11{\%}) using simulation for anticipated changing event rates and/or non-proportional hazards. Well-established analytical methods, Kaplan-Meier curves (98{\%}), the log rank test (88{\%}) and the Cox proportional hazards model (97{\%}), were used almost exclusively for the main outcome. Parametric regression models were considered in 11{\%} of the reports. Of the trials reporting inference from the Cox model, only 11{\%} reported any results of testing the assumption of proportional hazards. Conclusions: Our review confirmed that when designing trials with time-To-event primary outcomes, methodologies assuming constant event rates and proportional hazards were predominantly used despite potential efficiencies in sample size needed or power achieved using alternative methods. The Cox proportional hazards model was used almost exclusively to present inferential results, yet testing and reporting of the pivotal assumption underpinning this estimation method was lacking.",
keywords = "Event rates, Proportional hazards, Randomised controlled trial, Sample size calculation, Time-To-event outcome, Trial reporting",
author = "Kim Jachno and Stephane Heritier and Rory Wolfe",
year = "2019",
month = "5",
day = "16",
doi = "10.1186/s12874-019-0749-1",
language = "English",
volume = "19",
journal = "BMC Medical Research Methodology",
issn = "1471-2288",
publisher = "Springer-Verlag London Ltd.",
number = "1",

}

TY - JOUR

T1 - Are non-constant rates and non-proportional treatment effects accounted for in the design and analysis of randomised controlled trials? A review of current practice

AU - Jachno, Kim

AU - Heritier, Stephane

AU - Wolfe, Rory

PY - 2019/5/16

Y1 - 2019/5/16

N2 - Background: Most clinical trials with time-To-event primary outcomes are designed assuming constant event rates and proportional hazards over time. Non-constant event rates and non-proportional hazards are seen increasingly frequently in trials. The objectives of this review were firstly to identify whether non-constant event rates and time-dependent treatment effects were allowed for in sample size calculations of trials, and secondly to assess the methods used for the analysis and reporting of time-To-event outcomes including how researchers accounted for non-proportional treatment effects. Methods: We reviewed all original reports published between January and June 2017 in four high impact medical journals for trials for which the primary outcome involved time-To-event analysis. We recorded the methods used to analyse and present the main outcomes of the trial and assessed the reporting of assumptions underlying these methods. The sample size calculation was reviewed to see if the effect of either non-constant hazard rates or anticipated non-proportionality of the treatment effect was allowed for during the trial design. Results: From 446 original reports we identified 66 trials with a time-To-event primary outcome encompassing trial start dates from July 1995 to November 2014. The majority of these trials (73%) had sample size calculations that used standard formulae with a minority of trials (11%) using simulation for anticipated changing event rates and/or non-proportional hazards. Well-established analytical methods, Kaplan-Meier curves (98%), the log rank test (88%) and the Cox proportional hazards model (97%), were used almost exclusively for the main outcome. Parametric regression models were considered in 11% of the reports. Of the trials reporting inference from the Cox model, only 11% reported any results of testing the assumption of proportional hazards. Conclusions: Our review confirmed that when designing trials with time-To-event primary outcomes, methodologies assuming constant event rates and proportional hazards were predominantly used despite potential efficiencies in sample size needed or power achieved using alternative methods. The Cox proportional hazards model was used almost exclusively to present inferential results, yet testing and reporting of the pivotal assumption underpinning this estimation method was lacking.

AB - Background: Most clinical trials with time-To-event primary outcomes are designed assuming constant event rates and proportional hazards over time. Non-constant event rates and non-proportional hazards are seen increasingly frequently in trials. The objectives of this review were firstly to identify whether non-constant event rates and time-dependent treatment effects were allowed for in sample size calculations of trials, and secondly to assess the methods used for the analysis and reporting of time-To-event outcomes including how researchers accounted for non-proportional treatment effects. Methods: We reviewed all original reports published between January and June 2017 in four high impact medical journals for trials for which the primary outcome involved time-To-event analysis. We recorded the methods used to analyse and present the main outcomes of the trial and assessed the reporting of assumptions underlying these methods. The sample size calculation was reviewed to see if the effect of either non-constant hazard rates or anticipated non-proportionality of the treatment effect was allowed for during the trial design. Results: From 446 original reports we identified 66 trials with a time-To-event primary outcome encompassing trial start dates from July 1995 to November 2014. The majority of these trials (73%) had sample size calculations that used standard formulae with a minority of trials (11%) using simulation for anticipated changing event rates and/or non-proportional hazards. Well-established analytical methods, Kaplan-Meier curves (98%), the log rank test (88%) and the Cox proportional hazards model (97%), were used almost exclusively for the main outcome. Parametric regression models were considered in 11% of the reports. Of the trials reporting inference from the Cox model, only 11% reported any results of testing the assumption of proportional hazards. Conclusions: Our review confirmed that when designing trials with time-To-event primary outcomes, methodologies assuming constant event rates and proportional hazards were predominantly used despite potential efficiencies in sample size needed or power achieved using alternative methods. The Cox proportional hazards model was used almost exclusively to present inferential results, yet testing and reporting of the pivotal assumption underpinning this estimation method was lacking.

KW - Event rates

KW - Proportional hazards

KW - Randomised controlled trial

KW - Sample size calculation

KW - Time-To-event outcome

KW - Trial reporting

UR - http://www.scopus.com/inward/record.url?scp=85065888056&partnerID=8YFLogxK

U2 - 10.1186/s12874-019-0749-1

DO - 10.1186/s12874-019-0749-1

M3 - Article

VL - 19

JO - BMC Medical Research Methodology

JF - BMC Medical Research Methodology

SN - 1471-2288

IS - 1

M1 - 103

ER -