MMRM, Missing Data and Estimands in FDA Statistical Reviews of NDAs and BLAs

MMRM, missing data and estimands in FDA statistical reviews: what NDA and BLA reviewers look for

Mixed models for repeated measures (MMRM) have long been a standard primary analysis for continuous endpoints measured over time. FDA statistical reviews in Drugs@FDA show a consistent position: an MMRM does not solve the missing-data problem on its own. Its validity rests on a missing-at-random (MAR) assumption that the observed data cannot confirm, so reviewers ask whether that assumption is credible, whether conclusions survive plausible departures from it, and, more recently, which estimand the model actually targets. The ICH E9(R1) addendum, which FDA issued as final guidance in May 2021 ²¹², did not prohibit MMRM. It changed the question reviewers ask first. The question used to be "is the missing-data method acceptable?" Now it is "what treatment effect does this analysis estimate, and is that the effect of regulatory interest?"

The baseline position: MMRM is only as good as MAR

In reviews of applications decided before 2019, FDA statisticians treated MMRM as an analysis of observed repeated measurements whose interpretation depends on MAR. They did not regard MAR as something that pre-dropout responses could prove ²⁴²²⁴³. The practical response was to compare MMRM with analyses built on different assumptions about outcomes after dropout. Outcomes varied with the trial:

• Fetzima (levomilnacipran), NDA 204168. The sponsor used pattern-mixture models (PMMs) to test whether its primary MMRM findings held up under departures from MAR, with LOCF-based ANCOVA as another sensitivity analysis. The reviewer replicated the PMM results and noted some problems with the approach and how it was carried out, but concluded the efficacy finding remained supported ²⁴⁴²⁴⁵.
• Bydureon BCISE (exenatide), NDA 209210. The reviewer warned that the sponsor's MMRM on a modified ITT population could be biased if MAR failed, and that neither it nor the LOCF support analysis necessarily estimated a valid causal effect. FDA asked for post-hoc analyses that included post-rescue and post-discontinuation observations under missing-not-at-random (MNAR) assumptions: "return to baseline" PMM imputation in BCB118 and "wash out of treatment effect" in BCB120. A further review described the sponsor's approach as poorly suited to relatively high, imbalanced missingness. Alternative analyses, including trimmed means, confirmed efficacy in both Phase 3 studies ²⁴⁶²⁴⁷.
• Perseris (risperidone), NDA 210655. Both doses met the Day 57 PANSS objective under MMRM, and the sponsor's PMM and cross-pattern coefficient comparisons did not suggest a MAR violation ^248249250.
• Xelpros (latanoprost), NDA 206185. MMRM served as a sensitivity analysis. The reviewer called MAR untestable but reasonable after looking at the missing-data pattern, because fewer than 5% of patients discontinued early ²⁵¹.

The pattern is that reviewers could find a result robust to plausible alternatives to MAR, accept MAR where missingness was limited, or judge a sponsor's MMRM insufficient for the treatment-effect interpretation being claimed. No single FDA-wide verdict on MAR emerges from these reviews ^244247252.

Where MAR or the imputation approach was questioned

The Bronchitol (mannitol, NDA 202049) review shows the concern in its sharpest form. Reviewers found the primary MMRM's MAR assumption unjustified. Treatment-related early dropout excluded some randomized patients, and later missingness differed between arms, so estimated FEV1 benefits could be overstated. Reviewers examined multiple-imputation, pattern-mixture, tipping-point and BOCF sensitivity analyses. They pointed out that some supposedly conservative methods still retained the pre-discontinuation treatment benefit, and that BOCF understated uncertainty ¹⁷³¹⁷⁴.

For Trulicity, a BLA review questioned the LOCF-based primary analyses. It also cautioned that the secondary MMRM, which assumed MAR and excluded post-rescue measurements, estimates counterfactual outcomes after rescue or dropout that cannot be validated empirically ⁵⁶. In the Cimzia and Humira Crohn's disease reviews, reviewers warned that carried-forward imputation could be inappropriate when dropout followed worsening or lack of effect. They looked at longitudinal modeling and MMRM as alternatives ⁵⁷⁵⁸.

Single imputation: LOCF and BOCF lost standing

Reviewers repeatedly declined to treat LOCF, BOCF or hybrids as automatically conservative. In the AirDuo RespiClick review, the statistician said modified BOCF might look more conservative than LOCF but still carried the problems of single imputation. Citing the 2010 National Research Council report, the reviewer identified two risks: biased tests and confidence intervals if the imputation assumptions are wrong, and overstated precision when imputed values are treated as observed. The same reviewer noted that the sponsor had not clearly identified the estimand, and that stopping data collection at treatment discontinuation prevented reliable estimation of the effect regardless of adherence ⁸⁴. For Troxyca ER, FDA cited the same missing-data report, classified the proposed LOCF/BOCF mixture as single imputation, did not agree to it, and requested a method consistent with the report ⁸⁵.

The choice of method could decide the result. In Vimovo Study 309, the WOMAC pain comparison was significant under LOCF (p=0.027) but not under BOCF (p=0.35) ⁸⁶.

This matches the history of the guideline text. The original ICH E9 (1998) allowed imputation methods "ranging from" LOCF to more complex models, provided they were described, justified and shown to be robust. It did not make LOCF a default ¹¹⁵¹⁰⁰.

Stress-testing MAR: tipping points and reference-based imputation

The main way reviewers evaluated robustness was to make explicit, clinically interpretable MNAR assumptions and see whether conclusions held.

Tipping-point analyses. For ArmonAir/AirDuo, reviewers kept control-arm missing data under MAR and progressively worsened experimental-arm imputations. Overturning the positive trough-FEV1 comparisons required shifts of 2 to 10 times the observed treatment effects, which they judged generally implausible. They added that this supported the efficacy conclusion but did not establish the reliability of the estimated effect sizes for labeling ¹³⁹¹⁴⁰. For Rexulti (brexpiprazole) Study 230, delta-adjusted multiple imputation showed that robustness depended on which dropouts were assumed to be MNAR. The 4-mg comparison tipped only beyond a 100% reduction of the effect when the adjustment applied to lack-of-efficacy (or lack-of-efficacy plus adverse-event) dropouts, but tipped at 80% when it applied to all early dropouts ¹⁴¹¹²⁶. FDA has also asked for two-dimensional tipping-point grids that vary assumptions independently in each arm, as in the Olumiant review ¹⁴⁸.

Control-based imputation. In the Anoro Ellipta review, FDA found the sponsor's initial MAR, copy-differences-from-control and carried-forward analyses insufficient because they effectively kept a bronchodilator benefit after treatment stopped. FDA asked for imputation based on actual control-arm outcomes ¹⁴². The reviewer favored jump-to-reference (J2R) for the all-randomized-patients question. All comparisons remained significant, but effect estimates were about 20% to 30% smaller than under the primary model ¹⁴³. The reviewer preferred J2R over copy reference because a symptomatic bronchodilator's effect would be expected to have disappeared by the later assessment, whereas copy reference lets the advantage fade gradually ¹⁴⁵. Incruse Ellipta (umeclidinium) showed the same pattern: significance was retained with roughly 20% to 30% smaller trough-FEV1 effects under placebo-based J2R ¹⁴⁴.

The reviewer's choice between reference-based methods depended on pharmacology, meaning what would plausibly happen to a patient's outcome after stopping that drug. Statistical convenience was not the deciding factor.

How ICH E9(R1) reframed the question

ICH E9(R1) separates the estimand (the treatment effect to be estimated) from the estimator (the analysis method), the estimate, and sensitivity analysis. It defines an intercurrent event as an event after treatment start, such as discontinuation, additional treatment or death, that affects the interpretation or existence of a measurement ⁶⁹. An estimand has five attributes: treatment condition, population, variable, handling of intercurrent events, and population-level summary ^1011121314. Five strategies address intercurrent events ^{15161718192021}:

Four changes matter most for MMRM-based submissions.

1. Missing data are defined relative to the estimand. Data are missing only if they would be meaningful for the particular estimand. A post-discontinuation outcome is missing for a treatment-policy estimand but may be irrelevant, and so not missing, for another estimand. Treatment discontinuation is an intercurrent event, while study withdrawal produces missing data ^102103104105. FDA review comments on Ekterly (sebetralstat) Trial 301 apply exactly this distinction between study discontinuation and an intercurrent event ²⁸⁷.
2. Data collection after discontinuation is expected where the estimand needs it. Discontinuing, switching or adding treatment does not by itself make later outcomes unmeasurable. Data from patients followed after discontinuation can inform imputation models for those without follow-up ^106104107.
3. Estimators must align with the estimand, with assumptions stated and stress-tested. Stronger assumptions call for more extensive sensitivity analysis. If plausible departures cannot be examined, the estimand and estimator combination may be unsuitable for decisions. E9(R1) notes specifically that assuming discontinuers fare like patients who stay on treatment can be implausible when follow-up stops ^{10810911011123}.
4. Sensitivity and supplementary analyses are distinct. Sensitivity analysis tests robustness of inference about the same estimand. Supplementary analysis gives additional insight and plays a lesser role once the main estimate is robust ^112113114.

FDA's E9(R1) guidance asks for the primary estimand to be defined explicitly in the protocol, with an aligned main estimator and suitable sensitivity analyses prespecified ^213214215.

MMRM after E9(R1): treated as a hypothetical estimand

The clearest change in recent reviews is that reviewers now say outright that an MMRM fitted to on-treatment data, with post-event data excluded or ignored, implicitly targets a hypothetical estimand. They then ask whether that hypothetical is clinically meaningful.

• Eylea HD (aflibercept), BLA 761355 (PHOTON and PULSAR). FDA described the proposed MMRM as a hypothetical-strategy analysis in which outcomes after discontinuation or rescue are treated as MAR missing data. It found this acceptable only if intercurrent events and missingness were minimal and balanced. Otherwise it requested a specified alternative, suggested trimmed means, and recommended collecting post-discontinuation data for treatment-policy analysis. In PHOTON, few intercurrent events occurred, and the reviewer expected alternative handling to make little difference ^62172275276.
• Miplyffa (arimoclomol), NDA 214927 (NPC-002). The applicant itself described its MMRM as estimating outcomes as if everyone had stayed on blinded treatment through month 12. FDA disagreed that hypothetical scores after early escape or adverse-event discontinuation were clinically plausible. It said the analysis did not answer FDA's question of interest, recommended while-on-treatment and treatment-policy estimands, and questioned whether MAR described the missing hypothetical scores ¹⁷¹.
• Risvan (risperidone), NDA 214835. The reviewer read an MMRM primary analysis as applying a hypothetical strategy to all discontinuations. The reviewer then considered composite strategies that count treatment-related discontinuation as failure, and requested supplementary analyses and reference-based imputation ¹⁸⁰¹⁵².
• Jascayd (nerandomilast). For lung transplant, reviewers chose a while-on-treatment strategy over a hypothetical one, because MMRM-based hypothetical estimation would effectively extrapolate to unobserved post-transplant visits that lacked sufficient clinical meaning ²⁸²²⁸³.

MMRM remains acceptable where the hypothetical estimand is defensible, or where the model is applied to data that include post-discontinuation observations under a treatment-policy estimand. For Nexviazyme, the MMRM primary analysis used observed post-discontinuation data under a treatment-policy estimand and was supported by a tipping-point analysis against the noninferiority conclusion ⁴⁹⁵⁰⁴⁰. For Pombiliti with Opfolda (ATB200-03), FDA accepted the treatment-policy estimand but reserved judgment on whether MMRM estimated it adequately, given its normality and MAR requirements ⁵¹⁵². In the Omlonti (omidenepag isopropyl) review, the primary IOP MMRM under MAR was supported by delta-adjusted pattern-mixture imputation targeted at adverse-event, lack-of-efficacy and rescue-related missingness. Earlier FDA advice had asked for MNAR sensitivity analyses and continued data collection after discontinuation or rescue ^175176177. For Veozah (fezolinetant), FDA asked the applicant both to justify MAR and to run analyses that depart from it ¹⁷⁸¹⁷⁹.

Treatment policy, composites and retrieved dropouts

Post-E9(R1) reviews show no single FDA-preferred primary estimand. They do show a frequent preference for treatment-policy or composite strategies, with imputation tied to the strategy:

• Rhapsido (remibrutinib). FDA said a treatment-policy estimand covering all intercurrent events was of primary regulatory interest. The plan separated observed retrieved-dropout data from truly missing outcomes ^279280281.
• Tryvio (aprocitentan). The primary analysis used treatment policy for discontinuation and rescue. The reviewer preferred retrieved-dropout imputation in principle, but only 11 retrieved dropouts were available. The statistical team instead combined MAR imputation for treatment completers lacking Week 4 data with J2R for discontinuers, which gave results similar to the primary analysis ^277278146147.
• Tremfya (guselkumab), PSA3001/PSA3002. A composite estimand counted discontinuation, increases in specified concomitant drugs and prohibited therapy as treatment failure, with treatment policy as a supplementary analysis ²⁷⁰²⁷¹.
• Dupixent (nasal polyps). Rescue counted as treatment failure, post-discontinuation data were used, and MMRM served as a sensitivity analysis. FDA found the estimand, estimator and sensitivity analyses reasonable ³²⁵⁵.

FDA therapeutic-area guidance now writes these expectations in directly. The draft obesity guidance generally recommends a treatment-policy estimand, continued outcome collection after intercurrent events, retrieved-dropout multiple imputation and two-dimensional tipping-point analyses ¹⁸⁵¹⁸¹. The COVID-19 guidance recommends composite strategies for death and hospitalization and states that death is not missing data ¹⁸⁷. The draft chronic-pain guidance discourages single imputation and asks for imputation that reflects uncertainty ¹⁹⁷. The draft Crohn's disease and ulcerative colitis guidances call for an estimand for each endpoint ¹⁹⁵²⁰⁵.

Recurring review findings on estimand labels

Reviewers also check that the label a sponsor gives an estimand matches what the analysis actually does. For Lumvoa (veligrotug-vvze), the reviewer said a purported composite analysis that substituted baseline proptosis after an event behaved more like a hypothetical strategy. The difference did not change the conclusion given small missingness and large effects ²⁸⁴. For Pyzchiva (ustekinumab-ttwe), the sponsor proposed a principal-stratum primary estimand. The reviewer found the intercurrent events and hypothetical scenario insufficiently defined, treated affected data as missing, and ran additional sensitivity analyses ^28528659.

Practical implications for sponsors

Taken together, the reviews and guidance point to a consistent set of expectations:

• State the primary estimand and its intercurrent-event strategies in the protocol, and show that the MMRM or other estimator is aligned with it ²¹⁴²¹⁷.
• If MMRM excludes post-event data, present it as a hypothetical estimand and justify that hypothetical scenario clinically. Reviewers have rejected hypothetical scenarios they judged implausible ¹⁷¹²⁸³.
• Keep collecting efficacy data after treatment discontinuation, where possible, to support treatment-policy analyses and retrieved-dropout imputation ⁶²¹⁰⁶. Too few retrieved dropouts limits what can be done ¹⁴⁶.
• Prespecify MNAR sensitivity analyses (delta-adjusted or two-dimensional tipping points, J2R or copy reference) chosen to fit the drug's pharmacology, and expect reviewers to read both statistical significance and the attenuation of the effect size ^140143145.
• Do not rely on LOCF or BOCF as primary methods without strong justification ⁸⁴⁸⁵.