Observational data in healthcare decision making

Methods research to support the use of observational data in healthcare decision making 


For the purposes of this Highlight Notice, “observational data” refers to data from non-randomised comparisons. This can include data from single-arm studies, from registries, or data that were gathered within randomised trials but not analysed according to the randomised groups.  These data have the potential to provide information that is different and complementary to that from randomised controlled trials.  It is therefore important for organisations like the National Institute for Health and Care Excellence (NICE) and other government health-related agencies to understand and harness the utility of such data for informing decisions. 

Currently, observational data are used routinely to inform parameters in economic models such as the underlying rate of disease progression.  There is now increasing interest in using observational data to inform estimates of treatment effect. Observational analyses are subject to increased risk of bias, and decision-makers require analytical techniques that minimise this risk.  Decision-makers also need to be able to make an accurate assessment of the likely extent of bias and its impact on decision uncertainty (that is, uncertainty around whether to recommend a treatment for use in the NHS).

A range of new approaches for controlling for bias are starting to become more widely used which may be important in enabling the above.  These include instrumental variables (best known in the use of Mendelian randomisation in the bioinformatics arena), propensity scoring and graphical methods for exploring causality (e.g. Tetrad).  These may create additional methodological requirements for how observational data is collected and managed.

Methodology research is required to improve the use of observational data to augment information from randomised controlled trials, or to substitute for them in instances when randomised trials are infeasible, to ensure that care is informed by the best available evidence.  Accordingly, this highlight notice calls for methodological research to improve the ability of agencies like NICE to use observational data when considering treatment effects in forming health policy.

Case studies

The challenges currently facing NICE can be illustrated by a case study:

NICE technology appraisal 396, Ceritinib for previously treated anaplastic lymphoma kinase (ALK) positive non-small-cell lung cancer
Only non-randomised evidence was available to inform this appraisal. The data came from two single-arm studies of ceritinib and a separate retrospective analysis that measured overall survival in patients having best supportive care. The company did an indirect comparison of ceritinib with best supportive care, meaning the comparison was not adjusted for differences in patient or study characteristics between the studies. The appraisal committee concluded that ceritinib was likely to prolong life compared with best supportive care, but the extent of treatment benefit was highly uncertain because there was a high risk of bias from confounding. NICE would welcome methods to: A) quantify the possible extent of bias in this situation; B) provide a bias-adjusted estimate of effect size; and C) quantify the impact of bias on decision uncertainty. Such methods should ideally be suitable for situations when the analyst does not have access to individual patient data from all studies (as in this example, where the comparator data came from the literature).

Highlight notice

MRC and NIHR, through the Methodology Research Programme panel, are seeking to support the development of better methodologies for identifying, capturing, optimising, synthesising, interpreting and using observational data for healthcare decision making at the national level (such as by NICE).

Proposals should build on, rather than duplicate, previous work such as that completed by the GetReal initiative and methodological development from Bayesian approaches.

Use of Observational Data to Inform NICE Decision Making

MRC/NIHR seeks to support the development of methods for the use of observational data in informing NICE decision making, in two contexts:

(i) The synthesis of observational data with that from RCTs – how to incorporate these data types to add value and improve decision making.

(ii) Making decisions based on observational data alone, when no relevant trial data exist

Applications should aim to deliver tools which might be implementable by NICE in the near-to-medium term, and should cover methods for:

  • Optimal approaches to identification, assessment, quantification, control and adjustment for biases and confounding in the observational data to be used in decision making (approaches might include e.g. the use of instrumental variables, causal reasoning graphs and propensity scores).
  • Analysis and synthesis of data derived from different sources and study designs (i.e. observational and RCT) in order to inform NICE decision making. For example, methods to include observational data in network meta-analyses.
  • Methods for quantifying the impact of potential bias from observational data on decision uncertainty and how this could be accounted for in decision making.
  • New approaches for deriving robust estimates of intervention effectiveness from clinical databases, including routine care databases, disease-specific databases or national databases such as the Clinical Practice Research Datalink.

Applicants have the opportunity to benefit from discussion of their application with NICE prior to its submission.  See Contacts and Guidance below. 

Application process and schedule

Applications for projects are invited through the Methodology Research Programme Panel, to its regular deadlines and meetings. These will be in competition with other applications received, but the Panel will be mindful of the strategic importance of this area.

In accordance with the remit of the MRP applications should focus specifically on supporting methods development research where the proposed outputs are generalisable beyond an individual case study and where methods development is the primary purpose of the research.  Proposals are encouraged to include exemplar case studies which demonstrate the application of the developed methods.

Contact and guidance

The titles of all applications in response to this highlight should be prefixed with 'HOD1:' when filling out the JES form, E.g. “HOD1: A method for…”

It is essential to discuss your proposals with MRC Head Office at an early stage. All applications must be approved by the Methodology Programme Manager prior to submission. Please contact:

Dr Samuel Rowley


NICE wishes to help improve the quality of applications submitted under this highlight notice, and is able to offer advice and guidance on improving the scientific quality of proposals and their pathways to impact.  Applicants are encouraged to contact Dr Rosie Lovett (Senior Scientific Adviser) by emailing Please get in touch as early as possible.

This opportunity is only available to applicants who have already received confirmation from the MRC Programme Manager that their proposal falls under this highlight notice.

More information is available on the NICE website.