Methodological Research for Health/Care System Modelling: Identifying and Measuring Spillover Effects




Changes in health services, social care and public health delivery and policy are often aimed at an organisational rather than at a clinician and/or patient level. Such changes are often introduced in a non-experimental manner, have widespread consequences both for outcomes and costs, and are subject to equilibrating and offsetting mechanisms, all of which introduce layers of complexity.

While the methods for modelling the introduction of specific health technologies targeted at clinician and/or patient level are well-established, the methods for modelling health and social care service/public health innovations at an organisational/complex system level, and for using such models to guide evaluation of system/organisation-level changes, are less well-developed.

At a system level, there is a wide range of consequences that are of concern, including measures of access, quality, safety, efficiency/cost-effectiveness and equity of health care. Some interventions are targeted at particular outcomes but many have a generic focus and potentially wide consequences across a range of process and outcome measures. The scope of these changes both in terms of organisations and outcomes often precludes primary quantitative data collection, leading to a reliance on administrative data.

The inter-related nature of inputs, processes and outcomes in healthcare and public health organisations/systems, and the interactions between them, mean that changes to one component of a system will have multiple consequences for the system as a whole. Relatively little is known about change mechanisms, i.e. how the inputs at an aggregate level (e.g. staff, capital and equipment) are related to the outputs (i.e. the production process), and the reactions of the various actors and agencies in the system to each other.

The fact that organisations and teams perform multiple tasks, only a subset of which can be measured, adds considerable complexity. In services with multiple objectives such as health care, the question of whether these objectives are substitutes or complements has added pertinence (Kaarboe and Siciliani, 2011).

These wider, and potentially unanticipated, effects of introducing a change in health service/public health delivery/policy (which might not commonly be measured or anticipated as outcomes in an evaluation) are known as Spillover Effects, and have been shown to be substantial in several healthcare settings including interactions between (Baicker et al, 2013) and within organisations (Sutton et al, 2010).  Unanticipated Spillover Effects might be positive or negative.

Further background is provided in these recent vignettes, commissioned by the MRC-NIHR Methodology Research Programme Advisory Group, from Matt Sutton (University of Manchester) and colleagues (PDF, 111KB), and from Martin Utley (UCL) and colleagues (PDF, 262KB).

Highlight Notice

MRC and NIHR invite applications through the Methodology Research Programme to conduct research into methodologies for developing models which may enable identification, measurement and understanding of spillover effects in health service and public health systems.

The primary methodological research need that applications under this highlight notice should consider is the development of an underpinning methodology for modelling changes in healthcare/public health delivery and policy which will enable identification and measurement of spillover effects.

Development of such modelling methods should take account of the wider needs of research aiming at modelling changes in health service/public health delivery/policy, increasing the ability of modelling to deliver against challenges such as:

  • Measuring and modelling longer-term consequences of changes in health service delivery/policy, and understanding equilibrating and offsetting mechanisms.
  • Improving economic evaluation and cost-benefit analysis of changes in health service delivery/policy through modelling.
  • Improving the theoretical underpinning for empirical analyses of health care organisations, incorporating the theoretical literature on the behaviour of organisations and the individuals and teams that work within them.

Systems modelling approaches might be drawn from fields such as engineering/production modelling, or involve techniques such as complex system modelling through agent based modelling etc.

Application process and schedule

Applications for projects are invited through the normal MRC funding grant schemes (research grant, new investigator research grant etc.) to the standard MRP deadlines and will be considered at the regular MRP Panel meetings. These will be in competition with other response-mode applications received, but the Panel will be mindful of the strategic importance of this area.

MRC-NIHR Methodology Research Programme

Contact and guidance

The titles of all applications in response to this highlight should be prefixed with HMS: when filling out the JES form, and on any attachments, e.g. “HMS: A method for…”

It is essential to discuss your proposals with MRC Head Office at an early stage. All applications must be approved by the Methodology Programme Manager prior to submission. Please contact:

Dr Sam Rowley


Baicker, K., Chernew, M.E. & Robbins, J.A., 2013. The spillover effects of Medicare managed care: Medicare Advantage and hospital utilization. Journal of Health Economics, 32(6): 1289–1300.

Kaarboe O, Siciliani L. Multi-tasking, quality and pay for performance. Health Economics, 2011; 20(2): 225–238.

Sutton MA, Elder R, Guthrie B, Watt G. Record rewards: the effects of targeted quality incentives on the recording of risk factors by primary care providers. Health Economics, 2010; 19(1): 1-13.