Unravelling healthcare complexity – a machine learning perspective.

The arc of history drawn by Sir Cyril Chantler in 1999 is increasingly clear; “Medicine used to be simple, ineffective, and relatively safe. Now it is complex, effective, and potentially dangerous”.

Rising complexity in healthcare, results in ineffective decision-making that leads to detrimental effects on care quality and escalates costs. This viewpoint motivates with examples the usefulness of predictive Machine Learning (ML) models to unravel healthcare complexity which has a clinical and an administrative perspective; and emphasizes to unleash
Electronic Health Record (EHR) functionality – while duly respecting all ethical and legal concerns – to reap full benefits of ML.

From an administrative perspective, an important source of complexity is healthcare organisation, which – unlike an engineered complex system – is a system that emerged over time with independent stakeholders (patients, care-providers, technologists, tax-payers; and hospitals, clinics, laboratories, government etc.) involved [1]. Controlling the output of such a socio-technical complex system is – if not impossible – very challenging because of the high degree of inter-relatedness among the role of stakeholders which renders the overall system output not equal to the sum of outputs of all the sub-systems. Put differently, unravelling organizational complexity aims at identifying problem areas in healthcare where interventions can have significant impact on the overall system output.

From a clinical perspective, an important source of complexity is the uncertainty in medical knowledge: explained by the complex human biology. International Classification of Diseases (ICD-10) specified over 68,000 diagnoses and the list keeps growing as we await ICD-11. As a result, the gap between the rapidly advancing medical knowledge base and the application of that knowledge to escalating population size continues to widen. Since clinical decisions deal with human life, we want to be as certain as possible about the outcome before recommending treatment to patients. Put differently, unravelling clinical complexity aims at reducing the uncertainty in clinical decisions.

Big data will unravel healthcare challenges. Faith in this conviction led to widespread development of ML algorithms for predictive insights that would facilitate informed decision- making in the present complex healthcare system. However, the potential of ML to predict outcomes and demands in clinical and organizational domain remains largely untapped: primarily due to functional limitations of EHRs and the inability to see the healthcare system as a whole.

Addressing organizational complexity – example

One implication of healthcare fragmentation is the lack of coordination among actors which results in long waiting time for patients to access the next step in the care chain. The benefits of reducing waiting times are several: better value for patient’s time, less work stress among caregivers, improved care quality and quick monetary reimbursements. However to reduce waiting times, we want to predict demand – or patient path – in order to allocate resources (staff, drugs, equipment etc.) efficiently. Put differently, the ML predictive challenge is: which path will the patient follow in the healthcare system using data available at the time of entry.

One hurdle to predict patient flow is the lack of interoperability of EHRs. In healthcare, we consider care fragments (primary, secondary, ambulatory) as independent bodies; a patient views them as a single entity when travelling through them during the care process. In order to address the ML predictive challenge, it is necessary to merge data from all levels of the care chain to have a complete approach to healthcare delivery [2].

Addressing clinical complexity – example

Congestive Heart Failure (CHF) is one of the leading causes of hospitalization and readmissions [3]. Readmissions are problematic because of high associated costs and increased medical risks. Hence it is essential to know what decides if a CHF patient needs to be discharged. The benefits of discharge are both monetary: hospital stays are expensive, bed availability is increased; and non-monetary: patients are less prone to medical complications and can spend more time with family and return to work. However the benefits depend on patient outcome (prognosis), so discharging a patient is worthy only if he or she does not need to be re-admitted in near future, which makes it a prediction problem. Put differently, the ML predictive challenge is: which patients are likely to be readmitted using data available at the point of discharge.

One barrier to outcome prediction is that patient information after the course of treatment – the outcome data – is not always available in EHR. Most state-of-the-art ML methods are trained in a supervised manner that requires inputs and outputs to learn the mapping function between them. Though, patient reported outcome and experience measures are being developed and validated [4], its integration into EHRs remains a challenge.


EHRs are, in sum, valuable resources not only for clinical decision-making but also for administrative purposes. In order to reap full benefits of ML in healthcare, we need to integrate patient information across all fragments in healthcare and supplement EHRs with patient generated outcome measures and cost (or resource) data while respecting all ethical, privacy and legal concerns. This rejiggering of EHRs opens new research challenges in fields
like data security, to create sustainable and safe data warehouses to store confidential medical information and facilitate responsible access to researchers when required. Simultaneously, research in data mining is encouraged to integrate, interpret and transform massive amounts of heterogeneous data into medically relevant resource. Dual-purposing of EHRs also demands reorientation of ethical and legal rules because in addition to medical professionals, EHRs are being accessed by data scientists and administrators.

By: Awais Ashfaq


  1. William B Rouse. Health care as a complex adaptive system: implications for design and management. Bridge-Washington-National Academy of Engineering-, 38(1):17, 2008.
  2. G Kaplan, G Bo-Linn, P Carayon, P Pronovost, W Rouse, P Reid, and R Saunders. Bringing a systems approach to health. Institute of Medicine, 2013.
  3. Mirkin, Katelin A., et al. “Risk factors for 30-day readmission in patients with congestive heart failure.” Heart & Lung: The Journal of Acute and Critical Care 46.5 (2017): 357-362
  4. Michael E Porter, Stefan Larsson, and Thomas H Lee. Standardizing patient outcomes measurement. New England Journal of Medicine, 374(6):504-506, 2016.