AI-Assisted Decision Making: Healthcare’s Next Frontier

This report is based on a 2019 Healthcare Analytics Summit presentation given by Jason Jones, Chief Data Scientist Officer, Health Catalyst, entitled, “AI for Healthcare Leaders: The New Frontier for Improved Leadership Decision Making.”

Technology is rapidly transforming every industry, including–perhaps especially–healthcare. In 2015, the World Economic Forum conducted a survey to help predict when transformative technologies would become mainstream. Nearly half of the 800 executives surveyed answered that they expected the first Artificial Intelligence (AI) machine to be on a board of directors of a business by 2025 and the first transplant using a 3D-printed liver is likely by 2024.

Attendees at the 2019 Healthcare Analytics Summit (HAS) weren’t quite as ready to hand over decision making to machines. When Health Catalyst Chief Data Scientist Jason Jones posed the question to attendees of his course AI for Healthcare Leaders, “How likely is your Board of Directors to have a machine as a voting member by 2025?” 33 percent of attendees answered that it was “inconceivable,” while the rest (67 percent) answered that they were “unsure.” Jones posited that perhaps they needed a machine to help them answer the question.

AI-Assisted Decision Making at the Highest Levels of Leadership

While there may not be a machine at the helm of every major health system by 2025, AI should have a place at the table when it comes to healthcare decision making. A new frontier is expanding AI from artificial intelligence to augmented intelligence. Traditional AI focuses on improving analytics efficiency and effectiveness while augmented intelligence is about improving the decision-making ability of healthcare leaders. AI can help support leaders in driving systemwide outcomes improvement and answering important questions such as “Is there more opportunity in readmission or depression?” “How many staff are need in the ED on weekends?” and “How long does a nurse manager need to improve safety culture?”

There is an opportunity for AI to assist decision making in new and innovative ways, but if healthcare leaders aren’t prepared for this step forward, they will continue to underutilize AI and machine learning (ML). Leaders that embrace this paradigm can drive outcomes improvement using augmented intelligence.

The Importance of Strategic Direction

Because healthcare is at a technological tipping point, it is important that healthcare decision makers decide if and how to adopt AI technologies for their organizations. While the conversation around AI-assisted decision making is more meaningful because of increased efficiencies and algorithms, leaders need to implement AI thoughtfully in order to make sure AI supports their goals and values. A good analogy is that while drivers may be able to get somewhere faster because of Google Maps, the technology isn’t going to help them if they don’t know where they’re going in the first place. Healthcare leaders need to be able to clearly state, interrogate, and refine their goals and values along their AI journey.

Challenges of Leadership Reporting

If augmented intelligence can play a role in decision making, the first question to ask is what healthcare leaders need to be able to do and how AI can assist. The two biggest needs from AI are separating signal from noise and making decisions that impact the future (while leveraging data to accomplish both):

• Separating signal from noise – Is this hospital better than that one? Have we improved over time? If we set an improvement goal, is it statistically different from current performance?
• Making decisions that impact the future – Where will we be in a year? Are we satisfied with that? If not, what will we change and when can we expect to see a result? If we’re satisfied, is performance sustainable?

The sample report below (Figure 1) shows how common ML algorithms can be used to help answer these questions. In the example, the x-axis represents performance while seven different geographies are represented on the y-axis. The horizontal lines represent confidence limits that vary in size representative to the size differential between the geographies. Along the right side of the graph, letters represent whether these geographies are statistically different from one another. Additionally, the grey diamonds represent an automated forecast prediction of where these geographies will be in one year.

While the report shows some information that is included in most standard quarterly reporting, it adds new dimensions to give leaders more information with which to make key decisions while removing some of the cognitive burden of interpreting the data. The report uses a standard ML algorithm called recursive partitioning, which is often used in readmission prediction and case management selection. While these ML tools are frequently used at the point of care, few organizations are applying these same models to high-level decision making even though they are capable of providing both types of information.

The example report uses augmented intelligence to help separate signal from noise (by distinguishing the different geographies) and includes data that helps inform future decisions (by showing forecasted predictions of the geographies’ performance in one year). Healthcare leaders can take this information and see that while the performance of some geographies (geographies A, D, and E) are predicted to stay relatively even or decline slightly, others (geographies B and C) are predicted to improve significantly. That information changes the discussion from “How is the organization performing?” to “Are there significant differences between geographies B and C to A, D, and E that need to be accounted for?” and “Is this performance sustainable for those geographies?” While many leaders are reluctant to trust the forecasting of organizational performance to computers, the reality is that they’re much better at this than humans.

How AI-Assisted Decision Making Relates to Health Equity

An unexpected application of augmented intelligence relates to healthcare equity. The Agency for Healthcare Research and Quality (AGRQ) defines six domains of healthcare quality including safe, effective, person-centered, timely, efficient, and equitable. AHRQ defines the equitable aim for healthcare as “Quality does not vary by personal characteristics, such as gender, ethnicity, geography, and socioeconomic status.” The example report shown in Figure 1 above illustrates how geography impacts healthcare quality. While AI and ML can perpetuate biases in healthcare, it’s also possible that these same tools can improve healthcare equity, particularly when applied to decision making.

Healthcare organizations report on several hundred performance measures. What if they also knew how equitably they were performing each measure across race, gender, ethnicity, geography, and socioeconomic status? This could be done by using the Gini index of inequality, but instead of applying it to income equality, it was applied across healthcare performance measures (Figure 2).

Figure 2 shows how performance measures might be ranked using the Gini index to show areas of higher inequality that healthcare leaders could then use to determine areas of focus. In this example, the health system might focus on Hemoglobin A1C control, which shows inequity based on age. A common ML tool called the Receiver Operating Characteristic (ROC) curve is a measure of how well data scientists can predict an outcome. Just like data scientists can use this tool to predict what patients will contract severe sepsis or what patients will be readmitted, they can also predict how equitably the health system can control hemoglobin A1C across the patient population. Armed with that information, healthcare leaders can then improve healthcare inequalities across that measurement and others.

What’s Next for AI?

While many healthcare organizations have implemented AI and ML tools at the point of care, few have been able to apply them to decision making at the highest levels. The good news is that opportunities to do so abound. Augmented intelligence can be applied in a leadership context to help separate signal from noise, make future-oriented decisions, and be applied to some of the most complex problems, such as solving healthcare inequality. In order for healthcare to keep moving forward, healthcare leaders need to encourage the use of these tools both at the point of care and at the highest levels of decision making. As advances in AI allow data scientists new ways to make sense of data, leaders can embrace these technologies to improve healthcare through augmenting leadership decisions. 

Additional Reading

Would you like to learn more about this topic? Here are some articles we suggest:

1. Artificial Intelligence in Healthcare: A Change Management Problem
2. Machine Learning Tools Unlock the Most Critical Insights from Unstructured Health Data
3. A New Era of Personalized Medicine: The Power of Analytics and AI
4. How Artificial Intelligence Can Overcome Healthcare Data Security Challenges and Improve Patient Trust
5. Healthcare Data Management: Three Principles of Using Data to Its Full Potential