Explanation methods that help users understand and trust machine learning models often describe how much certain features used in the model contribute to its prediction. For example, if a model predicts a patient’s risk of developing heart disease, a doctor may want to know how strongly the patient’s heart rate data influences that prediction.
But if those functions are so complex or intricate that the user cannot understand them, does the explanation method make sense?
MIT researchers aim to improve the interpretability of functions so that decision makers will be more comfortable using the results of machine learning models. Based on years of fieldwork, they developed a taxonomy to help developers create features that are easier for their target audience to understand.
“We found that in the real world, even though we used state-of-the-art ways to explain machine learning models, there’s still a lot of confusion that comes from the functions, not the model itself,” said Alexandra Zytek, an electrical engineering doctoral student. and computer science and lead author of a paper introducing the taxonomy†
To build the taxonomy, the researchers defined properties that make functions interpretable by five types of users, from artificial intelligence experts to the people affected by the prediction of a machine learning model. They also provide instructions on how modelers can convert functions into formats that are easier for a layperson to understand.
They hope their work will inspire modellers to consider using interpretable functions from the beginning of the development process, rather than working backwards and concentrating on explainability afterwards.
MIT co-authors include Dongyu Liu, a postdoc; visiting professor Laure Berti-Équille, research director at IRD; and senior author Kalyan Veeramachaneni, principal investigator in the Laboratory for Information and Decision Systems (LIDS) and leader of the Data to AI group. They are joined by Ignacio Arnaldo, one of the key data scientists at Corelight. The research is published in the June issue of the Association for Computing Machinery Special Interest Group on Knowledge Discovery and Data Mining’s peer-reviewed Explorations Newsletter.
Lessons from the real world
Functions are input variables that are entered into machine learning models; they usually come from the columns in a dataset. Data scientists select and create attributes for the model, and their main focus is to ensure that features are developed to improve the model’s accuracy, not whether a decision maker can understand them, Veeramachaneni explains.
He and his team have been working with decision makers for several years to identify the usability issues of machine learning. These domain experts, most of whom have no knowledge of machine learning, often don’t trust models because they don’t understand the features that influence predictions.
For one project, they collaborated with clinicians in a hospital ICU who used machine learning to predict the risk that a patient will develop complications after heart surgery. Some features were presented as aggregated values, such as the trend of a patient’s heart rate over time. Although functions coded in this way were “model ready” (the model could process the data), clinicians did not understand how they were calculated. They’d rather see how these aggregated features compare to the original values so they can identify abnormalities in a patient’s heart rate, Liu says.
In contrast, a group of learning scientists preferred functions that were aggregated. Instead of a feature like “number of posts a student has made on discussion forums,” they prefer to group related features and tag them with terms they understand, such as “participation.”
“With interpretability not everyone fits. As you move from area to area, there are different needs. And interpretability itself has many levels,” says Veeramachaneni.
The idea that one size doesn’t fit everyone is key to the researchers’ taxonomy. They define properties that can make functions more or less interpretable by different decision makers and outline which properties are likely to be most important to specific users.
For example, machine learning developers can focus on having features that are compatible with the model and are predictive, meaning they are expected to improve the performance of the model.
On the other hand, decision makers with no experience with machine learning may be better served by functions that are human-worded, meaning they are described in a way that is natural to users, and understandable, meaning they refer to users in the real world. world. can reason.
“The taxonomy says: if you make interpretable features, to what level are they interpretable? You may not need all the levels, depending on the type of domain experts you work with,” says Zytek.
Putting interpretation first
The researchers also outline feature engineering techniques a developer can use to make features more interpretable for a specific audience.
Feature engineering is a process by which data scientists convert data into a format that machine learning models can process, using techniques such as aggregating data or normalizing values. Most models also cannot process categorical data unless converted to a numeric code. These transformations are often nearly impossible for laymen to unpack.
Creating interpretable functions may mean undoing some of that coding, Zytek says. For example, a common feature engineering technique organizes data series so that they all contain the same number of years. To make these characteristics more understandable, one could group age groups using human terms, such as infant, toddler, child, and teen. Or instead of using a transformed function like the average heart rate, an interpretable function could simply be the actual heart rate data, Liu adds.
“In many domains, the trade-off between interpretable features and model accuracy is actually very small. For example, when we worked with child welfare screeners, we retrained the model with only features that met our definitions for interpretability, and the performance degradation was almost negligible,” says Zytek.
Building on this work, the researchers are developing a system that would allow a model developer to process complicated function transformations more efficiently to create human-centered explanations for machine learning models. This new system will also convert algorithms designed to explain model-ready datasets into formats understandable to decision-makers.
#Building #explainability #components #machine #learning #models