The boundless benefits of artificial intelligence and machine learning are quickly traversing the globe, empowering every industry and business. Even the banking industry incorporates machine learning in order to build models that enhance decision making and help with risk management.
But of course, with boundless benefits come ample risks. Even though you may have validation frameworks and practices that aim to uncover and eliminate risks, the complexity of artificial intelligence models still amplifies some elements of model risks.
Due to this, various companies then restrict the use of machine learning algorithm models, designating them comparatively low-risk applications like digital marketing.
While their caution is indeed justified; however, a better approach is to enhance model-risk management. Not only would this approach be more sustainable, but it would allow you to not restrict your company from the benefits that artificial intelligence and machine learning have to offer.
How to Enhance Model-risk Management?
Primarily, what we are referring to as ‘enhancing’ includes some necessary policy changes. The altered policy decisions need to address what to include in a model inventory, along with determining the risk appetite and the associated model-validation practices.
However, the good news is that you don’t have to construct entirely new model-validation frameworks. Instead, all you need to do is enhance the existing ones to fit this purpose.
New Risks- New Practices!
Machine learning is no stranger to blame and finger-pointing. There have been various instances where the machine-learning models have had unintended consequences. For instance, when a self-driving car tragically failed to identify a pedestrian walking her bicycle across the street.
The cause of the risks that materialized in these machine-learning models is the same as the cause of the amplified risks that exist in all machine-learning models, whatever the industry and application. And that is increased model complexity.
Machine-learning models typically act on vastly larger data sets, including unstructured data such as natural language, images, and speech. The algorithms are typically far more complex than their statistical counterparts and often require design decisions to be made before the training process begins.
The response to such complexity does not have to be overly complex, however. If properly understood, the risks associated with machine-learning models can be managed within existing model-validation frameworks.
Incorporating the Salient Features
Listed below are the features that will represent the most substantive changes to the framework.
1.Interpretability
Machine learning models are widely reputed as being complex with high complicated intrinsic workings. This reputation has indeed been earned because depending on the model’s architecture, the results it generates can be hard to understand or explain.
Therefore, in certain situations, it can actually prove detrimental to act upon the model’s vague recommendations instead of just ignoring them.
The degree of interpretability required for your company is actually based upon your risk appetite. You can choose whether to have the same interpretability standards for all your machine learning models, or you can differentiate according to the model’s risks.
For example, a model’s recommendation to place a product advertisement on a consumer’s mobile app poses such minimal risk that wasting your resources on trying to understand the model’s reasons for doing so is entirely unnecessary.
Validators need also to ensure that models comply with the chosen policy. Fortunately, despite the black-box reputation of machine-learning models, significant progress has been made in recent years to help ensure their results are interpretable. A range of approaches can be used, based on the model class:
- Linear and monotonic models: linear coefficients help reveal the dependence of a result on the output.
- Nonlinear and monotonic models: restricting inputs so they have either a rising or falling relationship globally with the dependent variable simplifies the attribution of inputs to a prediction.
- Nonlinear and nonmonotonic models: methodologies such as local interpretable model-agnostic explanations or Shapley values help ensure local interpretability.
2.Bias
A model can be influenced by four main types of bias: sample, measurement, and algorithm bias, and bias against groups or classes of people. The latter two types, algorithmic bias and bias against people, can be amplified in machine-learning models.
To address algorithmic bias, model-validation processes should be updated to ensure appropriate algorithms are selected in any given context. In some cases, there are technical solutions. Another approach is to develop “challenger” models, using alternative algorithms to benchmark performance.
In order to address the bias against groups or classes of people, you must first decide what constitutes fairness. There are four primary definitions that are commonly used:
- Demographic Blindness: decisions are made using a limited set of features that are highly uncorrelated with protected classes, that is, groups of people protected by policies.
- Demographic Parity: outcomes are proportionally equal for all protected classes.
- Equal Opportunity: true-positive rates are equal for each protected class.
- Equal Odds: true-positive and false-positive rates are equal for each protected class.
After defining fairness, validators then need to make sure that developers have taken the necessary steps to ensure it.
It is also important to keep checking your models for fairness, and if necessary, correcting them at each stage of the model development process, from design to performance monitoring.
3. Feature Engineering
Feature engineering is often much more complex in the development of machine-learning models than in traditional models. There are three reasons why.
- First, machine-learning models can incorporate a significantly larger number of inputs.
- Second, unstructured data sources such as natural language require feature engineering as a preprocessing step before the training process can begin.
- Third, increasing numbers of commercial machine-learning packages now offer so-called AutoML, which generates large numbers of complex features to test many transformations of the data.
Models produced using these features run the risk of being unnecessarily complex, contributing to overfitting.
When it comes to feature engineering, you have to make a policy decision to mitigate the risk. This can be done by determining the level of support required to establish the conceptual soundness of each feature. You can choose to alter the policy according to the application of the model.
4. Hyper-parameters
The values of most parameters od machine learning models cannot be derived from available data and therefore, need to be defined before the the training process can begin.
Rules of thumb, parameters used to solve other problems, or even trial and error are common substitutes. Decisions regarding these kinds of parameters, known as hyperparameters, are often more complex than analogous decisions in statistical modeling. Not surprisingly, a model’s performance and its stability can be sensitive to the hyperparameters selected.
Validators should ensure that hyperparameters are chosen as soundly as possible. For some quantitative inputs, as opposed to qualitative inputs, a search algorithm can be used to map the parameter space and identify optimal ranges. In other cases, the best approach to selecting hyperparameters is to combine expert judgment and, where possible, the latest industry practices.
5.Production Readiness
Traditional models are often coded as rules in production systems. Machine-learning models, however, are algorithmic, and therefore require more computation. This requirement is commonly overlooked in the model-development process. Developers build complex predictive models only to discover that your company’s production systems cannot support them.
It is imperative that, for machine learning, you expand the scope of assessment. Doing so requires you to estimate the volume of data that will flow through the model, assess the production-system architecture, and the required runtime.
Concluding Thoughts:
The primary step is to ensure that your model inventory includes all machine learning-based models in use. The process of de-risking them will be gradual but effective.
Artificial Intelligence and Machine Learning are extremely powerful entities that if, used optimally, can lead you to unprecedented growth and profits.
Lucrative is an autonomous AI that incorporates machine learning algorithms to optimize businesses.
If you have any concerns, feel free to contact us.
[hubspot portal=”6539536″ id=”0216077c-fa40-4ee1-b68e-2ed513ba8e02″ type=”form”]
