Businessman Pointing Red Diminishing Arrow Over The Risk Text On Reflective Desk
This year, despite the challenges from the Covid-19 pandemic, large corporations in the financial industry are operationalizing their AI initiatives. Many mature organizations already have established processes. In the last few years, they've been implementing process workflows, software tools, and frameworks to quickly operationalize their models to capitalize on the changing business landscape.
However, as the business environment changed during the Covid-19 pandemic, organizations observed changes in their models' underlying assumptions. The urgency to rapidly deploy new models in a controlled environment to account for the market risks and take advantage of new opportunities proved to be challenging.
From conversations gathered from two notable roundtables from ModelOp and QuantUniversity with industry veterans from Wells Fargo, BP, Regions Bank, Vector Institute, ModelOp, and others, I’ve put together this list of key issues to consider when operationalizing AI at the enterprise level.
Rethinking Checks And Balances
In the wake of the Great Recession of 2008, banks put robust controls into place to ensure that data scientists, program managers, and corporate risk and corporate technology teams cooperate to maintain compliance. However, recent history has shown that AI models may change frequently. At the enterprise level, this calls for an even more intensive approach to governance.
Shrikant Dash, Strategic Advisor to ModelOp, says, "What I think needs to happen given the proliferation of data; given that there's a move towards more real-time digital channels; and given that there's also a proliferation of new AI and ML techniques which are more complex is that you have to strengthen those checks and balances. In addition to that, the first, second, and third lines of defense also need to have a collaborative focus with these emerging technologies. You have to understand how technology and how the architecture of modern production is changing."
Focus on the Connectivity Between Model Development and Model Operations
Until recently, enterprises have focused much of their effort on model development, especially as business units looked to exploit advances in AI technology. The ever richer pools of data and algorithms can translate to real business value. When the business units need to deploy their production models, they then turn to corporate IT teams. By that time, they realize there's a vast gap between development and production environments. This gap can often result in 9 to 18-month lags between model creation and model deployment. Unfortunately, during that time, many models can become obsolete. Long delays between development and production mean that these models miss the window of opportunity when they can yield the most business benefits. In the worst scenario, failure to operationalize these models can undermine the entire project.
Dash says, "So this connectivity between Model Development and Model Operations will be very, very important because you can build the best possible models in a development environment. However, if you're not putting them in production seamlessly without errors and you're not able to validate and monitor them as necessary, then you have a problem."
Create New Roles and Align the Organization for Enterprise AI
Operationalizing AI requires coordination across multiple stakeholders, none of whom typically have been responsible for the end-to-end process from development to deployments to governance. Interdepartmental roles that help to bridge the gap in strategy, operations, governance, and engineering are critical to ensure the success of Enterprise AI. In this new paradigm, the Enterprise AI Architect and the Model Operator are two new roles that have been created to fill the gap in operationalizing AI at the enterprise level.
Dash says, "I think it's imperative to have these kinds of roles... I think that's an essential criterion for fully leveraging the emerging technologies… How should a CIO be thinking about either hiring people with the relevant skill sets or training people within the technology organization to focus on these kinds of investments?"
Anticipate Shorter Model Life Cycles
The pace of AI/ML technology development is accelerating, and new techniques will continue to uncover new business opportunities. There's a component of automated decision making. Simultaneously, recent events like the pandemic show how ground-truth underlying data sets can change rapidly under real-world conditions. These factors point to greater rates of changes in models and their life cycles. The impacts of shorter life cycles are further exacerbated by sharing and re-using of model components. This heightens the need for robust infrastructure and tools to automate deployment while ensuring performance and risk management standards are met.
Dash says, “If you have all those emerging new classes of models and you don't have an architecture and infrastructure to integrate that into production, and monitor them in production and have shorter cycle times, then, what have you got you've got hype, but you haven't got real value to the enterprise. So in order for the promise of these newer AI and machine learning models to deliver enterprise value, it's essential to have an architecture and infrastructure like ModelOp Center or similar technologies that people might choose in this area fully leverage these new AI/ML techniques.”
Governance is Key: Know and Anticipate the Limits of Models and Data
Historically in the financial industry, the model risk group, a part of the corporate risk organization, is the one that’s responsible for designing testing, validation, deployment of models. Those focused on model risk are increasingly thinking about how to understand model limitations. In production, it’s critical to set up performance indicators and monitor how the model can run in production. Understanding how AI/ML models run on production data is a crucial part of risk mitigation.
Agus Sudjianto, EVP, Head of Corporate Model Risk, Wells Fargo, says, “We look at it from when the model is designed, and before model deployment, we apply very rigorous discipline in terms of model deployment, model testing, and validation to look at when the model can be wrong, and the un-intended consequences of the models. That can come from the data. So, we talk about the soundness of the framework, the limitations of the model. We know that every model has a limit. How are we going to monitor it? How are we going to design the key performance indicators that we are going to monitor? What are the mitigations and controls that we can use during model deployment?”
Consider How Model Re-use and Sharing Can Have Impacts Across the Enterprise
Information technology professionals are well-versed with the development life cycle and change management of software. However, models are different than software. Each model has a distinct life cycle. Depending on the changes in real-world data, models can have short or long lifespans. Understanding the modularity and the shareability of specific parts of the model can help teams work out dependencies related to change management and risk mitigation. This means that business units also need to understand the criticality of certain types of changes that will affect other parts of the enterprise when many parts of the enterprise share models and create new models from shared model components.
Cetin Karakus, Global Head of Quantitative & Analytical Solutions, BP, says, “When we talk about change management, we need to talk about the model life cycle, to version them. More importantly, you need to have the infrastructure where you make the transition. You need to have a controlled rollout. You don’t change the production version immediately. You need to have the next version. Another thing is reusability and modularity. You usually build more complex models out of the more basic ones. That’s the key to ensure that you have a single consistent modeling approach. These are building blocks. Your assumptions will then propagate across the enterprise.”
Move to Automated Testing and Integrate it into Model Lifecycles
One of the bottlenecks for model deployment has been testing. Larger banks have automated their testing. However, smaller banks are still in the process of moving to automated testing. As the model inventory grows from thousands of models to tens of thousands of models, automated testing will become the norm.
Jacob Kosoff, Head of Model Risk, Regions Bank, says, “For us, we are trying to keep up with this massively growing inventory. Our models have doubled in the last five years, 45% in the last two years. One of the areas we’ve been leaning on has been automated testing. That’s really been helping us. We can embed the testing inside the model development life cycle. Our validation and model testing has been automatically built throughout the model development process.”
Take a Broader View of Risk and Adjust Governance Accordingly
Since Enterprise AI initiatives classify existing and new AI models to be shared and re-used as enterprise assets, higher stakes are being placed on these initiatives. With increased visibility and business impact of these models, the magnitude of unintended consequences can be amplified across the organization. The amplification adds another layer of risk. When everyone has the mindset of governing for performance in the financial industry, raising the breadth and scope of governance may be called for.
Ron Bodkin, VP AI Engineering & CIO, Vector Institute, says, “I think we have to expand our horizon a bit. Simply governing against performance is not enough. Suddenly, organizations are realizing bias can be devastating for reputational risk. Increasingly, if you don’t understand the unintended consequences, it may be higher stakes. Governance to understand across multiple metrics, what might be going wrong, not just performing well, becomes important. We see models can have impact on privacy, or longer-term interest, and can have interesting feedback loops. Thinking about a broader definition of risk is an important emerging discipline. And, thinking about how to get standardization? Thinking about governance to allow effective collaboration while respecting privacy. That can include allowing collaboration to attack financial crime. How to allow consortia of financial institutions to work together?”
Ethical AI/ML Implementations Require Rigorousness and Automation
As ethical AI and responsible implementations become the main topic of conversation in the next few years, companies are well aware of the reputational risk at stake. Regulatory compliance is serious business. Just having a Model Risk Management is not enough. The department works in conjunction with enterprise wide AI assets to mitigate and minimize risks even as the business environment change rapidly. Rigorous governance of models require automated process flows that proactively adds transparency, mitigates biases and minimizes volatility.
When process flows are automated, management can spend more time thinking through unintended consequences and gradually improve the ethical climate.
Dash says, "An important emerging issue is the question of explainability and transparency as well as ethical bias in more complex AI/ML algorithms. The regulatory frameworks for model governance require more robust documentation, validation and governance of these new models in order to mitigate outcome bias and volatility. An enterprise class solution for model governance and tracking that is more automated will minimize errors and bias in new generation AI/ML models."
