Tag: KPIs

Categories
Blog

Revolutionizing Compliance with AI-Powered KPIs 

In the modern corporate landscape, traditional key performance indicators (KPIs) are struggling to meet the demands of dynamic compliance environments. These legacy metrics often fail to align operations, prioritize resources, and drive accountability toward strategic objectives. For compliance professionals, these shortcomings are particularly critical: ineffective KPIs can lead to missed risks, inefficient processes, and poor decision-making, ultimately jeopardizing organizational integrity.

In a recent article in the Sloan Management Review, entitled The Future of Strategic Measurement: Enhancing KPIs With AI, authors Michael Schrage, David Kiron, François Candelon, Shervin Khodabandeh, and Michael Chu explored these and other issues, which I have adapted for the compliance professional.  By incorporating artificial intelligence (AI), organizations are reimagining what KPIs can accomplish—not just as performance trackers but as drivers of strategic differentiation and value creation.

The Shortcomings of Legacy KPIs in Compliance

Legacy KPIs often focus narrowly on outputs, such as the number of training sessions conducted or hotline calls logged. While these metrics provide valuable data, they frequently fail to provide solid information in various ways. The first is that legacy KPIs are taken in a vacuum with no appreciation of the interconnected nature of corporate risks. Just as compliance does not (or at least should not) operate in a vacuum, risks in one area often cascade into others, yet traditional KPIs rarely reflect these interdependencies. The retrospective nature of KPIs. Metrics rooted in historical data are inherently backward-looking, limiting their utility for forecasting and proactive risk management.

Finally, corporate silos, which are a perennial challenge in compliance, and static KPIs can reinforce them rather than foster cross-functional collaboration. Legacy KPIs do not promote alignment across disparate corporate functions. These limitations hinder a compliance professional’s ability to effectively anticipate, prevent, and address misconduct.

Enter Smart KPIs: A New Era of Compliance Metrics

AI-powered KPIs offer a smarter, more dynamic approach to performance measurement. These metrics are descriptive, predictive, and prescriptive. Such metrics will allow a corporate compliance function to provide new and different insights, such as some of the following.

  • Analyze past and current compliance performance to identify gaps.
  • Anticipate future risks and compliance trends based on patterns in data.
  • Recommend actions to mitigate risks and optimize outcomes.

For example, AI can transform a traditional metric like the “number of third-party audits conducted” into a prescriptive KPI that evaluates audit results, predicts the highest risk areas, and recommends corrective actions.

Case Study: Wayfair and the Evolution of Lost-Sales KPIs

The article discussed Wayfair’s reengineering of its lost-sales KPI and offers valuable insights for compliance professionals. Initially, the retailer calculated lost sales on an item-by-item basis, but AI analysis revealed that many “lost” sales were category retentions, as customers purchased alternative items. This revelation led Wayfair to redesign its KPI to measure category-based retention. The result? Smarter metrics aligned product placement with operational constraints, improving customer satisfaction and operational efficiency.

This case study provides a clear set of lessons for corporate compliance and the compliance professional. Compliance teams can use AI to rethink KPIs that do not fully capture performance nuances. For instance, instead of merely tracking the number of training completions, a smarter KPI could evaluate behavioral changes post-training or identify employees most at risk of ethical lapses based on historical data. This, in turn, could provide greater insight into training effectiveness and how a compliance professional might think about targeted training.

KPI Governance: A Compliance Imperative 

One of the most critical aspects of AI-enhanced KPIs is governance. Organizations need robust governance mechanisms to ensure KPIs evolve with strategic objectives and maintain their relevance over time. For a compliance professional, this means several different approaches.

  1. Continuous Review of Metrics. Regularly revisiting KPIs to ensure they remain aligned with evolving regulatory landscapes and business priorities.
  2. Meta-KPIs for Quality Assurance. Developing “KPIs for KPIs” to assess their accuracy, relevance, and effectiveness.
  3. Cross-Functional Oversight. Establishing governance structures that bring together compliance, legal, and operational teams to oversee metric design and implementation.

The bottom line is that accountability for KPI performance, both the metrics themselves and the outcomes they drive, must be embedded into the compliance framework.

How AI Enhances Compliance KPIs

AI-enhanced KPIs bring new capabilities to compliance programs in three key manners. First, in risk anticipation. Predictive KPIs can identify emerging compliance risks, such as regulatory changes, third-party risk management, or shifts in employee behavior, enabling proactive mitigation. The second area is holistic insights. By analyzing data across functions, AI can uncover hidden correlations, such as how employee hotline reports, visits to the compliance department website, or even the number of requests to FAQs might signal compliance risks in supply chain operations. Finally is the area of targeted recommendations. Prescriptive KPIs can suggest specific actions, like prioritizing high-risk vendors for audits or tailoring training to address observed knowledge gaps. For example, AI could analyze whistleblower reports alongside financial data to identify patterns indicative of systemic fraud, providing actionable insights for remediation. 

 This more holistic approach also addresses one of the key risk areas around KPIs: stagnate KPIs. The 2008 financial crisis underscores the dangers of relying on outdated KPIs. Banks’ dependence on “value at risk” metrics, which failed to account for the growing influence of subprime mortgages, contributed to catastrophic losses. Compliance professionals must guard against similar pitfalls by regularly challenging assumptions underpinning legacy KPIs. AI can aid in this process by continuously analyzing data to reveal when a metric is no longer fit for purpose.

Steps to Implement Smarter Compliance KPIs

Compliance professionals can take the following steps to transition from legacy to AI-enhanced KPIs.

  1. Audit Existing KPIs. Assess whether current metrics adequately capture compliance risks and align with strategic objectives.
  2. Leverage AI for Data Analysis. Use AI tools to uncover hidden patterns in compliance data, such as correlations between employee turnover and ethics violations.
  3. Collaborate Across Functions. Work with IT, legal, and operations teams to ensure KPI redesigns reflect organizational priorities.
  4. Invest in Training and Culture. Equip compliance teams with the skills to interpret and act on AI-generated insights while fostering a culture of data-driven decision-making.
  5. Monitor and Improve KPIs. Establish processes for ongoing KPI evaluation, ensuring they evolve alongside regulatory and stakeholder input and business changes.

Challenges and Ethical Considerations 

While AI-enhanced KPIs offer immense potential, they also present challenges. These challenges include some of the following. Just as with more generative AI, algorithms can be biased. AI models are only as unbiased as the data on which they are trained. Compliance teams must ensure that their AI systems uphold principles of fairness and equity. Always remember the Human in the Loop to preclude over-reliance on AI. While AI can inform decision-making, it should not replace human judgment. Compliance professionals must strike a balance between algorithmic insights and ethical considerations. Finally, there are data privacy concerns. Collecting and analyzing large datasets for KPI development must comply with data privacy regulations.  

Conclusion: The Future of Compliance Metrics 

The rise of AI-enhanced KPIs marks a paradigm shift in measuring and managing compliance performance. By embracing smarter, more dynamic metrics, compliance professionals can gain deeper insights, anticipate risks, and drive better outcomes.  Much like Wayfair and other forward-thinking organizations, compliance teams must be willing to challenge the status quo, leverage technology, and prioritize continuous improvement. The era of static, backward-looking KPIs is over. In its place is a future where smart KPIs enable compliance functions to not only measure performance but actively enhance it—turning compliance from a cost center into a source of strategic value. The question is not whether your organization should adopt AI-powered KPIs but how soon your compliance program can reap the benefits. The time to act is now.

Categories
Blog

AI in Compliance Week: Part 5 – Continuous Monitoring of AI

This blog post concludes a five-part series I ran this week on some of the keys intersecting AI and compliance. Yesterday, I wrote that businesses must proactively address the potential for bias at every stage of the AI lifecycle—from data collection and model development to deployment and ongoing monitoring. In this final blog post, I deeply dive into continuously monitoring your AI. We begin this final Part 5 with some key challenges organizations must navigate to accomplish this task.

As we noted yesterday, data availability and high data quality are essential. Garbage In, Garbage Out. Robust bias monitoring requires access to comprehensive, high-quality data that accurately reflects the real-world performance of your AI system. Acquiring and maintaining such datasets can be resource-intensive, especially as the scale and complexity of the AI system grow. However, this is precisely what the Department of Justice (DOJ) expects from a corporate compliance function.

How have you determined your key performance indicators (KPIs) and interpretation? Selecting the appropriate fairness metrics to track and interpret the results can be complex. Different KPIs may capture various aspects of bias, and tradeoffs between them can exist. Determining the proper thresholds and interpreting the significance of observed disparities requires deep expertise.

Has your AI engaged in Model Drift or Concept Shift? Compliance professionals are aware of the dreaded ‘mission creep. AI models can exhibit “drift” over time, where their performance and behavior gradually diverge from the original design and training. Additionally, the underlying data distributions and real-world conditions can change, leading to a “concept shift” that renders the AI’s outputs less reliable. Continuously monitoring these issues and making timely adjustments is critical but challenging. Companies will need to establish clear decision-making frameworks and processes to address model drift and concept shift.

Operational complexity is a critical issue in continuous AI monitoring. Integrating continuous bias monitoring and mitigation into the AI system’s operational lifecycle can be logistically complex. This requires coordinating data collection, model retraining, and deployment across multiple teams and systems while ensuring minimal service disruptions.

Everyone must buy in or in business-speak – Organizational Alignment must be in place.  Not surprisingly, it all starts with the tone at the top. Your organization should foster a responsible AI development and deployment culture with solid organizational alignment and leadership commitment. Maintaining a sustained focus on bias monitoring and mitigation requires buy-in and alignment across the organization, from executive leadership to individual contributors. Overcoming organizational silos, competing priorities, and resistance to change can be significant hurdles.

There will be evolving regulations and standards. The regulatory landscape governing the responsible use of AI is rapidly growing, with new laws and industry guidelines emerging. Keeping pace with these changes and adapting internal processes can be an ongoing challenge. Staying informed about evolving regulations and industry standards and adapting internal processes will be mission-critical.

The concept of AI explainability and interpretability will be critical going forward.  As AI systems become more complex, providing clear, explainable rationales for their decisions and observed biases becomes increasingly crucial. Enhancing the interpretability of these systems is essential for effective bias monitoring and mitigation. This will be improved by developing robust data management practices to ensure the availability and quality of data for bias monitoring. The bottom line is that companies should prioritize research and development to improve the explainability and interpretability of AI systems, enabling more effective bias monitoring and mitigation.

A financial commitment will be required, as continuous bias monitoring and adjustment can be resource-intensive. It requires dedicated personnel, infrastructure, and budget allocations and investing in specialized expertise, both in-house and through external partnerships, to enhance the selection and interpretation of fairness metrics. Organizations must balance these needs against other business priorities and operational constraints. Companies must allocate the necessary resources, including dedicated personnel, infrastructure, and budget, to sustain continuous bias monitoring and adjustment efforts.

Organizations should adopt a comprehensive, well-resourced approach to AI governance and bias management to overcome these challenges. This includes developing robust data management practices, investing in specialized expertise, establishing clear decision-making frameworks, and fostering a responsible AI development and deployment culture.

Continuous monitoring and adjusting AI systems for bias is a complex, ongoing endeavor, but it is critical to ensure these powerful technologies’ ethical and equitable use. By proactively addressing the challenges, organizations can unlock AI’s full potential while upholding their commitment to fairness and non-discrimination.

By proactively addressing these challenges, organizations can unlock AI’s full potential while upholding their commitment to fairness and non-discrimination. Continuous monitoring and adjusting AI systems for bias is a complex, ongoing endeavor, but it is a critical component of responsible AI development and deployment.

As the AI landscape continues to evolve, organizations prioritizing this crucial task will be well-positioned to navigate the ethical and regulatory landscape, build trust with their stakeholders, and drive sustainable innovation that benefits society.

Categories
Compliance Tip of the Day

Compliance Tip of the Day: Metrics for Compliance

Welcome to “Compliance Tip of the Day,” the podcast where we bring you daily insights and practical advice on navigating the ever-evolving landscape of compliance and regulatory requirements.

Whether you’re a seasoned compliance professional or just starting your journey, our aim is to provide you with bite-sized, actionable tips to help you stay on top of your compliance game.

Join us as we explore the latest industry trends, share best practices, and demystify complex compliance issues to keep your organization on the right side of the law. Tune in daily for your dose of compliance wisdom, and let’s make compliance a little less daunting, one tip at a time.

In this episode, we explore why every compliance program needs metrics to measure their program and how to create Key Performance Indicators (KPIs) for compliance.

For more information on Ethico and a free White Paper on top compliance issues in 2024, click here.

Categories
Compliance Tip of the Day

Compliance Tip of the Day: Compliance Innovation Through KPIs

Welcome to “Compliance Tip of the Day,” the podcast where we bring you daily insights and practical advice on navigating the ever-evolving landscape of compliance and regulatory requirements.

Whether you’re a seasoned compliance professional or just starting your journey, our aim is to provide you with bite-sized, actionable tips to help you stay on top of your compliance game.

Join us as we explore the latest industry trends, share best practices, and demystify complex compliance issues to keep your organization on the right side of the law. Tune in daily for your dose of compliance wisdom, and let’s make compliance a little less daunting, one tip at a time.

In this episode, we consider innovation in compliance through Key Performance Indicators (KPIs).

For more information on Ethico and a free White Paper on top compliance issues in 2024, click here.

Categories
31 Days to More Effective Compliance Programs

One Month to a More Effective Compliance Program Through Innovation: Day 11 – Compliance Innovation Through KPIs

Measuring your compliance program’s effectiveness will be a critical criterion going forward. One of the mechanisms to do so is through Key Performance Indicators (KPIs). If you have been working towards your stated goals and reporting success, KPIs are critical in showing compliance program success or failure. And while specific requirements for this kind of reporting have been hotly debated in the industry for some time, KPIs are a regulatory requirement. Your KPIs will be specific and unique to your company and its business. Couple this with what goals you are trying to achieve as a whole as a compliance program, and you will see there is no set list of these metrics.

KPIs provide yet another mechanism for you to monitor and update your compliance program almost continuously. KPIs can be extremely low in cost and, therefore, something you can put in place without much approval from higher-ups in your organization that you might have to go to for budget approval. Finally, innovation can come in many ways. ComTech can be a huge jump forward. But sometimes innovation can occur at much less cost and a much more granular level. KPIs can be such a mechanism for you.

Three key takeaways:

  1. KPIs will be critical to assess a compliance program going forward.
  2. Set your KPIs.
  3. Decide on how to use KPIs and the blueprint for going forward.

For more information, check out The Compliance Handbook, 4th edition, here.

Categories
Greetings and Felicitations

Great Structures Week IV: The Gothic Cathedral and Compliance Incentives

Welcome to Greetings and Felicitations, a podcast where I explore topics that might not seem directly related to compliance but clearly influence our profession. In this special series, I consider many structural engineering concepts are apt descriptors for an anti-corruption compliance program. In this episode 4, I consider the Gothic Cathedral and incentives in your compliance program. Highlights include:

·      Why and how was the Gothic Cathedral such an engineering innovation?

·      What are the key principals for an incentive program?

·      How do incentives impact your compliance program?

·      What does the DOJ say about incentives?

·      What KPIs can you use to measure compliance incentives?

Resources

“Understanding the World’s Greatest Structures: Science and Innovation from Antiquity to Modernity,” taught by Professor Stephen Ressler from The Teaching Company.

Categories
Great Women in Compliance

Deb Barrett – On Top of Her Game

Welcome to the Great Women in Compliance Podcast, co-hosted by Lisa Fine and Mary Shirley.

Deb Barrett is Chief Compliance Officer of Qualcomm.  She shares some insights of what it was like being in a company that has undergone some regulatory scrutiny.  She and Mary Shirley discuss some ways to combat Compliance fatigue – important for any company with a robust Compliance program to consider but particularly ones that have prioritized Compliance initiatives for a period of years.  The episode is rich with takeaways and ideas, including Deb’s thoughts on Compliance KPIs.

 Are you planning on heading to the SCCE CEI in Phoenix in October?  Check out Lisa and Mary’s speaking sessions on the agenda and sign up!  We invite you to say hello and introduce yourself during the conference – it’s going to be a great time.

 The Great Women in Compliance Podcast is on the Compliance Podcast Network with a selection of other Compliance related offerings to listen in to.  If you are enjoying this episode, please rate it on your preferred podcast player to help other likeminded Ethics and Compliance professionals find it.  If you have a moment to leave a review at the same time, Mary and Lisa would be so grateful.  You can also find the GWIC podcast on Corporate Compliance Insights where Lisa and Mary have a landing page with additional information about them and the story of the podcast.  Corporate Compliance Insights is a much appreciated sponsor and supporter of GWIC, including affiliate organization CCI Press publishing the related book; “Sending the Elevator Back Down, What We’ve Learned from Great Women in Compliance” (CCI Press, 2020). If you enjoyed the book, the GWIC team would be very grateful if you would consider rating it on Goodreads and Amazon and leaving a short review.

You can subscribe to the Great Women in Compliance podcast on any podcast player by searching for it and we welcome new subscribers to our podcast.

Join the Great Women in Compliance community on LinkedIn here.

Categories
31 Days to More Effective Compliance Programs

Strategies For and With AI in Compliance


Today, I want to consider the article Strategy For and With AI by David Kiron and Michael Schrage. The authors premise is, “A company’s strategy is defined by its key performance indicators. Artificial intelligence can help determine which outcomes to measure, how to measure them, and how to prioritize them.”
Their article had several insights for the Chief Compliance Officer (CCO) or compliance practitioner who is looking to employ Artificial intelligence (AI) to help move their compliance program up a level. One of the first key insights is that it is not enough to simply have a strategy for AI. The authors stated, “Creating strategy with AI matters as much — or even more — in terms of exploring and exploiting strategic opportunity. This distinction is not semantic gamesmanship; it’s at the core of how algorithmic innovation truly works in organizations. Real-world success requires making these strategies both complementary and interdependent. Strategies for novel capabilities demand different managerial skills and emphases than strategies with them.”
This makes clear that AI does not supplant the compliance function or the compliance professional, AI complements what the compliance professional can do with the information available to them. Yet the authors believe that when it comes to machine learning, an appropriate compliance strategy is defined by the key performance indicators (KPIs) leaders choose to optimize. This means that a CCO who cannot clearly identify and justify their strategic KPI portfolios has no strategy.
The bottom line? AI plays a critical role in determining what and how compliance KPIs are measured and how best to optimize them. Optimizing carefully selected compliance KPIs becomes AI’s strategic purpose in the compliance function. Understanding the value of optimization is key to aligning and integrating strategies forand with AI and machine learning. KPIs create accountability for optimizing strategic aspirations, including compliance.
Three key takeaways:

  1. Use KPIs to define and measure your innovation strategy.
  2. AI should only supplement, not supplant a compliance professional.
  3. What are your compliance KPIs?

For more information on how an independent monitor can help improve your company’s ethics and compliance program, visit this month’s sponsor Affiliated Monitors at www.affiliatedmonitors.com.