Tag: auditing

Categories
Sunday Book Review

Sunday Book Review: August 27, 2023 The Internal Audit Edition

In the Sunday Book Review, I consider books that would interest the compliance professional, the business executive or anyone who might be curious. It could be books about business, compliance, history, leadership, current events or anything else that might interest me. In today’s edition of the Sunday Book Review, I continue my summer exploration of books on crime. Today, look at some of the top books on auditing, both for the audit professional and the compliance professional.

Categories
Blog

Auditing AI

The recent kerfuffle over an AI tool misinterpreting instructions to make a woman look more professional as making her look Caucasian has raised important questions about how to audit AI code to avoid undesirable outcomes. AI instruments are behaving in a fundamentally different way than most other types of apps and systems, and auditing AI code for implicit bias is not yet feasible. Matt Kelly recently wrote a blog post on this topic on Radical Compliance. I thought it would make a great podcast so this week’s episode of Compliance into the Weeds is dedicated to it. I also thought it was so important that I should blog about it as well.

It started when MIT grad student Rona Wang tested an AI tool called Playground AI to modify a photo of herself wearing an MIT T-shirt to look ‘more professional’. Rather than replacing the T-shirt she was wearing with more professional business attire to achieve a more professional look, the AI tool interpreted the instruction to make her look more professional as making her look Caucasian. Wang posted a before and after comparison of her photo on Twitter, which caused a big kerfuffle in the AI world about how this happened. The CEO of Playground AI responded to Wang on Twitter saying “We’re quite displeased with this and hope to solve it”.

We began with a discussion of the implications of implicit bias in AI code. Matt suggested that the code in the AI app may have been influenced by the disproportionate number of white people on LinkedIn. It may not be the fault of the AI program, but rather a result of structural bias and racism in the world. Matt believes that at this point, it is impossible for a human to audit the code of AI programs like Chat GPT, which evaluates data according to 1.76 trillion different parameters. Unfortunately, it is not possible to eliminate implicit bias in AI code by simply correcting a few parameters. Matt compared it to the difficulty of eliminating implicit bias in AI code to the difficulty of eliminating racism in the human brain.

AI can handle 1.7 trillion parameters of data, but it is difficult to audit for an ethical outcome. AI can misinterpret structural racism and inequities that exist in the world. AI can be used to filter out images that are not representative of the population as a whole. Auditing AI is difficult because there are few people who know how to design and audit these programs. AI decisions may have life and death consequences, but there is no way to audit them yet.

Companies using AI in the hiring process must consider whether they will scrap the AI tool and use another, use human HR people and recruiters, or have auditors and coders sit down and try and figure out the problem. Additionally, there is a risk of implicit bias when someone must define the pool of data that the AI is looking at. New York City has a regulation requiring employers to audit AI tools used in the hiring process at least annually, but this is only a small step towards addressing the issue of implicit bias in AI.

Auditing AI code for implicit bias is a complex process. AI tools used in the hiring process can range from keyword matching to Chat GPT. While it is important for companies to audit their AI tools, it is also important to consider the data that is being used to train the AI. If the data is biased, the AI will be biased as well. To ensure that AI tools are not biased, companies should consider using a diverse set of data and conducting regular audits of the AI tools.

The Wang incident over an AI tool misinterpreting instructions to make a woman look more professional as making her look Caucasian is a reminder of the importance of auditing AI code to avoid undesirable outcomes. AI instruments are behaving in a fundamentally different way than most other types of apps and systems, and auditing AI code for implicit bias is not yet feasible. Companies using AI in the hiring process must consider whether they will scrap the AI tool and use another, use human HR people and recruiters, or have auditors and coders sit down and try and figure out the problem.

Finally, there is a risk of implicit bias when someone has to define the pool of data that the AI is looking at. New York City has a regulation requiring employers to audit AI tools used in the hiring process at least annually, but this is only a small step towards addressing the issue of implicit bias in AI. To ensure that AI tools are not biased, companies should consider using a diverse set of data and conducting regular audits of the AI tools.

For the complete discussion of this issue check out this week’s episode of Compliance into the Weeds.

Categories
31 Days to More Effective Compliance Programs

Day 12 of One Month to Better 3rd Party Management – Auditing of Third Parties

Auditing third parties is critical to any best practices compliance program and an important tool in operationalizing your compliance program. This is a key manner in which a company can manage the third-party relationship after the contract is signed and which the government will expect you to engage in going forward. As stated in the 2020 Update, under the section entitled, Management of Relationships, is the following query: Does the company have audit rights to analyze the books and accounts of third parties and has the company exercised those rights in the past? This means you must not only have audit rights but also exercise them.

 Three key takeaways:

1. Be prepared.

2. It is not an investigative interview but an audit interview.

3. Listen, listen, and listen.

Categories
Corruption, Crime and Compliance

Episode 241 – Continuous Improvement, Testing and Auditing of Your Ethics and Compliance Program

The Justice Department and various regulatory agencies continue to emphasize the importance of continuous improvement, testing and review as part of robust assessment procedures in an effective compliance program. The Treasury Department’s Office of Foreign Asset Control has specifically stated that a sanctions compliance program should include “a comprehensive, independent, and objective testing or audit function” so that a company can determine “how their program[] [is] performing and should be updated, enhanced, or recalibrated to account for a changing risk assessment or sanctions environment.” The Health and Human Services — Office of Inspector General has made similar statements underscoring the need to conduct compliance audits and testing. An important part of every compliance program focuses beyond the design and operation of the program to the important issue of whether the program is working. In this respect, DOJ and regulatory agencies have noted that CCOs should be striving to develop “continuous” monitoring systems and avoid “snapshots” in time. In order to execute such monitoring, compliance has to maintain broad access to operational data across all key functions in a company. This data must be used to regularly update risk assessments, compliance policies and procedures and financial controls.

In this episode, Michael Volkov takes a broad review of the testing and auditing of ethics and compliance programs.

Categories
31 Days to More Effective Compliance Programs

Auditing of third-parties


Auditing of third-parties is critical to any best practices compliance program and an important tool in operationalizing your compliance program. This is a key manner in which a company can manage the third-party relationship after the contract is signed and one which the government will expect you to engage in going forward. As stated in the 2020 Update, under the section entitled, Management of Relationships, is the following query, Does the company have audit rights to analyze the books and accounts of third parties, and has the company exercised those rights in the past? This means you must not only have audit rights but also exercise them.
You should plan out the audit four to six weeks in advance, you should perform the audit with your legal counsel’s lead to preserve privilege, work with the Relationship Manager to establish key business contacts, discuss audit rights and processes with the third-party, you should prepare initial document request lists for financial information queries, take the time to review findings from previous audits and resolutions and also review details of opened and closed internal investigations, if there are any Code of Conduct questionnaires available take care to review and, finally, be cognizant of any related DOJ and SEC enforcement actions.
Three key takeaways:

  1. Be prepared.
  2. It is not an investigative interview but an audit interview.
  3. Listen, listen, and listen.
Categories
31 Days to More Effective Compliance Programs

Auditing of third-parties


Third-parties still present the highest risk around compliance. Indeed, in the area of third-parties the 2019 Guidance, posed the following question in a section entitled, Management of Relationships – How has the company considered and analyzed the compensation and incentive structures for third parties against compliance risks? How does the company monitor its third parties? Does the company have audit rights to analyze the books and accounts of third parties, and has the company exercised those rights in the past? 
It is therefore critical that you use monitoring and auditing when it comes to continuous improvement for this high-risk area. Next, we consider three aspects of a company’s audit program for its compliance function: the types and purpose of third-party audits, planning for third-party audits and interviewing third-parties.
Three key takeaways: 

  1. Start planning your third-party audit 4-6 weeks in advance of the actual audit.
  2. Use your business sponsor to help facilitate the process with the third-party.
  3. This is not a “gotcha” interview but an open Q&A process where you have a golden opportunity to educate as you ask questions.