Auditing how a business uses AI

A business can pick up AI tools in many corners of its work. Sooner or later a client, a board or an insurer may ask for an account of that use. This page sets out what an audit of AI use can cover. It draws on guidance from the UK government and the Information Commissioner's Office (ICO).

Three steps the government describes

The Department for Science, Innovation and Technology (DSIT) published an introduction to AI assurance in February 2024. It describes assurance in three steps. First, people measure how a system works, gathering data on its performance and its effects. Then they evaluate the risks and impacts against agreed benchmarks, such as standards and regulators' guidance. Finally, they communicate what they found, inside the business and sometimes to the public.

DSIT notes that the people doing the measuring may need the design documents for the system. A business that bought its AI from a supplier may have to ask for them.

Kinds of check

DSIT's guide names several techniques. An audit can use one or several of them.

• A risk assessment looks for risks such as bias, privacy problems and harm to the business's reputation.
• An impact assessment looks at wider effects, for example on equality, human rights or the environment.
• A bias audit checks the data going in and the decisions coming out for unfair bias.
• A compliance audit checks the system against the business's own policies and the law.
• Performance testing and conformity assessment check whether a system meets set requirements.

Personal data and the ICO

The ICO says that in the vast majority of cases, using AI will involve processing that is likely to be high risk to people. That triggers a legal duty to carry out a data protection impact assessment (DPIA). If the assessment shows a high risk the business cannot reduce, it must consult the ICO before it starts.

The ICO also says senior managers cannot hand these questions over to data scientists or engineers. The business as a whole stays accountable. The ICO notes that its AI guidance is under review after changes made by the Data (Use and Access) Act.

A yardstick for the findings

The government's 2023 white paper on AI regulation set out five principles for every sector. An auditor can test each system against them.

• Safety, security and robustness.
• Appropriate transparency and explainability.
• Fairness.
• Accountability and governance.
• Contestability and redress, so that people affected can challenge a harmful decision.

Standards and certification

Some businesses go further and work to a published standard. DSIT's guide points to ISO/IEC 42001, an international standard for an AI management system. It also explains that UKAS is the UK's sole national accreditation body. The government appoints UKAS to assess the bodies that check products and systems against requirements. Some of those bodies offer AI assurance.

DSIT also names data security as a baseline for any AI assurance. It points businesses to the National Cyber Security Centre for help, including the Cyber Essentials certification.

Useful links

• Introduction to AI assurance (DSIT)
• AI regulation: a pro-innovation approach (UK government white paper)
• Guidance on AI and data protection (ICO)
• Accountability and governance implications of AI (ICO)
• Data protection impact assessments (ICO)
• Guidelines for secure AI system development (National Cyber Security Centre)
• UKAS (the UK's national accreditation body)