ICO’s guidance on the use of artificial intelligence

The UK Supervisory Authority (ICO) recently published guidance on artificial intelligence (AI). The advice intends to provide organisations that are either using or developing AI technologies with practical recommendations on the steps they should take to comply with data protection law and principles. The following points are the key takeaways from the ICO guidance.

Why guidance on AI?

The ICO published the guidance to help organisations mitigate data protection risks and to explain how data protection principles apply to the use of AI in an organisation or business environment. The guidance not only draws on the European General Data Protection Regulation (GDPR) but also on broader data protection principles embedded in UK data protection law.

The guidance focuses on AI decision-making that often raises the issue of the so-called ‘black box’, i.e., decisions that are reached by AI systems with inner workings that are inaccessible to ordinary human understanding. This is often the case when machine learning algorithms are used in the analysis of large data sets to identify patterns and correlations. The developer may not know in advance what data will be interpreted and how variables will be analysed.

This triggers several conflicts with data protection principles. Transparency within the decision-making process is called into question in terms of what the AI system is doing and how it is using an individual’s data such that data subjects can exercise their rights. Other concerns include quality and security of the data entered into an AI system as well as accountability of the designer or operator of an AI system.

Data protection principles in the AI context

Accountability and data protection impact assessment

The accountability principle makes data controllers responsible for complying with data protection and demonstrating compliance. In the context of an organisation using AI systems, you are required to

  • be responsible for the compliance of the AI system,
  • assess and mitigate its risks, and
  • document and demonstrate how the system is compliant and justify the choices you have made.

In most cases, data controllers will be legally required to conduct a data protection impact assessment (DPIA) for the use of an AI system if personal data is processed. Conducting a DPIA will ensure that the reasons for and use of an AI system to process personal data are considered as well as the potential risks and how they can be mitigated.

To ensure that data protection and other fundamental rights in the context of AI systems are respected, you need to ensure that in the design or development stage, you identify and assess what rights might be at stake. Then, determine how you can manage them in the context of the purposes of your processing and the risks it poses to the rights and freedoms of individuals, and make sure that there is compliance with all data protection principles.

Lawfulness, fairness, and transparency

When an organisation uses AI systems, it must always ensure that it fulfils the principles of lawfulness, fairness, and transparency in the personal data processing. The ‘black box’ issue makes this particularly challenging for data controllers. Accordingly, even developers and designers often cannot predict how an AI system reaches decisions; thus, fairness cannot be guaranteed due to a lack of transparency.

Further, a lawful basis must be established whenever processing personal data. In many cases, consent that is freely given, specific and informed can be the appropriate legal basis, if you have a direct relationship with the data subjects. In AI systems, the particular purposes for the use of data cannot always be fully predicted as processes are not entirely transparent. Furthermore, individuals must always be able to withdraw consent, which can become complicated if personal data is put into an AI system.

Article 22 General Data Protection Regulation (GDPR)

Art. 22 GDPR, which applies to all automated individual decision-making and profiling, introduces additional rules to protect individuals from being subjected to a decision based solely on automated processing that has legal or similarly significant effects on them. This is only allowed if the processing is (1) necessary for the entry into or performance of a contract, (2) authorised by law, or (3) based on valid consent. Thus, to comply with Article 22 GDPR, you must ensure that you

  • give individuals information about the processing,
  • introduce simple ways for them to request human intervention or challenge a decision, and
  • carry out regular checks to make sure your systems are working as intended.

Data security and accuracy, data minimisation

AI systems usually process large amounts of data. The ICO refers to the risk of failing to comply with the principle of data minimisation. Accordingly, you may process the personal data that you need for the AI system. That means you must consider if the data you are processing in an AI system is “adequate, relevant and limited” to what is necessary. Especially in the context of machine learning AI, where AI is trained by processing large amounts of data, it can be challenging to comply with the data minimisation principle.

Furthermore, as pointed out by the ICO, if AI systems are used to make predictions or decisions about data subjects, you must ensure with a reasonable degree of confidence that the outcomes are accurate. This means that you should take steps to make sure that incorrect or erroneous interferences are corrected, and data subjects have the opportunities to exercise their rights.

Rights of data subjects in the context of AI

If you use data that falls under the ‘special category data’, such as ethnical, racial or religious information, potentially additional conditions may apply under the GDPR and the UK Data Protection Act 2018 (DPA 2018). This means that to process special category data in an AI system, explicit consent may be required.

If you may process personal data in an AI system to make decisions or predictions, you must ensure that suitable measures to safeguard data subjects’ rights and freedoms are in place. According to Article 22 (3) GDPR, you must ensure that a data subject can

  • obtain human intervention in decision-making,
  • express their point of view,
  • contest decisions.

If the lawful basis for processing is based on consent, data subjects must be allowed to withdraw the consent if they object the processing. If there is no other legal basis to justify the processing, controllers must immediately stop doing so. Furthermore, as envisaged in Recital 71 GDPR, you must ensure to correct inaccuracies, prevent errors, and prevent discriminatory or bias effects as a result of the AI system.

For controllers to fulfil these obligations when processing personal data in AI systems, you should adhere to the following best practices to ensure compliance with data protection principles.

Best practices for data protection compliance in AI systems

The ICO focuses on recommendations for organisations that use AI systems in making sure they are complying with their data protection obligations. A key challenge is the justification of output produced by AI systems. This is essential for controllers to be able to fulfil their obligations towards data subjects as well as to comply with the principles of lawfulness, fairness, and transparency. Organisations are therefore advised to do the following to ensure that data protection duties are not neglected:


First and foremost, organisations should identify if and which AI systems are used that make decisions or predictions about individuals. You should clarify if personal data is processed, and if special categories of personal data are processed, the purposes for the processing and how to establish a lawful basis.

Risk Assessment

Where personal data is at stake in the AI system, organisations will have to conduct a DPIA, which focusses on the potential risks to the rights and freedoms of affected individuals. This will allow you to identify the risks and possibilities of how to mitigate these risks.

Documentation and implementation of an AI system

Organisations that either develop an AI system themselves or supply it from a third party are data controllers and are thus obliged to comply with data protection laws and principles.

You should document the process behind the design and the implementation of the AI system and its outcomes. This documentation should be understandable to people with different levels of technical knowledge and should be used as evidence to explain how decisions were made.

Measures include those involving privacy by design and other technical and data security standards. If you procure the AI system from a third party, the contract with that party must ensure that relevant due diligence and data security obligations are incorporated.

In practice, this means that you must ensure that

  • the steps involved in the creation and deployment of an AI system are explained, helping to produce and develop company policies and procedures to address and mitigate specific risks;
  • each step that has been taken in the design and deployment phase is documented to explain potential outcomes; and
  • any third-party supplier of AI systems can explain how AI has produced its output.


To ensure that updates or changes of AI systems will not negatively affect the fulfilment of data protection obligations, controllers should undertake ongoing monitoring, particularly concerning data accuracy and quality.

Concluding remarks

The ICO emphasises that risk mitigation in the design stage is essential to building a basis of compliance with data protection principles. It further highlights that the development and use of AI are growing and evolving and that, accordingly, the authority will continue to work on clarifications as to how to ensure compliance with data protection principles.

Make data protection your competitive advantage. Our UK data protection support will help you!

Leave a Reply

Your email address will not be published. * Required fields.

Netiquette: We do not tolerate grossly unobjective contributions or advertising on our own behalf and will not publish corresponding entries but delete them. I have been informed about the processing of my data according to the privacy policy of activeMind.legal.