Can OpenAI do without EU?

By now ChatGPT hardly needs an introduction.  The chatbot, based on an advanced AI model (GPT-3) developed by OpenAI, has brought generative AI into the public consciousness and has become inescapable.

In the midst of all the excitement, on 31 March 2023 Italy's data protection authority (Garante per la Protezione dei Dati Personali, the 'Garante') stated that it would temporarily block access to ChatGPT effective immediately and would launch an investigation into ChatGPT's data collection policies.

Whilst the ban remains in place many rushed to access ChatGPT using virtual private networks (VPNs).  This highlights the difficulties that countries and regulators across the world face when seeking to ban or limit access to websites not least because companies like OpenAI use servers across the world in order to service their substantial userbase (thereby increasing someone's ability to 'hop around' using a VPN). 

The ban – and its consequences – does not affect OpenAI or ChatGPT users alone. It underscores issues that companies and developers at all stages of an AI's lifecycle need to reckon with in view of incoming regulation across jurisdictions.

How does ChatGPT collect data?

The model behind ChatGPT (GPT-3) was trained using text databases from the internet which ranged from books, Wikipedia, articles and the like. OpenAI routinely then updated GPT-3 (which is why users interacting with ChatGPT are told that the cut-off for its knowledge base is September 2021). If an article appears online, it is very likely that ChatGPT has ingested it for training purposes despite the fact that its author may not have consented to the article being used for algorithmic training purposes or where the text is actually copyrighted.

In addition to this, when users interact with ChatGPT, OpenAI collects vast amounts of data.  In its own privacy policy, OpenAI states that it may collect "the contents of any messages" a user sends in addition, the type of computer or mobile device used, IP address etc.  

The risk therefore is that, in addition to being trained on vast amounts of data, OpenAI can collect contents of all messages sent by a user when interacting with ChatGPT – including personal details, financial information and the like. 

Why did Italy ban ChatGPT?

The Garante in Italy launched its investigation as a result of a security breach on 20 March 2023 which resulted in users being shown extracts of other user's conversations and financial information.

Italy – which was the first Western country to block the chatbot - cited the data breach as a reason for the ban in addition to the following: 

• users are not provided with information noting that their data is being collected; 
• there are no real age verification systems in place; and
• under GDPR, there is no legal basis for OpenAI to collect and process the personal data on which ChatGPT's model (GPT-3) is trained.

What will ChatGPT's future be in Italy?

Italy has given OpenAI until 30 April 2023 to comply with the Garante's measures if it wants to continue operating in Italy and avoid paying a significant fine (which could amount to up to €20 million or 4% of the total worldwide annual turnover).

The Garante requires OpenAI to, amongst other things: 

• explain to users why the processing of certain data is required for ChatGPT to function;
• remove any reference to it being able to process user's personal data for algorithmic training by virtue of a 'contract' (as it must instead justify any collection and processing of personal data pursuant to GDPR principles of accountability, consent or legitimate interest);
• allow interested non-users to object to the processing of their personal data used to run and train the algorithms;
• present a clear action plan for an effective age verification system; and
• engage in a promotion campaign across all media forms in Italy to inform the public about use of their personal data for algorithmic training purposes.

Italy's grounds for banning the app in this case were that no age verification procedures are in place and that – even when users willingly state that they are minors – no changes to the app's interface are implemented meaning that minors could then be exposed to inappropriate content.  The Garante also cited concerns that the emotional attachment users may develop in interacting with the virtual friend might cause vulnerable users to be at risk. Lastly, Italy noted that Replika was in breach of its GDPR obligations because it could not claim that users had consented to processing of personal data where these were minors.

It's clear that the Garante – as well as other data protection authorities across the EU – are taking proactive steps to ensure that AI powered tools do not breach data protection regulations and user privacy.

It will be interesting to see whether OpenAI intends to comply with the above measures and, if so, whether they will implement them for all users across the EU (or more widely). The measures are incredibly unique and assessing their effectiveness will be important to companies, users and regulators across the world.

What has been the effect of the ban?

Italy's move to ban ChatGPT due to privacy and data protection concerns has emboldened other data protection authorities within the EU to do the same. Sweden, France, Germany and Ireland have also launched investigations into the way in which OpenAI collects data and then uses it.  

On 13 April 2023, the EU's European Data Protection Board (EDPB) also announced that it was launching a dedicated taskforce in order to ensure that data protection authorities in Member States could cooperate and exchange information on possible enforcement actions.  

This was a sensible move by the EDPB as without coordination across the EU (whilst we await the implementation of the EU AI Act) there is a risk that different data protection authorities may require companies like OpenAI to comply with different requirements for their user base leading to a lack of harmony across the EU and creating confusion for businesses developing and deploying AI products used by EU users.

Further afield, Canada's Privacy Commissioner has also launched a similar investigation.

How does this impact OpenAI and, more widely, other companies developing similar websites and apps?

It's important to note that OpenAI does not have an office in the EU.   However, in line with the upcoming EU AI Act (and with the extra-territorial scope of the GDPR), from the EU's point of view, what matters is whether outputs produced by the AI system are then used in the EU.  

OpenAI is not the sole target of data protection authorities. Companies and developers at all stages of an AI's lifecycle should therefore ensure that they are complying with data protection rules in all jurisdictions they operate in by, for example, ensuring that data is collected and processed in accordance with the GDPR.

It will be interesting to see whether the Garante – or indeed any other data protection authority - will also move to regulate companies operating in the metaverse on similar grounds (notably, Meta's Horizon has issues with age verification).

Although the EU AI Act has yet to come in force, this is not stopping data protection authorities and regulators from using their powers under the GDPR to challenge the way in which companies collect and process personal data to train their algorithms.

Whether these bans and investigations are effective in prompting companies like OpenAI and Replika to be more transparent and respect data-protection rules will be an important test-case for companies, users and regulators across the world.  Stay tuned for an update following the 30 April 2023 deadline….