The European Union officially published the AI Act in July 2024 with a strict timeline for its enforcement. Understandably, many AI developers and professionals are uncertain and under pressure to comply with this new regulation. General Purpose AI (GPAI) systems, such as ChatGPT, notably became part of the AI Act as a last-minute addition.
This article will dive into how companies using general-purpose AI like ChatGPT can stay compliant under the EU AI Act.
Who is responsible for AI compliance under the EU AI Act?
The risk of Generative AI products lets the legislators reconsider the demands they are putting on developers in the EU.
For companies building AI systems that utilize foundational General Purpose AI (GPAI) models such as ChatGPT, these additions remove the impossible burden of independently completing the full documentation, quality control, and risk management of the ChatGPT systems themselves. This task is not only burdensome but utterly impossible for companies that did not develop the GPAI.
To protect these developers, the EU shifted the obligation within EU regulations to the providers of GPAI systems. In other words, if OpenAI wants to provide services like ChatGPT in the EU, it needs to comply with the law, and the users of such a system can rely on this for their compliance.
If such users provide an AI system on top of ChatGPT, like a Retrieval-Augmented Generation (RAG) system, they are only obliged to fulfill the requirements for their part of the system.
AI tools like ChatGPT are widely used for many applications, such as IT service bots, question-and-answer systems for university students, or to respond to customer inquiries. They can boost productivity, reduce the workload of busy employees, and open new business models. However, applying these large language models (LLMs) often comes with risks of false information and liability.
Many of these users are not yet aware that they are subject to the EU AI Act’s requirements, the first of which took effect in February 2025.
A Concise Overview of GPAI Requirements Under the EU AI Act
The AI Act applies to specific AI systems and General-Purpose AI (GPAI) models. The AI systems are classified based on their level of risk. The AI Act differentiates four levels of risk: AI systems with minimal risk, AI systems with transparency risk, high-risk AI systems, and AI systems with unacceptable risks.
Depending on the risk level of the AI system, different requirements may apply. No direct requirements apply for AI models that are not considered “general purpose”.
This risk-based approach by the AI Act recognizes that the use of AI is not inherently dangerous and needs to be regulated, but it depends on the circumstances of its usage. All of the obligations of the lower-risk systems also apply to the higher-risk systems. Systems with unacceptable risk are prohibited in the EU.
GPAI systems are defined in Article 3(63) as AI models that:
- Display significant generality,
- Are capable of competently performing a wide range of distinct tasks and
- Can be integrated into downstream systems or applications
What are the risks of using ChatGPT in my organization?
OpenAI’s ChatGPT is considered a GPAI model by the EU AI Act. A user of ChatGPT might be regarded as a producer or deployer of AI systems, depending on their role in the market and how much they enhance the ChatGPT system.
A producer of an AI system is any natural or legal person who develops an AI system or a GPAI model or has it developed under its brand and puts it on the market. Open AI (ChatGPT) and Microsoft (Copilot) are examples of producers, but companies that are building on these systems to create new, distinct products are also considered producers.
Any natural or legal person using an AI system in a professional and non-personal manner is considered a deployer of an AI system and has to follow the obligations of a deployer.
Enhancing or using Chat GPT?
When a company or its employees use ChatGPT to accomplish their business goal, the company becomes a deployer of an AI system. If this system is considered a high-risk system, it must comply with the obligation in Article 26 of the AI Act.
Assessing the risk level of your AI system might feel like an intimidating process. Fortunately, there are free compliance checker tools available that can help you check the risk level of your ChatGPT application and how it can comply with the EU AI Act.
Some other applications are prohibited, like subliminal techniques, social scoring, or real-time remote biometrics. Additionally, a long list of specific applications are considered high-risk, often when the AI system is a safety component in a relevant domain, like aviation, motor traffic, or toys.
When you intend to modify a GPAI model significantly and put it in a system considered a high-risk AI system, you become a provider of a high-risk AI system. You then must comply with the extended obligations of a provider of a high-risk system.
As mentioned before, the original producer of the GPAI is obligated to provide sufficient documentation and information to enable these downstream providers to fulfill their obligations.
Using AI systems with minimal risk
Suppose you’re considered a deployer of an AI system with minimal risk. In that case, you only need to promote AI literacy among your staff and other persons dealing with the operation and use of your AI system on your behalf, taking into account their technical knowledge, experience, education, and training and the context in which the AI systems are to be used.
This obligation became effective as of February 2025. Additionally, deployers with systems that interact with people or generate content, like ChatGPT, must fulfill the transparency requirements (Article 50), and they must inform their users that they are interacting with an AI system or that the output of the AI system was created with the help of AI.
The EU AI Act does not specify a separate legal basis for liability claims. Nevertheless, you might be liable for actions that infringe on your obligation as providers or deployers, even for non-high-risk systems. Not providing AI literacy training for your employees who use AI systems or not conforming to the transparency obligations for the AI systems can become a liability risk in case something goes wrong.
And using LLMs is not a hands-off operation. LLMs are at risk of providing false or misleading information, they can produce inappropriate content, and they should always have guardrails and be tested for their specific application. A common practice is to include the option of consulting a human expert at any point in the conversation with the AI system to avoid misunderstandings and errors.
In addition to the EU AI Act, the ChatGPT application must comply with the General Data Protection Regulation (GDPR) when processing personal data. However, GDPR does not apply when non-personal data is used.
Ensure ChatGPT compliance with an AI Registry
As a first practical step, we recommend auditing all the AI systems you developed internally. To do this, you should write down all the technical implementation details, the intended use, the data used, and the potential risks for all systems in an AI system card.
Tools like the Daiki AI Registry can facilitate this process and ensure that your AI systems are of the highest standards by allowing for external validation of your AI system cards. Our AI assistant prompts you with all the essential questions you should ask about your system: What are the components of the system? Which AI model did you use? What is the data used, and where did it come from? The assistant then checks the results for completeness and assigns a trustworthiness score.
This step is vital because you need to know which AI systems you have in your inventory, which are running, and what their quality and risks are. This provides you with information on how to build your AI governance structures based on your relevant topics.
Governance is dynamic and should grow based on your working style, aided by standards like ISO/IEC 42001 and regulatory frameworks.
With this in mind, you can use the frameworks even if you’re not yet obliged to implement them as a guardrail to aid the cultural transformation of your organization. Incorporating new processes around new norms can be cumbersome, but it doesn’t have to be.
Be proactive and take action now; by assessing and registering your AI systems, you will be well on your way to achieving EU AI Act compliance.
Staying Compliant while Innovating
As AI developers and users, it is easy to see regulatory obligations like the EU AI Act only as barriers to market entry. You must do one more thing on your long list of compliance tasks.
But fundamentally, your company and the regulatory bodies have the same goal: to develop reliable and high-quality AI products that all market participants can safely use.
AI governance goes beyond checklists and tools. Specifically in AI, standards can help facilitate the evolution into an AI-literate company with high AI adoption rates. Companies that successfully adopt AI tools into production often share that they went through a cultural shift in how they think about the risks and benefits of AI and changed their processes accordingly.
Expensive proof-of-concept projects that don’t make it into production slow down this cultural adoption and cost money and time. In this light, compliance is necessary work that any diligent company can achieve by being proactive and open to the possibilities that AI technologies can enable.
FAQs about working compliantly with ChatGPT
Can ChatGPT be used to handle customer data?
Yes, but consider data privacy and the possibility of data leakage. If the customer data you want to use with ChatGPT contains any personally identifiable information, it is very unlikely to comply with GDPR.
What are the penalties for non-compliance?
The EU AI Act sets out a framework in which the market surveillance authorities determine the fines. For GPAI, the Act already contains binding rules on the fines. For violations of the obligations of operators of high-risk AI systems and GPAI models and breaches of transparency obligations, fines of up to €15 million or up to 3% of the total worldwide turnover may be imposed.
Is employee training necessary to use ChatGPT compliantly?
Yes. AI literacy training is required for all employees using a ChatGPT system.
How can ChatGPT potentially cause data breaches in my organization?
The information in the prompts to a ChatGPT system is sent to OpenAI and processed there. Other providers have a similar model. With complex AI systems such as RAG systems, the prompt can contain a lot of information, e.g., from internal documents.
Be clear that you are sending information to a third party that has the potential to read (and often the right) and to use your information for their purposes. Intellectual property rights not protected by patents can be violated here. Check with different vendors of large language models on their data handling policies, or consider a private language model instance on your server for complete control.
Are there any specific sectors or use cases where compliance risks with ChatGPT are higher?
Yes. Check the list of high-risk AI systems specified by the EU AI Act Article 6.
