General-purpose models under the EU AI Act
General-purpose models under the EU AI Act
A quick summary of the obligations that apply to OpenAI and other genAI developers
TL;DR
This newsletter is about the rules under the EU AI Act that apply to providers of general-purpose models. It looks at the different types of general-purpose models under the legislation, and the obligations imposed on each.
Here are the key takeaways:
The Act makes a distinction between AI models and AI systems. A general-purpose AI model is an AI model with general capabilities for a wide range of tasks, whereas general-purpose AI systems are AI systems that integrate general-purpose AI models to perform a variety of purposes.
Obligations are imposed on providers of two types general-purpose AI models; those with systemic risk and those without systemic risk.
In essence, the AI Act defines 'systemic risk' as risks arising from models with high-impact capabilities that could have a negative effect on public health, safety or security, fundamental rights or society as a whole and propagated at scale across the value chain. The European Commission can make its own assessment as to whether a general-purpose model presents such a risk.
Under Article 53 of the Act, providers of general-purpose models without systemic risks have two main sets of obligations. The first relates to the development of technical documentation that includes information on the model's architecture and training process, and the second relates to the provision of information for developers that may want to integrate the models into their own systems.
Providers of general-purpose models with systemic risk must comply with obligations set out in Article 55 in addition to those provided in Article 53. The Article 55 obligations include the testing, assessment, treating and documenting of systemic risks arising from the model and the implementation of appropriate cybersecurity measures.
The Act makes provision for the development of codes of practice that providers of general-purpose models can rely on to comply with their obligations. The EU AI Office and AI Board, both newly established under the Act, are responsible for ensuring the development of these codes.
What are general-purpose models?
It is important to note that the Act makes a distinction between AI models and AI systems.
A general-purpose AI model is an AI model that "displays significant generality and is capable of competently performing a wide range of distinct tasks."1 Such models can therefore be integrated into different systems or applications.2
Accordingly, a general-purpose AI system refers to an AI system that uses a general-purpose AI model to perform a variety of purposes.3 Such systems can be used directly or integrated with other AI systems.
Examples of general-purpose models include the popular foundation models in generative AI (genAI). This includes OpenAI's GPT models, Anthropic's Claude or Google's Gemini.
Systemic risk
The Act identifies two types of general-purpose models: models with systemic risk and models without systemic risk.
So what is systemic risk? This is how it is defined under Article 3.65 of the Act:
...a risk that is specific to the high-impact capabilities of general-purpose AI models, having a significant impact on the Union market due to their reach, or due to actual or reasonably foreseeable negative effects on public health, safety, public security, fundamental rights, or the society as a whole, that can be propagated at scale across the value chain.
Accordingly, under Article 51.1, a general-purpose AI model is classified as a model with systemic risk if it meets at least one of the following conditions:
It has high impact capabilities (Article 3.64 defines this as "capabilities that match or exceed the capabilities recorded in the most advanced general-purpose AI models").
A decision of the European Commission determines a model to have high impact capabilities.
Simply put, a general-purpose AI model with systemic risk is a model with high-impact capabilities that could have a negative effect on public health, safety or security, fundamental rights or society as a whole and propagated at scale across the value chain.
When the European Commission is determining whether a model has high-impact capabilities, it must do so on the basis of the criteria set out in Annex XIII.4 This criteria includes:
The number of parameters
The quality and size of the dataset
The amount of computation used for training (measured in FLOPs)
The input and output modalities of the model (i.e., text or images)
The benchmarks and evaluations of the model's capabilities
Whether it has a high impact on the EU market due to its reach (this is presumed if the model has been made available to at least 10,000 registered business users established in the EU)
Regarding the computation used for training, Article 51.2 states that AI models are presumed to have high-impact capabilities if the "cumulative amount of computation used for its training measured in [FLOPs] is greater than 10." I have written previously on the drawbacks of using FLOPs as a way to predict the risk of models as articulated by the machine learning researcher Sara Hooker, which you see below.
Under the Act, there are obligations that apply to models with and without systemic risk. However, there are additional rules pertaining to those with systemic risk to ensure that such risks are effectively addressed by providers.5
Obligations for models without systemic risk
Under Article 53 of the Act, providers of general-purpose models without systemic risks have two main sets of obligations.
The first relates to technical documentation. Model providers are required to maintain documentation that includes information on the training and testing and how the model was evaluated.6
The exact information that is required in the technical documentation is contained in Annex XI. This includes, among other things:
The tasks the model is intended to perform
The applicable acceptable use policies
The architecture and number of parameters
Information on the training process, including the training methodologies and techniques used
Information on the data used for training, testing and validating the model
The computational resources used to train the model (e.g., the number of FLOPs)
The second set of obligations concerns the information and documentation that must be made available to those who may integrate the model into their AI system.7 The purpose of this documentation is to enable developers to "have a good understanding of the capabilities and limitations" of the model, so that they themselves can comply with their obligations under the Act.8
Annex XII sets out what this developer information and documentation should contain.9 Among this includes the "the architecture and number of parameters" as well as "information on the data used for training, testing and validation, where applicable, including the type and provenance of data and curation methodologies."
This second set of obligations also require providers to do two more things:
Put in a place a policy to comply with EU copyright law and related rights10
Draw up and publish a "sufficiently detailed summary about the content used for training."11
However, providers of general-purpose models without systemic risk are exempt from the aforementioned obligations if the model is released under a free and open-source licence. For the purposes of Article 53.3, "open source" essentially means a release of the model that allows for both of the following:
Free access, usage, modification and distribution of the model.
Public availability of the parameters (including the weights) and information on the model architecture and model usage.12
Obligations for models with systemic risk
Providers of general-purpose models with systemic risk must comply with obligations set out in Article 55 in addition to those provided in Article 53.
The obligations under Article 55 can be split into two parts:
Risk management
Cybersecurity
Regarding risk management, providers of general-purpose models with systemic risk need to complete the following tasks:13
Evaluate the model using "standardised protocols and tools reflecting the state of the art." This needs to include adversarial testing (also known as red-teaming) to identify the model's systemic risks.
Assess and mitigate the systemic risks. These risks may stem from the development of the model, the placing of it on the market or the use of it.
Track, document and report "relevant information about serious incidents and possible corrective measures to address them."
Regarding cybersecurity, the Act requires providers to "ensure an adequate level of cybersecurity protection."14 Recital (115) elaborates on this:
Cybersecurity protection related to systemic risks associated with malicious use or attacks should duly consider accidental model leakage, unauthorised releases, circumvention of safety measures, and defence against cyberattacks, unauthorised access or model theft. That protection could be facilitated by securing model weights, algorithms, servers, and data sets, such as through operational security measures for information security, specific cybersecurity policies, adequate technical and established solutions, and cyber and physical access controls, appropriate to the relevant circumstances and the risks involved.
Codes of practice
The Act makes provision for the development of codes of practice that providers of general-purpose models can rely on to comply with their obligations under the legislation.15
The Act establishes an AI Office, which sits within the European Commission and is responsible for developing EU "expertise and capabilities in the field of AI."16 This is in addition to the establishment of a European AI Board,17 which is responsible for the "consistent and effective application of [the] Regulation."18
Both the AI Office and Board are required to ensure codes of practice that cover the obligations for providers of general-purpose models. This includes codes of conduct on:19
The provision of technical documentation
The adequate level of detail for the summary on the content used for training
The identification of the type and nature of systemic risks, including their sources
The measures, procedures and modalities for the assessment and management of systemic risks
The AI Office can invite providers of general-purpose models to participate in the development of these codes. Civil society organisations, industry, academia and other relevant stakeholders may also be involved in the process.20
However, excluded from this definition are models which are used for research, development or prototyping activities before they are 'placed on the market' (which essentially means before making it available to users in the EU), as per Article 3.10.
As per Article 52.1, providers of models with systemic risk are required to notify the European Commission of such models. But if the Commission becomes aware of a model presenting systemic risks that it has not been notified of, then it can make its own determination.
EU AI Act, Article 53.1(b)(i).
EU AI Act, Article 53.1(b)(ii).
On 'model usage', the Act does not define exactly what this means.
EU AI Act, Article 55.1(a)-(c).
Great breakdown, and nicely detailed.