Privacy is going to have a renaissance in the age of AI.

Privacy and data protection will need to be built into AI apps and tools such that they become integral parts of the user experience.

They will become indispensable. Not add-ons or after-thoughts. Top priority requirements.

This view is not easy to appreciate because most people see privacy and data protection as relics of a distant past. As our digital world evolves, the concept of responsible and ethical use of people’s data falls further into a black hole of data maximisation, function creep and invisible processing.

Companies grab whatever data they can, use it for whatever purposes they want and all without the data subject’s knowledge.

This is the current state of affairs when it comes to privacy and data protection - nobody seems to really care.

But this is not entirely true.

In terms of the different attitudes towards privacy data and protection, you can broadly fit companies into three categories:

1. Those that care but struggle with it. They want to be compliant but find the frameworks burdensome, expensive, fragmented, and unpredictable.

2. Those that don’t care because they’ve made a rational calculation. They’ve looked at the expected cost of non-compliance (probability of enforcement multiplied by severity of penalty), compared it to the opportunity cost of diverting resources from product to compliance, and concluded that privacy and data protection is not worth prioritising yet. They’re not necessarily opposed to compliance; they just don’t think it’s material at their stage.

3. Those that don’t care because privacy and data protection have never been made to feel important. These companies haven’t even run the calculation of the second group. Compliance exists in a category of things they’ve filed under “boring, abstract, probably not relevant to me right now.” It’s homework. Nobody does homework voluntarily.

There are not many companies in the first group, there are slightly more in the second, but a lot are in the third group.

And you might think AI will only continue this trend. AI development has so far encouraged norms and practices that vanquish basic data protection principles, including purpose limitation, data minimisation and so on.

But this might be about to change.

Read more

I changed my mind on AI. And you probably should too.

When ChatGPT first came out and was blowing up, I wasn’t all that impressed. I tried it a few times and thought it was cool but did not see the value in talking to a chatbot apart from asking random useless questions.

I had come across these GPTs before. They came up during some research I was doing on how security and intelligence agencies were using AI for their operations. A 2020 report by the Royal United Services Institute detailed how AI could be used by such agencies for ‘cognitive automation’ in which AI can help analyse large volumes of data. It mentioned in this context OpenAI’s GPT-2 and how this model could be used for language analytics purposes, including for analysing transcriptions of captured audio. But not much else was written on the matter, and so I never looked into these language models any further at the time.

And then a short time later in 2022 when a wave of generative products came crashing in, including DALL-E 2, Midjourney and of course ChatGPT, I was wholly sceptical. Suddenly there was lots of talk about artificial general intelligence, AI safety, the prospect of AI taking all of our jobs and so on. But my understanding and interest remained fairly limited, and so I interpreted much of this as hype that should not be taken all that seriously.

And this is the attitude I had for a long time, barely using the tools whilst I stood to the side and criticised. I would read about AI but hardly use it myself.

It is the classic move for most people working in tech law and policy - dealing with the object of that to be governed at arms length and with only a minimal understanding of its mechanics. It is like the difference between static and dynamic analysis of an application. Legal professionals tend to be stuck constantly doing the former and therefore largely ignorant to how these AI systems work in real-world environments.

And there is little motivation to see it any other way. Legal experts are relied on for their risk-aversion and being able to see the worst case scenarios so that they can be mitigated against or avoided. Seeing AI this way is not particularly inspirational and certainly does not encourage one to actually want to test out these systems for themselves.

Eventually my passivity passed though and I decided that instead of sitting on the sidelines, I was going to get stuck in.

This is when I started experimenting with AI for tasks that I wanted to either offload or make much easier to do.

Among these experimentations included building an AI workflow for GDPR vendor reviews, a rather tedious and time-consuming task. I was able to work out the appropriate instructions and harnessing, with safeguards for reducing hallucinations whilst also making verification straightforward.

At this point I was no longer just talking about AI from a distance. I was enhancing my understanding through hands-on experience, something most legal professionals would not think to do.

The challenge I have ran into now, however, is not a technical one. Using AI is something I feel far more comfortable with now, and my experimentation with it will no doubt continue.

The challenge I now face is whether my increased use of AI is actually a good thing.

Read more

People often get distracted by the shiny new object.

The evidence that this is happening with AI is undeniable at this point.

CEOs and senior leaders get dazzled by the stories told by the frontier labs. Predictions are abound that these AI systems are so capable that they will replace all knowledge workers. New models or features get released and SaaS stocks tank. Some companies go as far as engaging in massive layoffs, thinking that large chunks of the workforce can be replaced by an army of cheaper AI agents.

AI has many advantageous of course. Despite being a legal professional and trained to see things with a more risk-averse eye, I can still see the various ways that AI can boost productivity, help make sense of unstructured information and uncover blindspots in my thinking.

But I can also see that AI does not solve all problems at once. It is not a silver bullet.

Like any other piece of technology, AI has its limitations. Hallucinations, getting stuck in recursive loops and all the quirks that come with operating large, non-deterministic systems programmed implicitly.

We are still in the very early days in terms of AI diffusion and adoption. Most people are not heavy users of AI, and therefore a minority know how to use it well.

And it is because of this that most people do not understand what AI is and what it is not.

It is not this magical machine that you can one-line prompt and it will immediately give you everything you need.

Yet some people basically have this perception of AI, probably because of the way it gets hyped by its developers. This mistaken perception leads to AI being treated as a solution looking for a problem.

The erroneous thinking here is something like this:

Because AI is apparently so amazing, and because everyone is using it, then surely I or our organisation can also use and benefit from it too?

This is a trap driven by FOMO and exacerbated by clouded judgment.

However, this is not just a case of misunderstanding the technology that AI is. I think there are other important aspects at play here.

Read more

The way a piece of technology is built and and how it works is always downstream of the incentives, opinions and choices of its builders and the context they are operating in. This is an obvious point that is often forgotten.

That technology is not neutral sometimes allows us to trace the impacts of technology back to the actions and thinking of its builders. Understanding the incentives, opinions and choices of these people is key for understanding why technology develops in a certain way or why it has certain impacts.

This backdrop is important when thinking about how the internet and digital products and services look today. Many of the products and services we have become accustomed to over the years are available for no upfront monetary cost for users. Search engines, social media platforms, and now AI chatbots all have free versions that essentially anyone with an internet connection can use.

But much of this free internet as we know it has been powered by ads. Google was one of the first companies to really work out how to build a business model that enabled its product to be free to use whilst still generating revenue. It figured out how to make revenue-generation essentially invisible to users yet still providing and updating a useful product.

And now OpenAI is doing the same by introducing ads to ChatGPT. The AI provider is currently rolling this out to ChatGPT free and Go users, with advertisers reportedly committing between $50,000 to $100,000 in ad spend.

The reasoning for implementing ads is similar to the constraints that Google and others had to face in the journey to profitability. However, I think the nature of its implementation for AI systems like ChatGPT will be different.

OpenAI’s pivot to ads is a sign of the reality they face - they need a reliable way to generate revenue. The company needs a way to cope with the immense increases in infrastructure spend on top of the high level of investments it has secured over the years. It would seem that subscriptions and enterprise deals are not sufficient.

Maybe OpenAI’s mission is still to build AGI eventually. But it may not survive to achieve this feat unless it develops a workable business model in the meantime to fund and power its efforts toward this grand milestone.

Google faced a similar problem in its early days. It bulked up its ad businesses due to pressure from investors and the need to somehow make money from its product.

When it comes to ads on Google, these are aligned to inferences made about users based on what the queries they enter into the search engine. On social media platforms, these inferences are derived from a wider range of user activity, including the people they follow and the content they engage with.

Ads in AI chatbots will be aligned to something different, something much deeper and higher-resolution than the data points that search engines or social media platforms can utilise.

AI ads will be aligned to a user’s actual thoughts, something very distinct from the search queries, social graphs and engagement metrics of yesteryears.

In this newsletter I go through this argument and explore:

• How AI systems sit closer to user intentions than any other system

• Why AI ads are inevitable

• What happens to data rights

Read more

The conflict between Anthropic and the US Government is a big deal, but not in the way that most people think.

It is not just a contract disagreement or an AI ethics debate. I think it represents something bigger.

To give a very simplified overview of what happened between Anthropic and the US Department of War:

• In mid‑2025, the Pentagon awards large multi‑year AI contracts (up to about 200 million dollars each) to several firms, including Anthropic, to help build out military AI capabilities

• As the government builds an internal generative‑AI ecosystem (often described as GenAI.mil), negotiations focus on how broadly military users can deploy Anthropic’s models

• The Department insists on “all lawful uses” language; Anthropic agrees except for two red‑lines: no mass domestic surveillance and fully autonomous lethal weapons - these safety guardrails become the core of the dispute

• The administration publicly hardens its stance, promoting an “AI‑first” warfighting strategy and rebranding rhetoric around the Pentagon as the “Department of War,” signaling a more aggressive doctrine

• In late February 2026, defense officials issue Anthropic an ultimatum: remove the contractual bans on mass domestic surveillance and autonomous lethal weapons or risk severe consequences

• Anthropic refuses before the deadline, reiterating that those uses are outside what its models can safely or ethically support

• After talks break down, the administration orders federal agencies to stop using Anthropic’s AI tools and begin transitioning away over several months

• The Department of War designates Anthropic a “supply chain risk to national security,” a label normally associated with foreign adversaries; this effectively bars defense contractors from using Anthropic products in government work

• Anthropic publicly rejects the designation as unlawful and politically motivated, arguing it is being punished for insisting on safety guardrails

• In March 2026, Anthropic files suit against the U.S. government, challenging the supply‑chain‑risk classification and seeking to protect its right to limit high‑risk military uses of its AI

I’ve read good takes on the situation by various writers here on Substack. You shoudl also check out the coverage from Privacat, Jasmine Sun and Dean W. Ball, just to name a few.

When reading all of this commentary, what came to mind was an overarching set of themes that could be threaded together.

In fact, the fight between Anthropic and USG felt quite familiar to me. My gateway into tech law and policy was state surveillance and the Snowden revelations in 2013. I became fascinated with the way technology and law collided together, a convergence of forces that advance and organise our societies.

This for me signalled one of the most important questions of our time (for which I have quoted Jamie Susskind, author of Future Politics: Living Together in a World Transformed by Tech, many times before because he puts it far me eloquently than I have):

To what extent should builders of powerful digital systems be concerned with data rights, and what does this look like in practice?

What I see here with Anthropic vs USG are three converging patterns:

1. The state depends on private tech firms more than the reverse, and this will only continue in the future

2. Safety guardrails for LLMs are somewhat of a technical illusion that creates a big governance gap

3. LLMs could bring surveillance, whether by the state or other entities, to a whole new level

In this newsletter, I cover these three conversing ideas (power inversion, safety guardrails and the evolution in surveillance) and how Anthropic vs USG is a glimpse at the defining tension of the 21st century.

Share

Read more

Many might be excited about adopting AI, or anxious that not doing so will leave them far behind the competition.

So they start ideating and brainstorming about how these systems could be used to improve internal processes or create better products for customers.

In doing so, some will have the foresight to think about the importance of governance - there should be a focus on building the thing the right way and not just building anything.

But then this is where organisations make the biggest mistake when starting their AI governance journey.

They start by drafting a policy.

Maybe they look for a template online. Or maybe they even use AI to generate one for them.

They draft it, put is somewhere on their intranet, and then that’s it. They think they have completed governance and now they can just focus on building or using their AI systems however they like.

Simple, right?

Well not really.

Policies are just words. They might represent the principles, rules, guardrails and standards you want to follow when building or using AI. But these things alone are simply not enough.

These things just represent intentions.

And while crystallising intentions in a formal document that everyone agrees to follow is important, starting your governance journey with a policy means that your policy will not be:

1. Aligned with how your organisation is using and/or developing AI systems

2. Backed up by practical measures that implement the policy

3. Actually known, understood and respected by staff

Without these things, your AI governance framework just never really gets going.

If you want to get AI governance right, do not start by writing a policy. You need to do other things alongside the drafting of the policy that will make your AI governance framework actually work.

You need to at least:

• Understand the state of play in your organisation

• Develop an AI strategy

• Build and implement the right measures (organisational, technical and legal)

• Establish feedback loops

In this newsletter, I go through each of these steps, including why they are needed and what they entail. If you like this content, make sure to share it with others who might also benefit.

Share

Read more

If you have been paying attention to the AI rhetoric these days, you will have likely heard a lot of people talk about ‘taste’.

I myself started seeing this concept being banded around for a while, and an increasing amount at the beginning of the year with all the craze around Claude Code.

Claude Code opens the door for everyone to build (almost) anything. And this opportunity is not just available to those who know how to code - you simply need natural language to get going. Explain what you want built, and the coding agent will do the rest.

But with AI taking away the expense and friction of execution, focus has now moved to the other parts of problem-solving with code. Because in a world where you can now build (almost) anything, what matters more than ever is building the right thing in the right way.

And this got me thinking; there seems to be such a thing as having good taste in AI development and deployment, which I do think is genuinely important. But what about having good taste when it comes AI governance? Is there even such a thing?

Initially, I thought this was a silly idea. Taste connotes creativity, imagination and novelty. These are not words that people think of when they think of governance. On the contrary they might instead think: onerous, tedious and a blocker.

However, the more I thought about it, the more I realised that taste in governance is not only real, but crucial.

Governance without taste fulfils the dreary perception that most have about the field.

Governance with taste fulfils legal requirements but also helps unlock the full potential of AI.

Tasteful governance melts the idea that legal professionals are mere luddites who stifle products and demonstrates a way to balance the engineering and lawyerly incentives.

What tasteful governance actually means

For me, taste is essentially a filtering mechanism.

Over time, you accumulate various bits of knowledge from the work you do. You build an understanding of the different problems that you or your clients face as well as the appropriate solutions to them.

These problems and corresponding solutions are compressed into retrievable principles and samples that can be applied to new sets of problems you encounter later on.

Taste is therefore a way of deciding how to sample from that knowledge base. It is about selecting the learned principles and samples from previous work that are most appropriate for the present work.

This is how taste works in other domains too, whether it be music, movies or people. Of all the music one has listened to, of all the movies one has watched, and of all the people one has interacted with, they know how to pick well by using a learned criteria that helps distinguish between the good, the decent and the bad.

Taste is therefore the ability to pattern-match, abstract and see the signal in the noise in various situations based on past learnings.

To apply this to governance, taste is not just about knowing what the law says. That is the easier bit really. You look up the correct rules for a client’s given situation and interpret their meaning accurately.

It is the next bit where taste is important - knowing how to implement the rules. And this task requires creativity, context-sensitivity and commercial awareness to not merely come up with a solution, but the solution. That is tasteful governance in a nutshell.

Read more

The idea that LLMs ‘contain’ personal data was always an ambitious take.

Probably.

When I first looked at this topic a little over a year ago, I was not sure where I stood. I thought that the implications of concluding that models did not include personal data were simpler, thereby making that conclusion much more attractive. You can read my newsletter on this here:

If you have no idea what I am talking about, I explain everything in the newsletter linked above. But this is the gist:

• LLMs are trained on lots of data, much of which is scraped from the internet

• In that training data is lots information that would constitute personal data under the EU’s GDPR

• There is an argument that, if that personal data is used for training the model, the resulting trained model ‘contains’ the personal data it was trained on

• If the model contains personal data, and that model is used by another entity, then that entity may be processing personal data

• The point made in my previous newsletter on this is that this is all quite complicated and I was never convinced one way or another; whether models do or do not contain personal data

But now my thoughts have evolved since the decision by the Court of Justice of the European Union (CJEU) in EDPS vs SRB, which I have also written about (twice):

That decision feels like a significant juncture in the development of EU data protection law that has ramifications for a number of data processing operations, including those pertaining to AI.

The SRB case clarified a point that even I thought was not up for debate: not all pseudonymised data is personal data. Even when I wrote about the Advocate General’s opinion on the case prior to the Court’s decision, I thought this might be one of the rare instances in which the CJEU would diverge from the AG.

I was wrong.

It turns out that there is nuance in data protection after all. Not all of it is rigid and strictly interpreted, and it allows for a flexibility that makes it workable in different contexts.

Perhaps this is a better way - such an approach to pseudonymisation probably makes sense.

If you encrypt some data, and share the cipher text with another entity without a copy of the cryptographic keys, the receiving entity does not have personal data in GDPR terms.

They have something that has been pseudonymised, but if they have no means to decrypt it, reverse engineer or otherwise transform the cipher text into its original form, then they just have unintelligible gibberish. Or at least they have information that could not be linked to any person and therefore identify a particular person. There is no personal data there.

There are some who might say that this opens the door for compliance escapism. If the information I have received cannot be used to identify a person, even if indirectly, then I don’t need to bother with GDPR obligations. Why would I when the information is not personal data?

From this perspective, SRB opens up a new gateway for avoiding the perceived compliance headaches of one the EU’s flagship regulations. And this gateway may be something that deployers of AI system take full advantage of.

Here is what I am getting at: if we take the principle from SRB and apply it to the question of whether AI models contain personal data, the answer is...still complicated.

If you look under the hood of a model, you could point to some parts and say, ‘yep, that is definitely personal data.’ But then there might be other parts where this is not the case.

The reason for this is because AI models are not like databases. It is not the internet indexed and searchable through a chat interface. Not at all.

The way model’s ‘store’ information is much different. If you look under the hood, you will see a probability distribution with fragments of words with their embeddings all numerically represented with no obvious organisation or structure.

The only way you could possibly argue that there is personal data anywhere in that mess is if you can demonstrate that the model has memorised verbatim some of its training data and that memorised training data is personal data.

This is not a crazy idea. Systems like ChatGPT have previously shown the tendency to do this kind of memorising and spit out personal data buried deep in its massive text corpus. This includes phone numbers, email address, physical addresses and more.

But to expose this vulnerability, you need the right prompt. You need a specific prompt attack that reveals the relevant personal data that the model may have memorised.

This leads to the critical next question - what are the prompts that do this? Or in SRB terms, what means would a deployer of AI systems have to extract and process the personal data contained in the system? How does a deployer know which personal data have been memorised and therefore can be extracted with the right prompt?

If you could not tell by now, this post is a very nerdy data-protection-crosses-technical-realities deep dive akin to what I did when I first looked at this issue of model’s containing personal data.

So all the details may not be all that exciting, but if you are a deployer of AI systems, whether its ChatGPT, Claude, Grok or others, then the thrust of this piece is highly relevant:

The SRB decision invites a compliance strategy based on ignorance; AI system deployers intentionally depriving themselves of the means to ‘re-identify’ any personal data in the model they are using.

It is a big take, but nevertheless a realistic one that is worth exploring. And in the remainder of this post, I attempt to explain it in the simplest way possible so that you can really understand both the point I am making, why I am making it and the implications it has for organisations using AI systems built by others.

As always, if you find this content useful, share it.

Share

Let’s dive in.

The SRB principle

Pseudonymised data is not always personal data.

Now I think it is first of all worth explaining the concept of ‘personal data’ and what it actually means in the world of GDPR.

Simply put, ‘personal data’ means information that can be used to identify a person.

So personal data is not information that might be considered, in some colloquial sense, personal or sensitive. Sometimes when I hear people talking about personal data, they put emphasis on the personal so as to mean information that is particularly special, unique or intimate for the person it belongs to.

It certainly can be, but the concept of personal is much wider than that.

To really understand this, it is important to break the definition of personal data down into its constituent parts:

• any information

• relating to

• identified or identifiable

• natural person

Any information literally means any information, and this can be objective or subjective information about someone. Think names, email addresses, phone numbers but also opinions, assessments or even predictions about a person.

To be personal data, that information needs to be about an individual. This means that either the content, purpose or effect of the information must be linked to a particular person:

• The content element is satisfied if the information itself is about an individual, such as the exam result of a student

• The purpose element is satisfied if the information can be used to evaluate or analyse an individual

• The effect element is satisfied if the use of the information has an impact on an individual’s rights or interests

To be identified or identifiable is about whether the entity holding the information can use it to single out a person from other people. I will come back to this later on.

Finally, a natural person is just a legal term for a person. So personal data does not include information about a corporation or organisation or anything that is not a human. The GDPR also does not apply to deceased persons.

With a sufficient understanding of personal data, we can then turn to the concept of pseudonymised data.

Generally, a pseudonym can be thought of as a cover name or a replacement for a true value or a kind of derivative of some original information. Pseudonymisation is therefore the process of taking data and applying some transformation to it that turns it into pseudonymised data.

Let’s say you have an email address: mahdiassan@email.com. If you wanted to pseudonymise this piece of information, there a couple a different ways you could do it.

You could pseudonymise the email address using a technique called masking whereby you simply replace certain characters in the address:

# masking_example 

original_data = mahdiassan@email.com

pseudo_data = m********n@email.com

A more complicated way to pseudonymise the data is encrypt it whereby a cryptographic protocol is applied to the email which outputs some cipher text:

# encryption_example

original_data = mahdiassan@email.com

pseudo_data = 92edfa8361b7af3e637

This is where I want to return to the idea of identifiability.

Identifiability exists on a spectrum. On the one end, you have data points that directly identifies a specific individual (like a name) and on the other end you have data points that only indirectly identify individuals (like a userID). It is important to remember two things here though:

• Indirect identifiability includes data points that can be linked to a person even if it is not known exactly who that person is

• Anonymity is the complete opposite of direct identifiability - this where data cannot be linked to any person at all (as can be the case with aggregated statistics)

Pseudonymisation is about reducing the identifiability of personal data. It reduces the identifiability of data such that it can no longer be used to identify a specific individual. In other words, without the use additional information, it would be difficult to identify exactly who the pseduonymised data relates to.

Let’s go back to the encryption example above. When you encrypt data, you produce cipher text but also a set of cryptographic keys. These keys can be used to encrypt as well as decrypt the data. So if I encrypted some data and shared only the cipher text with someone else, and I kept the keys to myself, it would be very difficult for that person to use that data to identify someone - all they have is a hash value that bascially looks like a bunch of gibberish (92edfa8361b7af3e637).

However, a question one may have is, even if the person I shared the data with only has the cipher text, is that cipher text still personal data? After all, the keys to decrypt the data, and turn it back into its original form (mahdiassan@email.com), are still in my hands and therefore I still have the ability to see the personal data that has been encrypted. But regarding the third party I have shared only the cipher text with, what are they holding?

There are two different approaches to this question: a strict approach and a relative approach.

Under the strict approach, the cipher text in the hands of the third party is still personal data. That the cryptographic keys are in still existence, and therefore could be used by me to decrypt the data and link it to an individual, means that the encrypted data is still ultimately personal data. The means of identification are still there.

The relative approach, however, adds some nuance to this. Though the means for identification exist, it does not mean that the person that the data relates to is always identifiable. This depends on the means available to the person holding the information in question.

For a while, the strict approach seemed to be a dominate view among the data protection community. But a judgment from the CJEU in EDPS vs SRB last year has declared something different; that the relative approach should be taken regarding the concept of personal data.

I will not go over the case details again in this post; you can read all that in my previous post on the topic. But what I will reiterate here are the principles that can be derived from SRB.

Using my encrypted data example again, if I share only the encrypted data with the third party, and that third party has no access to the cryptographic keys, then, from the perspective of that third party, they are not holding personal data. This is as long as the following is true:

1. The third party cannot ‘lift’ the pseudonymisation (or in this case the encryption) preventing re-identification

2. The third party cannot perform re-identification through cross-checking with other available information it may have access to (including information it can search on the internet)

3. The risk of identification is insignificant considering the cost, time and the technology available

Accordingly, from the position of the third party with whom I share the encrypted data with, they are not holding personal data because:

• They do not have the cryptographic keys to decrypt the data

• There is no other information they can use to perform re-identification using the cipher text only (i.e., they cannot reverse-engineer the cipher text)

• If I use a sufficiently complex cryptographic protocol, they cannot reproduce the cryptographic keys needed to decrypt the data (maybe barring access to a sufficiently powerful quantum computer)

The key thing to understand from SRB is that the nature of personal data’s relativity ultimately depends on the entity holding it. In essence, whether pseudonymised is personal data depends on who is looking at it and what they can do with it, not just on the data’s inherent properties.

Applying SRB to AI systems

I used the example of encrypted data earlier because it has a particular relevance to the second part of my thesis, which is about what is inside an LLM.

LLMs are giant prediction machines. They take your natural language input and spit out something that they think you need.

But if you look under the hood of an LLM, it is complex to say the least. It consists of tokenisers, embedding layers, positional encoders, transformer blocks and a probability distribution.

If you want a detailed explanation of how LLMs work, you can go back to my previous post:

But for now, the key point I make in that post is that the type of information that LLMs ‘store’ and the nature of that storage is not like how we traditionally think of data stored in databases.

When an LLM is being trained, internally it is building something called a vocabulary (or a vocab). This part of the model contains all the tokens it has come across in its training. A token represents a word of a fragment of a word in a numerical form (e.g., ‘computer’ might be tokenised to 2886). Additionally, each token will have assigned to it a word embedding, which encodes the tokens relationship with all the other tokens in the vocabulary.

So every time you give a model a prompt, it will refer to its vocab and work out the probability of each token being the appropriate tokens for the response to the prompt. And then it will select those tokens that are more likely to form the right output.

This vocab consists of a matrix that does kind of look like a table:

An LLM’s vocab can be very large - tens of thousands worth of tokens. This means that the matrix could have tens of thousands of columns representing each token along with tens of thousands of rows of values representing the word embeddings for each token.

But this matrix is not a database. The information in the vocab consists of values for the parameters that are learned during training, and those values codify the relationships between all the tokens that the model comes across during training (and this is in addition to the weights stored in the neural networks of each transformer block).

So altogether then, an LLM stores:

• All the tokens it comes across during training

• The embedding vectors for each token

• The weights for the neural networks in each transformer block

This information does not itself relate to an identifiable natural person. It is a bunch of numbers that humans cannot comprehend. And even if we could comprehend these numbers, linking that back to particular people whose personal data may be in the training data would still be incredibly difficult.

In effect, all you have inside an LLM is a matrix of seemingly random numbers that are only readily interpretable to the LLM.

Looking inside an LLM to identify the personal data contained in there is no use. However, deconstructing the black box that is an LLM is not how most people interact with such digital artefacts. Most people interact with LLMs by prompting the text box on their screen.

Now this is where the encryption analogy comes in.

LLMs sometimes memorise verbatim the data they comes across during training. This does not happen with all the data points, but there is research out there showing that they are capable of doing this.

After training on a large dataset of text, the LLM has an understanding of language in the sense that it understands the correlations and patterns between words. However, the model does not necessarily learn factual information about the language it learns.

Any “factual knowledge” that the model is able to produce is merely derived from its understanding of language and the correlations between different words (or tokens). So by understanding language, it is possible for LLMs to “learn” factual information.

In particular, such factual information could be learned if the relevant patterns appear frequently enough in the training dataset. The more often the model comes across a certain pattern during training, the more prominently that pattern will be represented in its parameters.

Whenever the model then receives an input containing text relating to that more frequent pattern, it will rely on that pattern to produce its response. In doing so, the model could produce data that it has essentially ‘memorised’ from its training data.

So while the information contained within LLMs is not intelligible to humans, the personal data that is in there somewhere and has been memorised by the model could be extracted by prompting the model in certain ways.

If I ask ChatGPT when Donald Trump was born, it says he was born on 14 June 1946:

But if I ask ChatGPT who was born on 14 June 1946, I get this:

Clearly ChatGPT has learned from is training data that “14 June 1946” and “born” and “Donald Trump” are all strongly associated with each other, hence why I get the outputs I do.

Obviously though the prompts for extracting this information are pretty straight forward here given that Donald Trump, or any other public figure, will likely feature multiple times in the training data. This increases the propensity of the LLM to memorise certain information about such figures and therefore increases the ease of which such information can be extracted through prompting.

But the point here is that personal data that has been memorised by the model can be transformed from an unreadable to a readable format by using the right prompt. Which is similar to the how encryption works.

Recall that with encryption, you have:

• The original data

• The cryptographic protocol applied to the data

• The generation of the encrypted data

• A set of cryptographic keys that can decrypt the encrypted data back to the original data

And you could describe LLMs and personal data extraction in a similar way:

• Personal data is contained in the large training datasets consisting of data scraped from the internet

• That data are tokenised and mapped to word embeddings

• The data are therefore converted into a bunch of values representing these tokens and embeddings and organised into a matrix

• With the right prompt, that unintelligible bunch of numbers can effectively be converted into readable text that may constitute personal data

With this logic, the information you have inside LLMs is pseduonymised data. The tokens and word embeddings are in a pseudonymised form as a result of model training, and then transformed into intelligible personal data if the right prompt is used with the model at inference.

Extending this, if a developer takes a trained LLM and incorporates it into a new product, then that developer will not necessarily be holding personal data. Applying the SRB principle, the developer is only holding personal data extractable from the model if they use the means to extract it, which would be the prompts.

But if the developer does not know what those prompts are and does not use them, then that extractable, memorised personal data remains pseudonymised. The GDPR obligations that would therefore otherwise apply (selecting an appropriate legal basis, adhering purpose limitation and data minimisation etc) are not of concern to the developer regarding the pseudonymised personal data ‘inside’ the model it is incorporating into its product.

I would argue that this perspective on the matter is reinforced by a compliance strategy that has been implicitly endorsed - a strategy based on ignorance.

Compliance based on ignorance

I want to be clear about a couple of things here before I go on explaining this take.

Firstly, I am not saying that this potential compliance strategy is right in the sense of it is a good thing. But I do predict that this will be a strategy that many developers will be tempted towards because it makes their compliance work simpler.

Secondly, this potential compliance strategy does not absolve developers of all of their data protection responsibilities. This only absolves them from the needing to address data protection issues arising from the use of the model or building on top of it - the personal data contained in these models is not their responsibility. However, the way that the system they are building using the model processes personal data is still their problem, and data protection responsibilities definitely still apply there. This is the case regarding personal data used to develop the system or personal data collected when people use the system, but neither of these processing operations are the subject of this post.

The compliance strategy explored in this piece specifically concerns developers using trained models to build new products - do those products already contain personal data by using a model trained with lots of personal data that might be ‘stored’ in the model?

I argued in the last section that these models do not necessarily contain personal data. In this section, I look at how developers could go about demonstrating this by essentially remaining ignorant about the details of the models they are using.

Read more

I started using it properly about a month ago, initially just testing out its competency at one-shot app creation, for which I was careful to manage expectations of course.

Watching this coding agent work through your request, while keeping you in the loop to check that its work is aligned with your preferences, is quite fascinating. There is something quite magical and intriguing about how this little bot can just take your initial prompt and crank out a full-fledged application. It is equally cool to see how it can carry out various tasks that would otherwise require a decent level of coding proficiency.

I have found a lot of inspiration from a number of Substack newsletters on the joys of using Claude Code, including from Privacat, Nathan Lambert and Jasmine Sun.

The interesting thing about Claude Code is that, despite it being a coding agent and therefore seemingly only of interest to coders or those more technically capable, it is capturing the attention of those outside the AI bubble. Even the Wall Street Journal is writing about Claude Code, so it must be something cool even among the normies.

As fun as all this has been (and maybe even a little addicting), my experience with Claude Code did alert me to something which I think could very easily be overlooked for those who have been ‘Claude-pilled.’

Products like Claude Code make it easier and cheaper than ever to build software solutions. And if these codings agents continue to improve, so it is quite striking to think of the possibilities down the line.

The temptation that this fuels is to just build a bunch of new stuff. If all you need is a bit of natural language prompting and a machine will just turn your idea into reality with a little further input from you, then think of all the ideas that do not need to be just ideas anymore. Imagine all the things you could build. Maybe you really are just one weekend away from a building the next killer app that makes you a fortune in a week.

The obvious problem here is that as people get carried away with the magic of Claude Code, they start to forget the fundamental questions about what they are doing. They forget about the aspects of building that AI cannot necessarily do for them.

AI can do many things, but it cannot do everything.

In the midst of your Claude psychosis, you may forgo why you are even building the thing in the first place as well as the principles for building it well.

And this is crucial, because if you end up building the wrong thing in the wrong way, and then deploy it for other people to use, you may disappoint your users at best or even harm them at worse.

To avoid this, you need to think about what it will take to actually build things properly with products like Claude Code:

• Taste. Are you solving a real problem that actually needs solving?

• Distribution. How do you get people to care about your solution to the problem?

• Trust. Does your solution actually work well?

Trust is particularly important. In a sea of new apps being built and deployed everyday by eager vibe-coders leveraging the power of agents, why should people use your app? What makes it so much better than the rest?

Even if your app solves a problem that others are struggling with, and even if you have an audience willing to use it, if it does not deliver on the promises, then you just get a high churn rate. And all your late nights with Claude Code will go to waste.

Alternatively, if you take the time to build a solid solution that works well, then you have a much better chance of gaining trust. And this thens feed into the other factors; the more people believe in your product, the more they share it with others which leads to more users and therefore more feedback on how well your app is doing.

The key to gaining this trust is embracing governance.

Governance is not just about legal compliance. Fundamentally, governance is about implementing measures that help you build more reliable products.

You should think of governance as an enabler. If you get it right from the start, then not only do deal with any compliance issues, but you also give people another reasons to trust your product. And you feed the growth flywheel.

To get governance right when building with Claude Code, I think you need to start by managing three things: quality, data and risk.

Get these three things right and you will be well on your way to establishing a unique selling point for your product and a moat that most will not think to work on until it is too late.

In this post, I walk through the steps and prompts for quality, data and risk management when using Claude Code. If you are building apps with Claude Code that you plan to release into the wild, then this is essential reading.

If you find this content valuable, make sure to share it with others.

Share

Read more

There a lot of people who do not know where their data goes when using an AI chatbot.

They will open up ChatGPT or Claude, type in their prompt, maybe attach some files, and watch this seemingly magical artificial entity generate a response that at least aligns with, but preferably exceeds, their expectations.

But in the midst of being so mesmerised, many will not bother to think what really happens inside that black box.

Not many people will really think about what happens to their data once they submit it to the machine.

Some people may say: ‘who cares?‘

And maybe this is a reasonable response. Why care about the journey your information takes when it enters the complex bowels of a language model and gets transformed into new output rendered token-by-token to gradually build a response to your query? Surely all that matters here are the ends, not the means.

But the mistake with this mindset, if you have it, is that you are making some quite stark assumptions about those means.

You are assuming that the use of your data is within the realm of your expectations.

You are assuming that the use of your data is limited to simply running inference on the model to get the answer it thinks you need.

You are assuming that the the use of your data is sufficiently proper and ethical.

But what if these assumptions are wrong?

What if your data is being used for purposes that you did not expect and might even object to?

What if your data is being shared with other entities you did not even know were involved?

What if your data is out of your control?

Here is the painful truth when it comes to navigating our digital word: every time you use a digital service, you give up some data. And when you surrender your data to these digital systems, your agency over it automatically diminishes.

Moreover, you are giving your data up to systems built and controlled by others with incentives and priorities that may not be conducive to yours.

And control over your data also means control over your digital experience. Sometimes that experience may be absolutely fine, but the point is that this is not determined by you. It is determined by other people.

A few weeks ago, OpenAI announced that it would start displaying ads on ChatGPT.

If you have been reading my content over the past few months, you would see how I have talked about such a decision becoming a reality. Investments in AI have been enormous, fuelled by the dramatic promises of generative AI. Companies like OpenAI, with Altman as the prime advocator, have been constantly glamourising their models as magical technologies that would solve so many of humanities great problems and become beacons of growth and progress.

But in the end, when the hype fades away, OpenAI and others were always going to need to demonstrate to their investors how they were going to get returns on their investments. AI developers needed to show how they were actually going to make money with their products.

In the end, OpenAI has chosen a path that has worked very well for the tech companies before it. Google, Facebook and others turned to surveillance capitalism to strengthen their business models, involving the construction of data extraction systems that turned behavioural insights of users into revenue via targeted advertising. The cycle has simply repeated itself.

If OpenAI is doing this, then the question of what actually happens to your data when you use ChatGPT and other tools like it really does become important.

The first step to solving any problem is to understand the problem really well. And to help with this, in this post I uncover the system that lies behind the chatbots that you are probably well-familiar with or at least heard of.

I set out the different components that make up the system, how they fit and work together, and how your data moves through the system when you use it. I also look at the data risks involved and some simple steps you can take to better-protect yourself.

This will be relevant to everyone who uses an AI chatbot. It does not matter if you are using it for throw-away personal questions or if you are part of the ‘shadow AI’ clique at work. You need to know what happens to your data when you use a chatbot.

If you understand the systems you are engaging with, you put yourself in a better position to mitigate the risks. You cannot fully solve a problem unless you really understand the problem.

Make sure to share this newsletter with others if you find it valuable. And subscribe if you want more insights like this in your inbox every week.

Read more

AI will kill all lawyers.

This was the title of an article in the Spectator that I recently read. And it prompted some thoughts. Many thoughts.

The article gives the views of an anonymous English barrister on AI and legal practice. And the picture he gives is pretty gloomy: AI will ‘completely destroy’ the law as we know it.

Why such a stark warning? It is based on this particular barrister’s recent experience using AI for legal work, as he explains:

‘Last week we did an experiment, a kind of simulation. We took a real, recent and important case – a complex civil court appeal which I wrote, and it took me a day and a half. We redacted all identifying details, for anonymity and confidentiality, and we fed the same case to Grok Heavy AI. And then we asked it to do what I did. After some prompting, the end result was…’ He shakes his head. ‘Spectacular. Actually staggering. It did it in 30 seconds, and it was much better than mine. And remember, I am very good at this.’

[...]

‘It was at the level of a truly great KC. The best possible legal document. And all done in seconds for pennies. How can any of us compete? We can’t.’

Lawyers belong to one of the most conservative, risk-averse and arguably outdated professions that exist. They are used to holding high positions in society because their services are so necessary; as long as we have societies organised by a large, complex body of rules, we need people who can understand them and provide guidance on how to follow those rules.

Or do we?

Because if you read this barrister’s view on AI and its potential impact on the legal profession, you may think that if AI can do just as good as a qualified legal expert, then what is the point of having these experts?

You could go further: what is the point of knowledge work done by humans if we can get AI to do it?

These general-purpose machines, trained on massive amounts of data with huge amounts of computing power, seem capable of pretty much any cognitive task a human could do. Just feed it the right prompt and you have your desired output, produced in seconds or minutes at the most and on the cheap.

But as I was reading this Spectator article, I could not help but notice a rather glaring omission which reveals why ‘AI displacement’ is more complicated than what some people present.

AI is not about to take everyone’s job.

Those who believe in a mass exodus where AI renders human roles completely redundant are not thinking carefully enough about what AI is and the impact it can actually have.

AI can do many things. And it can do many things really well.

But it cannot do everything. This is the simple truth.

This means that AI should really be seen as a force multiplier, not a replacement.

And this is the case even for the legal profession.

To thrive in the age of AI is to understand AI’s limitations as well as its capabilities and the gaps it therefore creates in our society.

Those gaps are where humans still have an important role to play, whether this is in the legal industry or any other domain where AI is being deployed.

In this newsletter, I cover three core ideas about how to navigate the trials and tribulations of AI in our modern world:

• What AI can do

• What AI cannot do

• What you should do

Read more

Most people will be focusing on the wrong things when it comes to AI in 2026.

Most people will be focusing on the emerging paradigms, the new breakthroughs, or the next groundbreaking model claimed to be AGI.

They are wrong.

What is far more important is something that too often gets overlooked. Something that is lethally ignored until it is too late. Something that, if done well, unlocks the real power of AI in a way that is not only sustainable but beats 99% of the competition.

To build good AI products, it is a mistake to obsess over the latest and greatest models. Chip Huyen, an experienced computer scientist and author of AI Engineering: Building Applications with Foundation Models, makes this point exactly. For her, the things that actually contribute to better AI products are talking to users, building a more reliable platform, using better data, optimising for end-to-end workflows and writing better prompts.

If you want customers to invest in your product, they need to be a position to trust your product. When the hype and bravado eventually fade away, trust is the currency that keeps customers around.

What all this means is that the most important thing for AI developers to focus on in 2026 and beyond is governance.

This is not because it is required by law. Or because it is a ‘nice-to-have’. Or even because it is good for PR.

Governance is a business-enabler. It is a way of deepening moats. It is the dynamic equilibrium that balances innovation and order.

Good governance is what underpins good, reliable AI products that actually work, build trust and deliver value.

As my last newsletter for 2025, I want to look to 2026 and what I think is next for AI from a data rights and governance perspective. I want to share what I think the AI landscape will probably look like, the problems it will produce, and why governance is the way to solve them.

Read more

Bubbles pop eventually.

When the dot-com bubble popped in 2000, it caused the worst decline in the NASDQ’s history, dropping by more than 70%. Many companies that had so confidently slapped ‘.com’ on the end of their names suddenly had to face a reckoning, and many did not survive.

They did not survive because aesthetics would only get them so far. The shiny new object, which back then came in the form of a website on the world-wide web, would eventually need to prove its worth.

A bubble popping represents a reversion back to reality and fundamentals, and that is when the promises and predictions of technology are really tested.

And one of the more significant players carrying out this testing are investors. Having a cool product with potential is fine on Day 1. But on Day 2, investors are looking at the prospect of their returns. They want to know what they will end up getting out of it all.

Google was no exception to this.

Read more

With the AI boom and hype comes many AI systems. Some are built for general-purpose use - household names like ChatGPT and Claude may come to mind. Others are designed for more specific use cases, like RobinAI (for contract review) or Cursor AI (for writing code).

The plethora of systems available provide plenty of opportunities for organisations to innovate. AI can help produce new products or services or improve existing processes for building products or services.

Such opportunity still exists despite organisations needing to resort to the use of systems built by others. It might sometimes be preferable to develop your own system specifically built for your own use case with your own data, giving you a wide range of customisability. However, these systems built by the likes of OpenAI and Anthropic still come with plenty of flexibility. Their models are general-purpose which can be built on top of, fine-tuned or engineered with context and other tools to construct systems for a range of domains.

But with this opportunity comes risk, and these risks are just not limited to those which are legal in nature. AI development is itself an empirical science, whereby assessing behaviour and performance can only really be done by using models and monitoring them post-deployment. Models are often characterised as black-boxes possessing a level of complexity that renders them opaque and difficult to control. This can mean that AI systems risk failing to meet important business and legal requirements, with potentially significant consequences; poor ROI, user complaints and even regulatory intervention and legal action.

And so with these risks come responsibility. As Ethan Mollick notes in his book Co-Intelligence: Living and Working with AI, with AI becoming increasingly capable, “we’ll need to grapple with the awe and excitement of living with increasingly powerful alien co-intelligence.”1

For organisations procuring AI systems, this responsibility comes in the form of being aware of what you buy. In the world of AI, the principle caveat emptor (’buyer beware’) is crucial.

Read more

2025 was supposed to be the year of agents.

Generative AI (genAI), central to the current AI hype cycle, represents a significant leap from the previous generation of AI. It took AI from systems that could recognise patterns and ma…

Read more

So the European Commission is exploring ways to simplify several different EU laws as part of its ‘Omnibus’ reform package, and one of the drafts for this was leaked last week.

There are some interesting things to note from this leak. The part that sto…

Read more

Th first chapter is titled ‘Engineers vs. Lawyers’ and Wang uses this frame to convey the contrast between progress in China and progress in the US. China is an engineering society, while the US is…

Read more

Hallucinations describe outputs of LLMs that are factually incorrect. It is a common flaw of these models and can be a major source of risk.

But why do LLMs hallucinate? What is the cause of this tendency to generate incorrect responses?

The abstract of OpenA…

Read more

Machine learning is about building machines that fit mathematical models to observed data to produce a desired output. And a deep neural network is a type of machine learning model.

So wha…

Read more

This newsletter is about how the AI Act and GDPR intersect regarding the use of sensitive data for AI development. It looks what is required under both pieces of legislation, the potential gaps in the legal framework and what steps developers could take regarding compliance.

Here are the key takeaways:

• Under the GDPR, personal data cannot be processed without an appropriate legal basis, of which are provided under Article 6 of the Regulation. Additionally, sensitive personal data cannot be processed unless one of the exceptions listed under Article 9.2 apply.

• The AI Act permits, under Article 10.5, the use of sensitive data for the purposes of bias mitigation regarding the development of AI systems. It states that providers may exceptionally process sensitive data “to the extent strictly necessary for the purpose of ensuring bias detection and correction in relation to the high-risk AI systems.”

• Article 10.5 itself cannot be a legal basis for processing sensitive data for AI development. The provision is written in such a way that entertains the possibility of processing sensitive data for AI development, but only for the purpose of bias mitigation and only if its use meets certain other conditions both under the AI Act and the GDPR.

• Steps that developers can take to help comply with the requirements under the AI Act and GDPR regarding the use of sensitive data for AI development include:

  • Determining whether sensitive data are needed for the development of the AI system

  • Documenting the justifications/explanations for using sensitive data in a record of processing operation

  • Applying data minimisation to the sensitive data

Read more