Cosplaying expertise while the chat window sits empty
How acquaintance softens prejudice in AI governance
I like Aesop’s fables because they provide simple and accessible ways of conveying interesting themes about human behaviour.
Take the fox and the lion, which goes like this:
A very young Fox, who had never before seen a Lion, happened to meet one in the forest. A single look was enough to send the Fox off at top speed for the nearest hiding place.
The second time the Fox saw the Lion he stopped behind a tree to look at him a moment before slinking away. But the third time, the Fox went boldly up to the Lion and, without turning a hair, said, “Hello, there, old top.”
When I read this, it makes me think about the legal profession and its perception of AI.
I recently had a conversation with a friend about AI in which we talked about this topic. She mentioned to me how she meets so many lawyers who talk about AI and how to govern it have barely used it themselves, or sometimes not even at all.
There is a perplexing irony in legal professionals engaged in AI governance work, helping organisations work out how to use AI responsibly and ethically, and yet they themselves have never used ChatGPT or any other AI system.
It is the classic tension between theory and practice. It reminds me of one of the many things I learned from reading Atomic Habits about the difference between being in motion and taking action.
When you are in motion, you are constantly planning and thinking about doing something. But this is distinct from actually taking action and doing the thing.
Being stuck in motion is often comforting because it feels like we are doing work without facing any actual risk. The obvious consequence, however, is that no progress is being made and you end up having nothing to show for all the “work” that you have done.
And I think legal professionals sometimes fall into this trap with AI precisely because of the perceived risk. Since they belong to an incredibly cautious and risk averse industry, the incentive to dive into the latest and greatest technology is not particularly high, and they’d rather deal with it from a safe distance while pontificating about why it’s good or bad.
One thing that I think contributes to this are the court cases involving lawyers using AI to produce documents for legal proceedings that contain fake citations or authorities. Remarkably, there have been a few of these cases in the UK. [See Ayinde, R (On the Application Of) v London Borough of Haringey [2025] EWHC 1383 (Admin) and Munir v Secretary of State for the Home Department [2026] UKUT 81 (IAC)]
There have been some in the US too. In 2023, in the early days of the current AI hype cycle, a US lawyer admitted to using ChatGPT to submit a brief citing several previous cases which in fact did not exist. This lawyer had no idea that a system like ChatGPT could generate false information, probably thinking it worked like a search engine or some other database retrieval system.
To be clear, what I don’t think is necessarily required of legal professionals is a deep, technical understanding of how these tools work. I don’t think it is essential for a legal professional to know the full block equation for the scaled dot-product attention mechanism for transformers, or the composition of affine maps and nonlinear activations in the context of feed-forward neural networks.
But they should have some understanding of what they are dealing with. They should know the data types these systems take as input, how the output consists of a prediction of the best response to that input, the importance of prompt and context engineering, how RAG works, the difference between LLMs and agents and so on.
And I think a lot of that can come from actually using the tools available.
Especially when it is so easy. Anyone can use ChatGPT, Claude, Gemini, Perplexity and the many other AI systems out there. You just make an account and play with the models.
This hands-on experience is one of the best way to really see for yourself what these systems are like. Yes you can read about how they work and hear others rant about how amazing they are or how they are just stochastic parrots. But sometimes you need to try the thing out for yourself.
And when you do, you realise that AI is not god-in-box. It is not something you can one-line prompt and it magically do everything you ask it to do with no mistakes and no need to verify what it has done.
What we have is something that is very good at generating stuff, cheaper and quicker than ever before. It provides access to huge buckets of knowledge that was previously hard to obtain and use.
But, crucially, the current state-of-the-art AI systems, despite all that they can generate, need direction. If you want to get the best out of AI, you still need to know what you are doing. You still need to know how provide clear and detailed instructions, share the right amount of relevant context and examples, and verify whatever the system spits back out.
And you can only really understand this when you have used the systems yourself. Thoroughly.
A few weeks ago I wrote about how important the verification layer is for AI. These systems are sycophantic - they tell you what you want to hear and it is so crucial to not be seduced by this and recognise the sometimes subtle ways that it provides incorrect information or leads you down the wrong path.
Legal professionals need to learn how to use AI so that they can have a better understanding of what it can and cannot do and therefore what should be done about it.