Why AI requires its own governance

The difference between AI systems and other computer systems

AI consists of technologies designed to mimic human behaviour. It therefore covers a range of different technologies and techniques, and among them is machine learning.

Machine learning is about building machines that fit mathematical models to observed data to produce a desired output. And a deep neural network is a type of machine learning model.

So what is the difference between AI and traditional software?

With traditional software development, a developer would write ‘rules’, and then instruct the computer to apply those rules to some data to produce ‘answers’.

AI is different. With AI, the developer provides the computer with data and answers (or sometimes just data), and then instructs the computer to come up with a set of rules that will take the data and produce the desired outputs as accurately as possible.

In other words, traditional software development is about building systems explicitly, whereas AI is about building systems implicitly.

So herein lies the reason why AI needs its own governance mechanisms? The difference between traditional software development and AI development results in further implications that require a specific approach to governance.

What are those further implications? Here are a few:

• The workings of AI models are probabilistic (many different possible outputs) whereas traditional software development is more deterministic (certain inputs produce certain outputs). AI governance therefore requires measures that deal with the complexity and unpredictability of AI systems.

• With traditional software development, it is easier to measure and validate outputs and ensure that the system is working as intended. This task is more difficult for AI systems as their complexity renders them harder to interpret and control. The evaluation measures used for governance AI systems must account for this.

• Relative to traditional software development, AI development involves the use of lots of data. This activity often falls within the scope of data protection laws, not only forming a core element of AI governance but also presenting unique challenges to protecting personal data in the digital age.

AI requires its own governance because AI is just different.