Engineering Responsible AI: Q&A with Professor Lionel Tarassenko FREng FMedSci

Professor Lionel Tarassenko FREng FMedSci

Lionel Tarassenko is a pioneer in the application of machine learning technologies in clinical care settings. His research has created new methods for assessing patient deterioration and chronic disease management. One of his machine learning systems designed to monitor critical care patients was the first of its kind to gain FDA approval back in 2008.

The Academy’s Digital and Physical Infrastructures team spoke to Professor Tarassenko about how to define generative AI and explain its value.

What are your hopes for the development and deployment of generative AI? 

There are many useful applications for generative AI (also known as Large Language Models), but I also think that there is a lot of confusion among the public about what generative AI is and what it can do. ChatGPT behaves like an intelligent human being when a user interacts with it; it appears to understand what you are asking or saying to it, but it does not – it is just an extremely smart predictive text system.

Part of our role as engineers and computer scientists is to explain to the general public what a large language model does.

For context, when a Large Language Model (LLM) works with text, the first step is to tokenise it, which maps words and word segments to a set of tokens, a set of numbers. An LLM learns associations between tokens from scratch using what is called an attention mechanism during its training phase by going through billions of training runs.

By doing that over and over again, it gradually learns how words relate to each other, and which words to associate with others in the same sentence to make connections based on existing patterns. It is not learning the meaning of words, but rather learning the statistics associated with a language. 

My expectation is that generative AI will be transformative across a whole range of sectors including healthcare. But to access that potential you must have the right data. We have enough healthcare data within the NHS, but the issue is that it’s not aggregated properly, which limits our ability to leverage the value of generative AI for healthcare.

For almost any disease, early detection improves outcomes. Current AI models can detect some diseases relatively early, but I believe that given the right data to use with these large language models, using transformer architectures, earlier detection will become possible for a vast range of diseases. Across different application fields, there are predictive tasks we were not able to do before that we will be able to do now through the development and deployment of generative AI.

Do you have any fears about the progress and implementation of generative AI?

One of the issues is the trustworthiness of input data. Training a generative pre-trained transformer (GPT) using the data available on the internet is not what you want to do for healthcare. If you are trying to detect cancer early, you need oncology textbooks, the last ten years of research papers in oncology, and electronic patient records to develop your own training database to train the generative AI model.

I am also worried about capacity. At the moment, there are probably six big tech companies in the world that can duplicate what OpenAI has done with ChatGPT-3 and 4. Huge amounts of resources—billions of dollars, huge amounts of electricity, huge amounts of time—are required to compete with these companies. So, I am concerned about the future of AI being in the hands of big tech companies in the US and China (probably the only two countries that have the right scale) and the potential lack of control from stakeholders to apply oversight, starting with governments.

Another one of my worries with LLMs, which ties into my concern above, is that certain tasks can only be achieved when you have a large enough model and huge amounts of training data. For example, when conversing in French or German, ChatGPT-3 would only assign grammatical gender in its outputs randomly. However, once GPT-4, a bigger model, was trained (in exactly the same way), it became 100% accurate in the assignment of grammatical gender. This is because GPT-4 has billions more weights, with a bigger training dataset, and enough free parameters to achieve this task. But that is something that we do not fully understand. An LLM must be large enough before certain emergent properties arise, but we do not know why and that is a concern.

At this juncture, what questions do we need to be asking ourselves to enable the safe and impactful use of AI? 

My main question is, where is the human in the loop? When you use these AI tools, they still need to be validated by a human being. For example, with healthcare, when you are predicting the outcome for a patient, you still need the doctor to be involved to validate the prediction from the AI model that you've used to make that prediction. That involves asking, ‘What was the training set and is there any bias in it?’.

One question which is very important when considering the possible dangers of AI tools, are there any actuators connected to their outputs?

My main question is, where is the human in the loop? When you use these AI tools, they still need to be validated by a human being.

 If the output is simply text or images on a screen, then we can control harmful effects, but if the AI system is connected to actuators that generate physical actions, we need even tighter regulation to ensure that these systems are safe.

What are the respective roles of researchers, those deploying the technologies, regulators, and policy makers in enabling safe use of generative AI?

Part of our role as engineers and computer scientists is to explain to the general public what an LLM does. People are potentially being misled about what generative AI is and those of us regularly working with these new technologies can bring clarity to the discussion.

I am also very much in favour of having broad and global discussions between stakeholders. The Prime Minister has called for an AI Summit where we will have the US, the UK, and Europe involved, but we need China in the room. It is really important, whatever our differences are with China, that we have a global discussion at the level of national governments. The UK can be nimble outside the EU, but that does not mean that we should be independent.

What is not being talked about in the ongoing media discussion around generative AI? 

As I said earlier, a LLM is just an extremely smart predictive text system. Even though there may be a sense that we are having a conversation with an LLM, the bot does not understand in the way that people do. Some of the descriptions used in relation to LLMs are not helpful, and some of the hype is just because people do not fully understand what LLMs do.

One interesting aspect of AI-enabled products is that, through the use of a tool like ChatGPT, a not very efficient worker can be made significantly more efficient – while an efficient worker will only be made slightly more efficient. The use of the tool effectively reduces the gap between those at the top of the profession and those who are further behind – an interesting development to think about in terms of jobs and the future of work.