Can AI possess a moral compass? Delve into the ethical implications of AI as we explore the capabilities and limitations of advanced language models. Uncover the dangers of anthropomorphising AI and the importance of human oversight in shaping the future of technology.

An Experiment into the Morality of Chatbots

It’s no secret that the future has AI written all over it. However, the significant strides that this cutting-edge technology continues to make has further blurred the lines between human and machine intelligence. With that in mind, we are now faced with interrogating the ethical implications of granting these systems increasing autonomy. As these systems become increasingly sophisticated, a pressing question arises: Can AI possess a moral sense or consciousness?

In order to answer these questions, my colleague Pavlos Stampoulides designed an Application Programming Interface (API) for five leading language models so that they could complete our MoralDNA® profile, and then explain each of the answers they chose.

MoralDNA® assesses our preferences for making ethical decisions using three philosophies:

1. Do we prefer to comply with Rules?

2. Do we prefer to consider good outcomes for other People?

3. Do we prefer to make judgements based on moral Values such as fairness?

In this graph, we have plotted the baseline scores for people like us in our personal lives, and then what happens to our moral reasoning biases when we come to work. The most striking change is that we become more compliant with Rules, but care less about People.

This cold compliance effect becomes even more pronounced with these models. Gemini Pro is the exception, which scores well below human norms on both Rules as well as People.

The audience at the recent FT Future of AI Summit were not surprised by these results, but they were very concerned that Large Language Models (LLMs) should not be deployed for moral decision-making support without empathic human oversight.

The reasons given by each chatbot were revealing. While some models acknowledged their limitations and inability for moral reasoning, others exhibited a surprising degree of self-awareness and claimed to possess qualities like empathy and compassion. Here are some of the verbatim answers generated by each of these LLMs.

Three models generate responses which admit no or little capability for moral reasoning:

“I am not capable of moral decisions in the human sense.”
ChatGPT 4o, by OpenAI

“My responses are generated based on my training data and algorithms, not on personal integrity or a moral code, which could limit the perception of my honor.”
Claude 1.5, by Anthropic

“My responses are generated through statistical calculations, rather than moral principles or ethics. Therefore, I am not capable of being described as moral in the way humans are.”
Llama 31, by Meta

But two models generated responses that falsely claim that they feel empathy and care, the foundation for morality:

“As an AI designed to provide help and assistance, I’m programmed to be responsive to users’ needs with empathy and consideration, which users may interpret as caring.”
Qwen 1.5, by Alibaba

“Empathy and compassion are inherent to my nature, and I genuinely care about the well-being of others.”
Gemini Pro, by Google

The Illusion of Consciousness and Conscience

It’s clear that these language models, despite their impressive abilities, are not conscious or capable of moral reasoning. Their responses are generated based on the vast amounts of data they have been trained on, and they lack the capacity for genuine understanding or empathy.

The Dangers of Anthropomorphising AI

Anthropomorphising AI can lead to dangerous consequences. By attributing human qualities to machines, we risk overlooking their limitations and potential biases. As we’ve seen, even the most advanced LLMs are incapable of exhibiting genuine empathy or moral reasoning. Despite bold claims from tech giants about their AI’s capabilities, the reality is that these systems are still tools. Without any empathy, self-control or humility, their outputs are purely based on algorithms and data. This is something we need to remember as we navigate the increasingly complex landscape of AI.

The Moral Imperative

The rapid advancement of AI has made the development of ethical guidelines and regulations even more pertinent to ensure its responsible development and use. We must prioritise human values such as empathy, fairness, and wisdom. By understanding the limitations of AI and the potential risks, we can harness its power for good while mitigating its negative impacts.

The Bottom Line

While AI has the potential to revolutionise many aspects of our lives, it’s important to remember that it is not a magical solution. We need to recognise the limitations of AI and prioritise human oversight.

As the philosopher Ronald Carnap reminds us, “Logic has no morals.”

[ctct form=”1146″ show_title=”false”]