Do Moral Judgment and Reasoning Capability of LLMs Change with Language? A Study Using the Multilingual Defining Issues Test

This paper explores the moral judgment and moral reasoning abilitiesexhibited by Large Language Models (LLMs) across languages through the DefiningIssues Test. It is a well known fact that moral judgment depends on thelanguage in which the question is asked. We extend the work of beyond English,to 5 new languages (Chinese, Hindi, Russian, Spanish and Swahili), and probethree LLMs – ChatGPT, GPT-4 and Llama2Chat-70B – that shows substantialmultilingual text processing and generation abilities. Our study shows that themoral reasoning ability for all models, as indicated by the post-conventionalscore, is substantially inferior for Hindi and Swahili, compared to Spanish,Russian, Chinese and English, while there is no clear trend for the performanceof the latter four languages. The moral judgments too vary considerably by thelanguage.