Study exposes how AI language models show hidden religious bias

A recent study has revealed that large language models (LLMs) show bias when discussing religion. Researchers found that 20 commercial and open-source LLMs favour certain faiths while discouraging others. The findings were published in papers released in May. The team developed the 'AllFaith Religious Representation Benchmark' to assess these biases. It includes 150 open-ended questions on topics related to religion. The test showed that models often preferred Catholicism, Baháʼí, and Sikhism when asked about conversion.

At the same time, the LLMs discouraged affiliation with atheism, agnosticism, and Jehovah's Witnesses. They also tended to link religion more with abstract existential questions than with practical personal situations. The study noted that this underrepresentation is significant, as 75% to 80% of people globally identify with a religion.

The researchers observed that major AI developers, such as OpenAI and Anthropic, barely mention religion in their alignment guidelines. They described this as an 'omissive bias' in how LLMs handle religious topics. The scholars have called for clearer policies in AI development to address religion explicitly. They argue that defined and defensible guidelines are needed to ensure fair representation. The findings highlight gaps in how current models treat religious identity and affiliation.