Jantanee Runpranomkorn/Shutterstock

The Bible Society recently published a report claiming that church attendance in England and Wales increased by more than half between 2018 and 2024. The revival was especially striking among young men, with reported church attendance jumping from 4% to 21% over this short period.

As a quantitative social scientist who has studied religious change in modern societies for more than 25 years, I’m surprised – and sceptical. I do not doubt that the Bible Society acted in good faith, but they haven’t engaged with the mountain of evidence, some of it very recent, pointing to religious decline.

The annual British Social Attitudes survey – widely regarded as the best and most reliable source of data on such matters – shows that the share of adults in England and Wales who said that they were Christian and went to church at least monthly fell by nearly a quarter (from 12.2% to 9.3%) between 2018 and 2023, the last year available. The Bible Society surveys suggest that churchgoers were 8% of the adult population in 2018 and 12% in 2024.

The main Christian denominations (Anglican, Catholic, Methodist, Baptist) conduct and publish their own attendance counts every year. Those show that while churchgoing continues to rebound from the lows of the COVID lockdown, attendance at worship services remains substantially lower than it was in 2019, before the pandemic. In the Church of England, average weekly attendance is down about 20% from pre-pandemic levels, and the story is similar in other denominations.

Get your news from actual experts, straight to your inbox. Sign up to our daily newsletter to receive all The Conversation UK’s latest coverage of news and research, from politics and business to the arts and sciences.

The Bible Society report claims that “Catholicism has risen sharply.” According to their figures, Catholics were 23% of churchgoers in 2018 and 31% in 2024. As total churchgoing supposedly increased by 56% over that period, from 3.7 million to 5.8 million, the implication is that Catholic mass attendance has more than doubled.

We know from the Catholic church itself, however, that the reality is far different. The Catholic Bishops’ Conference of England and Wales counted 701,902 people attending Sunday mass in 2019. In 2023, there were 554,913 – a drop of 21%.

The findings are also inconsistent with other data from YouGov, the polling firm that collected the data for the Bible Society. A decade ago, the British Election Study (BES) commissioned YouGov to create an online panel. This panel, which includes more people than the Bible Society surveys, was asked about religious affiliation and church attendance in 2015, 2022 and 2024.

According to YouGov’s data for the BES internet panel, the share of Christian churchgoers in England and Wales declined from 8.0% to 6.6% between 2015 and 2024, whereas YouGov’s surveys for the Bible Society apparently show an increase from 8% to 12% between 2018 and 2024.

The fact that the findings were completely different in the two cases suggests that this kind of polling is not a reliable way of measuring trends in church attendance.

What could be the problem with the data?

Gold standard social surveys are based on random (probability) samples of the population: everyone has a chance to be included. The British Social Attitudes survey is one such example – and found that churchgoing fell by nearly a quarter from 2018-23.

By contrast, people opt in to YouGov’s survey panel and are rewarded after completing a certain number of surveys. The risk of low-quality or even bogus responses is considerable.

YouGov creates a quota sample from its large self-selected panel. The sample will match the population on a number of key characteristics, such as age and sex, but that does not make it representative in all respects. As quota samples do not give each person in the population a known chance of being selected, statistical inference is not possible and findings cannot be reliably generalised.

To write (as in the Bible Society report) that because thousands of people participated in the two surveys, they “give a 1% margin of error at a 99% confidence level” is misleading.

This study is not the first time such non-probability sampling has led to dubious findings. In late 2023, the Economist ran the story that one in five young Americans believed that the Holocaust was a myth, based on another YouGov poll. A study by the Pew Research Center showed that that finding was almost certainly fallacious, and the Economist added a disclaimer acknowledging the problem.

The trouble with young adults

The Bible Society claims that the alleged religious revival is being driven by young people flocking to church (and reading their Bibles). There are numerous reasons to be sceptical of survey findings about young adults. They are what survey researchers call a hard-to-reach population. They tend to be in transition between the parental home and education or employment; they are often out of the house and difficult for interviewers to find or for online survey companies to recruit.

Those who do respond to surveys may not be representative of their age group. They are more likely to be living with their parents, less likely to be out with friends, more likely to be compliant, less likely to be suspicious of authority, and so on. Such characteristics are associated with religious participation.

The Bible Society report claims 21% of men aged 18-24 are regular churchgoers.
Yuri A/Shutterstock

Other findings from the report are also surprising. The Bible Society asserts: “Men are now more likely to attend church than women.” Most churchgoers would probably be surprised by this news, which would make England and Wales an exception to the religious gender gap present in most western countries. For example, recent research by Pew in the US has found that, although the gender gap is less pronounced among the youngest adults, “women remain more religious than men … by a variety of measures”.

It would be fascinating to probe all of these issues further, but regrettably the Bible Society has not published the dataset. (When contacted about this, the Bible Society pointed to aggregate statistics published by YouGov and said it plans to publish more summary tables in the coming months.) Open access to all data is now a basic expectation in scientific work.

That the Bible Society report has generated some enthusiastic coverage is not surprising – it appears to challenge conventional wisdom, and there are plenty of anecdotes to be provided as supporting “evidence”.

But this doesn’t mean the data should be taken at face value. We need to place more trust in surveys based on probability sampling and less in data collected from opt-in online panels. That’s particularly the case when people are pushing a story that runs counter to everyday experience – and years of data.

In response to the arguments made in this article, the Bible Society said it was committed to producing rigorous and high-quality research that equips the church and provokes conversation in culture. “We are well aware of the limits of non-probability panels, but also the demonstrated strength of this method in producing valid and actionable insights when paired with quota controls and post-stratification, as widely acknowledged in existing survey methodology literature according to academic standards. [Our data] points to both increased engagement with Christianity and a changing spiritual atmosphere, and we are happy to acknowledge it may be on the upper end of a range that future data sets will nuance.”

A spokesperson for YouGov said: “YouGov’s methodology is robust. We have a proprietary panel of millions of people to take part in our surveys. YouGov draws a sub-sample of the panel that is representative of British adults by range of demographic factors, and invites this sub-sample to complete a survey.”

David Voas does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond his academic appointment.