To communicate effectively about climate change and to design public policies aimed at mitigating and adapting to it, it is crucial to understand public perceptions of this phenomenon. Researchers seek this understanding by quantifying two main dimensions: climate awareness and climate literacy. Awareness refers to concern about and belief in climate change, while literacy encompasses the knowledge, attitudes, and skills that lead to informed decision-making and action. Climate awareness and literacy vary across geographical regions and demographic groups and correlate with individual factors such as physical exposure to the phenomenon and educational attainment.
A large body of research is investigating awareness and literacy across different groups. The most comprehensive and international studies include the Sulitest and The People’s Climate Vote. Many others focus on specific interest populations around the globe, ranging from national populations to age groups and rural versus urban populations. These studies often analyze the group’s perceptions across different sub-dimensions of awareness and literacy, such as the causes and effects of climate change, energy, economy, transportation, and agriculture. In these surveys, response categories are pre-determined and include a range of textual answers or incremental values through which respondents express their fear or concern about, interest in, and opinions on these topics. For example:
“How serious of a threat is global warming to you and your family?” Response categories included: Not at all serious, Not very serious, Somewhat serious, and Very serious.
This survey method allows for the comparability of answers among individuals within a group and among different interest groups. However, by pre-determining topics of interest and standardizing answers expressing individuals’ perceptions of climate change, this type of analysis may not identify all concerns and interests that are specific to each group and may not identify knowledge gaps critical to understanding climate change awareness and literacy, which in turn are essential to inform outreach efforts.
We propose a novel method to gauge the public perception of climate change by analyzing the questions posted to ClimateQ&A, a tool that Ekimetrics’ innovation lab developed to make IPCC and IPBES reports more accessible. The tool offers a chatbot interface that gathers over 14,000 pages of knowledge from the two institutions and uses LLMs to generate answers adapted to the user’s language and level of understanding. The answers systematically quote sources to trace content back to the original document.
Our method consisted of extracting a sample of the questions posted to the platform to analyze the users’ topics of interest and concerns. The originality of our approach stems from the fact that these topics and concerns are captured with the questions that users can formulate freely, with the ‘space’ to express themselves in an open chat.
To perform our analysis, we first removed from the sample the questions suggested by the platform and the ones that were too short to identify relevant topics. We used classical open-source NLP techniques (BERTopic) to categorize questions across 130 salient topics. We conducted two different analyses by manually reviewing each cluster:
In our sample, the most prevalent topic was Climate Change and GHGs (43.1%), well ahead of Biodiversity (2.9%). This may stem from the fact that the tool emphasizes climate-related queries (e.g., through the naming ‘ClimateQ&A’ and the suggested questions) and could also be attributed to the lower coverage of biodiversity in mass media and public discourse compared to climate change.
Another interesting finding was that our intent analysis revealed that up to 25.8% of questions express personal interrogations regarding nature phenomena as defined above, suggesting that users often want to know how they are concerned and affected personally by these phenomena.
These findings imply that social surveys aimed at understanding public perceptions of climate change and biodiversity loss may benefit from complementary alternative techniques where users can freely express themselves. This allows for the identification of topics of interest and personal concerns. They also imply that IPCC and IPBES reports, intended for policymakers, may not meet this demand for science-based information and may affect awareness and understanding of these issues. Therefore, these organizations could consider alternative communications efforts to engage with broader audiences.
To continue gathering insights on public perceptions of climate change, we are considering extending the sample of analysis, improving our topic classification, and tracking the frequency of occurrences of each topic across time.
Further, improving the tool on several fronts would be beneficial, for instance by including additional data sources from the two institutions.
Additional content
