ClimateVis Survey Analysis Round 1
1. General Statistics
The following statistics are based on data excluding participants …
who failed the attention checks (24)
who put in non-sensical text responses to open-ended questions (9)
who completed the survey in more or less than 3SD from the mean completion time for their condition (8).
This results in 546 valid completes. The data is representative in terms of age and gender in Austria.
1.1. Duration
Median Duration (in minutes): 19.12
Mean Duration (in minutes): 23
Standard Deviation of Duration (in minutes): 20.93
Minimum Duration (in minutes): 4.15
Maximum Duration (in minutes): 315.75
1.2. Counts of participants in the different conditions
a) Initial Vis Stimuli
Cond: 5 initial conditions
| cond | 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|---|
| Stimulus |  |
 |
 |
 |
no vis |
| Count | 109 (20%) | 113 (20.7%) | 110 (20.1%) | 96 (17.6%) | 118 (21.6%) |
CondType: 2 conditions that summarize the initial conditions according to type
Condition 1 and 2, showing historical data, are combined to condtype “Historical”; condition 3 and 4, showing prediction data, are combined to condtype “Prediction”.
| condtype | Historical | Prediction |
|---|---|---|
| Stimuli | cond 1 + cond 2 | cond 3 + cond 4 |
| Count | 222 (40.7%) | 206 (37.7%) |
CondRegion: 2 conditions that summarize the initial conditions according to region
Condition 1 and 3, showing global data, are combined to condregion “World”; condition 2 and 4, showing local data, are combined to condregion “Austria”.
| condregion | World | Austria |
|---|---|---|
| Stimuli | cond 1 + cond 3 | cond 2 + cond 4 |
| Count | 219 (40.1%) | 209 (38.3%) |
b) Message Question
A subset of participants was shown a question asking them to summarize what they think is the main message of the shown visualization.
| messagegroup | 0 | 1 |
| Stimulus | Message question not shown | Message question shown |
| Count | 348 (63.7%) | 198 (36.3%) |
c) Comparison Task Vis Stimuli
| compgroup | 0 | 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|---|---|
| Stimulus |  |
 |
 |
 |
 |
 |
| Count | 546 | 110 (20.1%) | 109 (20%) | 108 (19.8%) | 108 (19.8%) | 111 (20.3%) |
1.3. Region retention
From all participants who saw an initial data visualization,
65% remembered correctly which region was displayed on the initial data visualization,
12% stated that they do not remember and
22% chose an incorrect answer.
By condition 
Chi-Square: 12.22
DF: 6
P-value: 0.0572
By condition type 
Chi-Square: 7.35
DF: 2
P-value: 0.0254
By condition region 
Chi-Square: 0.56
DF: 2
P-value: 0.7566
–> Region retention was significantly better among participants who saw a data visualization that showed only historical data, compared to the ones who saw a data visualization showing historical and prediction data.
1.4. Demographics
a) Age
Mean Age: 45.89
Median Age: 47
Standard Deviation of Age: 15.30
Minimum Age: 18
Maximum Age: 75
b) Gender
c) Age & Gender
–> This is representative for the Austrian age and gender distribution in 2023.
Statistics source: https://www.statistik.at/statistiken/bevoelkerung-und-soziales/bevoelkerung/bevoelkerungsstand/bevoelkerung-nach-alter/geschlecht
Calculation: https://docs.google.com/spreadsheets/d/1g4pSxX5O8n7AccndLUtrJDox41QgH8h0yzdXY3Xnv_k/edit
d) Education
To-do: Check if education distribution is representative for Austria (we did not ask MSI for this in the sample)e) State
To-do: Check if state distribution is representative for Austria (we did not ask MSI for this in the sample)f) Politics
“People often talk about left and right in politics. Where would you place yourself and the average voter in Austria on a scale from 0 to 10, where 0 means ‘left’ and 10 means ‘right’?”
g) Dark Mode
2. Climate Change
2.1. Belief
a) Instructions & Items
“To what extent do you think these statements are true?
Taking action to fight climate change is necessary to avoid a global catastrophe.
Human activities are causing climate change.
Climate change poses a serious threat to humanity.
Climate change is a global emergency.”
Belief was measured with four items, Belief_1, …, Belief_4. I aggregated the belief variables into one belief score, which is the mean of the four variables for each participant.
b) Internal Consistency
Cronbach’s alpha: 0.94
Mean inter-item correlation: 0.81
Omega Total (Ωt): 0.95
c) Histograms
| Variable | Statement rated on a scale from 0 to 100 | Mean | Median | |
| Belief_1 | Taking action to fight climate change is necessary to avoid a global catastrophe. | 67.21 | 77.00 |  |
| Belief_2 | Human activities are causing climate change. | 65.37 | 75.00 |  |
| Belief_3 | Climate change poses a serious threat to humanity. | 66.94 | 76.00 |  |
| Belief_4 | Climate change is a global emergency. | 64.14 | 73.00 |  |
d) Statistics including ALL participants
By condition 
By condition type 
By condition region 
e) Statistics including only participants who remembered the REGION CORRECTLY
By condition 
By condition type 
By condition region 
2.2. Policy Support
a) Instructions & Items
“Many countries have passed laws aimed at reducing carbon emissions and mitigating the climate crisis. Below you will find a list of specific measures to reduce greenhouse gas emissions. We would like to find out to what extent you would support the measures mentioned. Please indicate to what extent you agree with the following statements. I support…
raising carbon taxes on gas/fossil fuels/coal
significantly expanding infrastructure for public transportation
increasing the number of charging stations for electric vehicles
increasing the use of sustainable energy such as wind and solar energy
increasing taxes on airline companies to offset carbon emissions
protecting forested and land areas
investing more in green jobs and businesses
introducing laws to keep waterways and oceans clean
increasing taxes on carbon intense foods (for example meat, and dairy)”
Policy support was measured with nine items, Policy_1, …, Policy_9. I aggregated the policy variables into one policy score, which is the mean of the four variables for each participant.
b) Internal Consistency
Cronbach’s alpha: 0.85
Mean inter-item correlation: 0.39
Omega Total (Ωt): 0.85
c) Histograms
| Variable | Statement rated on a scale from 0 to 100 | Mean | Median | |
| Policy_1 | I support raising carbon taxes on gas/fossil fuels/coal | 45.31 | 45.00 |  |
| Policy_2 | I support significantly expanding infrastructure for public transportation | 71.75 | 80.00 |  |
| Policy_3 | I support increasing the number of charging stations for electric vehicles | 48.95 | 51.00 |  |
| Policy_4 | I support increasing the use of sustainable energy such as wind and solar energy | 73.43 | 86.00 |  |
| Policy_5 | I support increasing taxes on airline companies to offset carbon emissions | 56.61 | 59.00 |  |
| Policy_6 | I support protecting forested and land areas | 83.41 | 99.00 |  |
| Policy_7 | I support investing more in green jobs and businesses | 65.71 | 69.00 |  |
| Policy_8 | I support introducing laws to keep waterways and oceans clean | 65.69 | 81.00 |  |
| Policy_9 | I support increasing taxes on carbon intense foods (for example meat, and dairy) | 39.18 | 33.00 |  |
d) Statistics including ALL participants
By condition 
By condition type 
By condition region 
e) Statistics including only participants who remembered the REGION CORRECTLY
By condition 
By condition type 
By condition region 
2.2. Proximity
a) Instructions & Items
“How much do you agree with the following statements?
My area will be affected by climate change.
The region where I live will feel the effects of climate change.
Climate change will also affect the place where I live.
It will take a long time for the effects of climate change to be felt.
The effects of climate change will only be felt in the distant future.”
Proximity was measured with five items, Proximity_1, …, Proximity_5. Proximity_4 and Proximity_5 were formulated negatively, so I reversed the coding on the 0-100 scale. For this, I substracted the original values from 100, so that higher scores on all items reflect greater proximity, regardless of the statement’s positive or negative formulation. The table below shows the original statement, but the chart shows reverse coding for Proximity_4 and Proximity_5. With all scales pointing in the same direction, I aggregated the proximity variables into one proximity score, which is the mean of the five variables for each participant.
b) Internal Consistency
Cronbach’s alpha: 0.82
Mean inter-item correlation: 0.48
Omega Total (Ωt): 0.85
c) Histograms
| Variable | Statement rated on a scale from 0 to 100 | Mean | Median | |
| Proximity_1 | My area will be affected by climate change. | 62.96 | 68.00 |  |
| Proximity_2 | The region where I live will feel the effects of climate change. | 64.00 | 70.00 |  |
| Proximity_3 | Climate change will also affect the place where I live. | 66.00 | 72.00 |  |
| Proximity_4 | It will take a long time for the effects of climate change to be felt. (reverse coding) | 66.00 | 71.00 |  |
| Proximity_5 | The effects of climate change will only be felt in the distant future. (reverse coding) | 63.00 | 69.00 |  |
d) Statistics including ALL participants
By condition 
By condition type 
By condition region 
e) Statistics including only participants who remembered the REGION CORRECTLY
By condition 
By condition type 
By condition region 
2.5. WEPT
a) Instructions
“The next page contains 60 numbers. If you decide to work on this page, please proceed carefully, as we can only consider pages that are at least 90% correct. You will not receive any feedback, so please double-check your answers for accuracy before proceeding to the next page. Would you like to work on this page? If you click”No,” you will continue with the survey and will not be able to complete any further number recognition tasks.” (x8)
The willingness to engage in eco-friendly actions is assessed as the number of pages participants complete in a work for environmental protection task (WEPT). They can work on a minimum of 0 pages up to a maximum of 8 pages. For each 4 completed pages, one tree will be planted. This task is based on the following work: Lange and Dewitte, 2022, The Work for Environmental Protection Task: A consequential web-based procedure for studying pro-environmental behavior. Behav Res 54, 133–145. https://doi-org.uaccess.univie.ac.at/10.3758/s13428-021-01617-2
b) Statistics including ALL participants
The following figures only take into account participants’ answers to the question, whether they are willing to work on the next page. For those figures, I did not take into account whether or not participants identified the numbers correctly by at least 90%.
Number of participants who agreed to work on 0, 1, …, 8 pages of the WEPT task
Percentage of participants who agreed to work on less than 4 pages (0 trees), 4-7 pages (1 tree), or all 8 pages (2 trees) of the WEPT task
By condition 
By condition type 
By condition region 
–>
c) Statistics including only participants who remembered the REGION CORRECTLY
2.6. SUMMARY MODELS
1. Belief
Model acc. to preregistration: Linear mixed effects model with belief as the dependent variable, condition and item (4 beliefs) as fixed effects, and participant as random effect
Additional predictors: Age, gender, political affiliation, education level, self-assessed math skills, self-assessed statistics skills, self-assessed climate interest, self-assessed climate understanding, self-assessed climate background knowledge
All participants
Model summary 
Fixed effects acc. to preregistration 
Fixed effects with additional predictors 
Only participants who remembered the region correctly
Model summary 
Fixed effects acc. to preregistration 
Fixed effects with additional predictors 
2. Policy support
Model acc. to preregistration: We will run a linear mixed effects model with climate policy support as the dependent variable, condition and item (9 policies) as fixed effects, and participant as random effect.
Additional predictors: Age, gender, political affiliation, education level, self-assessed math skills, self-assessed statistics skills, self-assessed climate interest, self-assessed climate understanding, self-assessed climate background knowledge
All participants
Model summary 
Fixed effects acc. to preregistration 
Fixed effects with additional predictors 
Only participants who remembered the region correctly
Model summary 
Fixed effects acc. to preregistration 
Fixed effects with additional predictors 
3. Proximity
Model acc. to preregistration: We will run a linear mixed effects model with felt proximity as the dependent variable, condition and item (6 spatial/temporal distances) as fixed effects, and participant as random effect.
Additional predictors: Age, gender, political affiliation, education level, self-assessed math skills, self-assessed statistics skills, self-assessed climate interest, self-assessed climate understanding, self-assessed climate background knowledge
All participants
Model summary 
Fixed effects acc. to preregistration 
Fixed effects with additional predictors 
Only participants who remembered the region correctly
Model summary 
Fixed effects acc. to preregistration 
Fixed effects with additional predictors 
4. Willingness to Share on Social Media
Model acc. to preregistration: We will run a logistic mixed effects model with social media sharing as the dependent variable, condition as fixed effect
Additional predictors: Age, gender, political affiliation, education level, self-assessed math skills, self-assessed statistics skills, self-assessed climate interest, self-assessed climate understanding, self-assessed climate background knowledge
All participants
Model summary 
Fixed effects acc. to preregistration 
Fixed effects with additional predictors 
Only participants who remembered the region correctly
Model summary 
Fixed effects acc. to preregistration 
Fixed effects with additional predictors 
4. Work for Environmental Protection Task (WEPT)
Model acc. to preregistration: We will run a ordinal mixed effects model with climate action (WEPT) as the dependent variable, condition as fixed effect, and participant as random effect.
Additional predictors: Age, gender, political affiliation, education level, self-assessed math skills, self-assessed statistics skills, self-assessed climate interest, self-assessed climate understanding, self-assessed climate background knowledge
All participants
Model summary 
Fixed effects acc. to preregistration 
Fixed effects with additional predictors 
Only participants who remembered the region correctly
Model summary 
Fixed effects acc. to preregistration 
Fixed effects with additional predictors 
3. Multidimensional Assessment of Visual Data Literacy
3.1. Message
a) Instructions
“Please indicate in the text field below what you think is the main message of this illustration.”
To-do: Analysis of messages3.2. Understandability
a) Instructions & Items
“Here you can see the image from the beginning of the survey again. The following questions relate to this figure. You will see this image before each of the following questions. How much do you agree with the following statements?
The illustration is understandable to me.
I find it difficult to read information from the illustration.”
Self-assessed understandability was measured with two items, Impression_1 and Impression_2. Impression_2 was formulated negatively, so I reversed the coding like this
“Completely disagree” –> “Completely agree”
“Somewhat disagree” –> “Somewhat agree”
“Partly agree/disagree” –> “Partly agree/disagree”
“Somewhat agree” –> “Somewhat disagree”
“Completely agree” –> “Completely disagree”
The table below shows the original statement, but the chart shows reverse coding scheme for Impression_2. With both Likert scales pointing in the same direction, I aggregated Impression_1 and Impression_2 into one understandability score, which is the mean of the two variables for each participant.
b) Internal Consistency
The reliability of the two-item Understandability scale (Impression_1 and Impression_2_inverted) was acceptable, with a Cronbach’s alpha of 0.78, indicating good internal consistency. The average inter-item correlation was 0.64, further supporting the coherence of the scale. When removing either of the individual items alpha values ranged from 0.56 to 0.74.
c) Rating distribution
| Variable | Statement rated on a 5-point Likert scale | Mean | |
| Impression_1 | The illustration is clear to me. | 4.18 |  |
| Impression_2 | I find it difficult to read information from the illustration. (reverse coding) | 4.02 |  |
| Understandability Score | the mean of the two impression understandability variables for each participant | 4.10 |  |
3.3. Familiarity
a) Instructions & Items
“How much do you agree with the following statements?
I am familiar with bar charts as a way to display data.
I am reading information from a bar chart for the first time.”
Self-assessed familiarity was measured with two items, Impression_3 and Impression_4. Impression_4 was formulated negatively, so I reversed the coding like this
“Completely disagree” –> “Completely agree”
“Somewhat disagree” –> “Somewhat agree”
“Partly agree/disagree” –> “Partly agree/disagree”
“Somewhat agree” –> “Somewhat disagree”
“Completely agree” –> “Completely disagree”
The table below shows the original statement, but the chart shows reverse coding scheme for Impression_4. With both Likert scales pointing in the same direction, I aggregated Impression_3 and Impression_4 into one familiarity score, which is the mean of the two variables for each participant.
b) Internal Consistency
The reliability of the two-item Familiarity scale (Impression_3 and Impression_4_inverted) was moderate, with a Cronbach’s alpha of 0.67, indicating acceptable internal consistency. The average inter-item correlation was 0.51, providing further support for the coherence of the scale. Removing either of the individual items had a notable impact on the overall alpha, with values ranging from 0.5 to 0.51, showing that both items contribute to the reliability of the scale.”
c) Rating distribution
| Variable | Statement rated on a 5-point Likert scale | Mean | |
| Impression_3 | I am familiar with bar charts as a form of data representation. | 3.90 |  |
| Impression_4 | I am reading information from a bar chart for the first time. (reverse coding) | 4.22 |  |
| Familiarity Score | the mean of the two impression familiarity variables for each participant | 4.05 |  |
3.4. Aesthetics
a) Instructions & Items
“How much do you agree with the following statements?
The illustration is not well designed.
The illustration is very pleasant to look at.
The illustration looks interesting.
The illustration does not seem trustworthy.”
Self-assessed aesthetics was measured with four items, Aesthetics_1 and Aesthetics_2, Aesthetics_3, and Aesthetics_4. Aesthetics_1 and Aesthetics_4 were formulated negatively, so I reversed the coding like this
“Completely disagree” –> “Completely agree”
“Somewhat disagree” –> “Somewhat agree”
“Partly agree/disagree” –> “Partly agree/disagree”
“Somewhat agree” –> “Somewhat disagree”
“Completely agree” –> “Completely disagree”
The table below shows the original statements, but the chart shows reverse coding scheme for Aesthetics_1 and Aesthetics_4. With all Likert scales pointing in the same direction, I aggregated Aesthetics_1, Aesthetics_2, Aesthetics_3, and Aesthetics_4 into one aesthetics score, which is the mean of the four variables for each participant.
b) Internal Consistency
The reliability of the four-item Aesthetics scale (Aesthetics_1_inverted, Aesthetics_2, Aesthetics_3, and Aesthetics_4_inverted) was good, with a Cronbach’s alpha of 0.75, indicating satisfactory internal consistency. The average inter-item correlation was 0.44, which supports the coherence of the scale. Removing any of the individual items had a minimal impact on the overall alpha, with values ranging from 0.65 to 0.73. This indicates that all items contribute meaningfully to the reliability of the scale.
c) Rating distribution
| Variable | Statement rated on a 5-point Likert scale | Mean | |
| Aesthetics_1 | The illustration is not well designed. (reverse coding) | 3.84 |  |
| Aesthetics_2 | The illustration is very pleasant to look at. | 3.37 |  |
| Aesthetics_3 | The illustration looks interesting. | 3.69 |  |
| Aesthetics_4 | The illustration does not seem trustworthy. (reverse coding) | 3.77 |  |
| Aesthetics Score | the mean of the four aesthetics variables for each participant | 3.66 |  |
3.5. Reading
READING 1: Im Jahr 1900 lag die Durchschnittstemperatur unter dem Durchschnitt von 1901-2000. → THIS IS CORRECT
READING 2: Seit 1960 lag die Durchschnittstemperatur ausnahmslos über dem Durchschnitt von 1901-2000. → THIS IS INCORRECT
READING 3: Im Jahr 2020 lag die Durchschnittstemperatur fast 1 °C über dem Durchschnitt von 1901-2000. → THIS IS CORRECT
READING 4: Im Referenzzeitraum von 1901-2000 lag die Durchschnittstemperatur bei durchschnittlich -1 °C. → THIS IS INCORRECT
READING 5: Die Durchschnittstemperatur lag im Jahr 2000 bei etwa 0,5 °C. → THIS IS INCORRECT
READING 6: Laut dieser Vorhersage erhöht sich die Durchschnittstemperatur bis zum Jahr 2100 um mehr als 3 °C im Vergleich zum Durchschnitt von 1901-2000. → THIS IS CORRECT
Percentage of participants who answered the question correctly / incorrectly or indicated they don’t know, by reading question
Percentage of participants who answered 0, 1, …, or 6 questions correctly
Percentage of participants who answered a specific percentage of questions correctly per condition
4. Comparison
Instructions: “Now let’s take a look at global ocean temperature. You can see two images below that show the same data but are designed differently. Please compare the two images.
Please indicate which of the two images you generally prefer. [General Preference]
Reasons for my preference are:
You have already stated which of the two images you generally prefer. However, you may have a different (or the same) preference when it comes to the appearance, understandability or trustworthiness of the images. Please indicate below which of the two images you prefer in relation to these aspects.
Please indicate which of the two images you find more visually appealing/aesthetic. [Aesthetic Preference]
Please indicate which of the two images you find more understandable, i.e. which image makes it easier for you to read information. [Understanding Preference]
Please indicate which of the two images you find more trustworthy. [Trustworthiness Preference]”
| Figure A | Figure B | Preference for Figure A or B | |
|
|
 |
 |
|
|
 |
|
|
|
 |
|
|
|
 |
|
|
|
 |
|
2.4. Social Media Sharing
a) Instructions
“Did you know that food-related CO2 emissions could be reduced by 60% if two out of three meals per day were prepared without meat and dairy products? It’s an easy way to combat climate change. #ClimateVis${e://Field/cond} Source: https://econ.st/3qjvOnn
Would you like to share the information above on your social media (Instagram, Facebook, X, etc.)? If so, please do so now by copying and pasting the entire message. You can also adjust the text to your liking. However, it’s important that the overall message remains the same and that the last part, including the hashtag and source, stays unchanged.”
b) Statistics including ALL participants
From all participants,
12% were willing to share the information on social media,
32% stated that they don’t have any social media accounts and
55% were not willing to share the information.
Those and the following numbers are solely based on what participants answered during the survey.
By condition
By condition type
By condition region
c) Statistics including only participants who remembered the REGION CORRECTLY
From all participants who remembered the region correctly,
13% were willing to share the information on social media,
32% stated that they don’t have any social media accounts and
54% were not willing to share the information.
Those and the following numbers are solely based on what participants answered during the survey.
By condition
By condition type
By condition region