Yale Climate Opinion Maps for Strategists

Methods

This site provides estimates of U.S. climate change beliefs, risk perceptions, and policy preferences at the state and local levels – a new source of high-resolution data on public opinion that can inform national, state and local decision-making, policy, and education initiatives. The estimates are derived from a statistical model using multilevel regression with post-stratification (MRP) on a large national survey dataset (n>22,000), along with demographic and geographic population characteristics.

Validating models is essential for producing accurate results. Our estimates were validated using three different methods. First, cross-validation analyses were conducted within the dataset. The dataset was divided into two sets of respondents, with one part used to run the model and the other kept aside for validation. The model estimates were then compared to the results of the set aside respondents to directly quantify the percentage of correct answers the model predicted. These cross-validation tests were repeated multiple times using different sample sizes and dividing the data in different ways. Second, the model estimates derived from the full dataset were compared to the results of independent, representative state- and city-level surveys conducted in California, Colorado, Ohio, Texas, San Francisco, and Columbus, Ohio in 2013. The mean absolute difference between model estimates and validation survey results was 2.9 percentage points (SD = 1.5) among the four states (CA, TX, OH, CO) and 3.6 percentage points (SD = 2.9) among the two metropolitan areas (Columbus, OH, and San Francisco, CA), well within the margins of error for the survey results alone (at a 95% confidence level). Estimates have also been validated internally through a series of technical simulations. Third, some model estimates were compared with third-party survey data collected by other researchers in previous years.

For the 2018 model estimates, uncertainty ranges are based on 95% confidence intervals using 999 bootstrap simulations. These confidence intervals indicate that the 2018 model is accurate to approximately ±7 percentage points at the state and congressional district levels, and ±8 percentage points at the metro and county levels. Such error ranges include the error inherent in the original national surveys themselves, which is typically ±3 percentage points.

The availability of 2014, 2016, and 2018 estimates prompts questions about what has changed between these years in particular places. From our surveys, we know that public opinion on some questions has shifted over this period at the national level. However, our statistical model, as well as the census and election data that go into our model, have also all been updated for accuracy. Thus, estimated shifts in public opinion at the subnational level since 2014 may be due not only to opinion changes, but also changes in demographics and other variables that we are using to model opinions. In addition, our model tends to be conservative when making estimates, especially for places with smaller populations (where there are likely to be fewer survey respondents), so variations over time between states, counties, etc. may be underestimated. As a result, we recommend caution when interpreting any changes over time using different versions of our model, because we can’t distinguish between what is a true shift in actual opinion between 2014, 2016, and 2018, and what reflects the model’s improved ability to map climate opinions at the local scale.

For more details, please see the peer-reviewed paper: Howe, P., Mildenberger, M., Marlon, J.R., and Leiserowitz, A., “Geographic variation in opinions on climate change at state and local scales in the USA,” Nature Climate Change. DOI: 10.1038/nclimate2583.