Has the Supreme Court become just another political branch? Public perceptions of Court approval and legitimacy in a post-Dobbs world
Data files
Jan 10, 2024 version files 75.48 MB
-
Figure1_Gallup_Trust.csv
-
Figure2_NYTimes_Content.csv
-
Figure3a_SC_Favorability.csv
-
Figure3b_SC_Trust.csv
-
Figure4_Importance_Nominations.csv
-
Figure5_APPC_2005.csv
-
Figure5_APPC_2006.csv
-
Figure5_APPC_2007.csv
-
Figure5_APPC_2011.csv
-
Figure5_APPC_2013.csv
-
Figure5_APPC_2014.csv
-
Figure5_APPC_2015.csv
-
Figure5_APPC_2016.csv
-
Figure5_APPC_2017.csv
-
Figure5_APPC_2018.csv
-
Figure5_APPC_2019.csv
-
Figure5_APPC_2020.csv
-
Figure5_APPC_2021a.csv
-
Figure5_APPC_2021b.csv
-
Figure5_APPC_2022.csv
-
Figure5_APPC_2023.csv
-
Figure5_TAPS_2013.csv
-
Figure5_TAPS_2014.csv
-
Figure5_TAPS_2015.csv
-
Figure5_TAPS_2016.csv
-
Figure5_TAPS_2017.csv
-
README.md
-
Sample_AIOD_Instrument.docx
-
SI_Table21_2023_Reform.csv
-
SI_Table3-Data.csv
-
SI_Table4_Favorability.csv
-
SI_Table5_Trust.csv
-
SI_Table6_2023_Favorability.csv
-
SI_Table9_Nominations.csv
-
Supplemental_Materials_SA_Resubmission.pdf
-
Table1_Change_Legitimacy_Reform.csv
Abstract
Have perceptions of the U.S. Supreme Court polarized, much like the rest of American politics? Because of the Court’s unique role, for many years it remained one of the few institutions respected by both Democrats and Republicans alike. But the Court’s dramatic shift to the right in recent years—highlighted by its Dobbs decision in 2022—potentially upends that logic. Using both 8 waves of panel data and 18 nationally representative surveys spanning two decades, we show that while there was little evidence of partisan polarization in earlier years, in 2022 and 2023, such patterns are clear in favorability, trust, legitimacy, and support for reform. Factors that used to protect the Court—like knowledge about it and support for key democratic values—no longer do so. The Court has also become more important to voters, and will likely remain a political flashpoint, with disquieting implications for the Court’s place in our polity.
README: Has the Supreme Court become just another political branch? Public perceptions of Court approval and legitimacy in a post-Dobbs world
https://doi.org/10.5061/dryad.m63xsj48k
Description of the data and file structure
Figure 1: Data for Figure 1 comes from publicly available Gallup data:
Supreme Court: https://news.gallup.com/poll/4732/supreme-court.aspx
Congress: https://news.gallup.com/poll/1600/congress-public.aspx
Executive Branch: https://news.gallup.com/poll/4729/presidency.aspx
This figure can be replicated using the Figure1_Gallup_Trust.R script and the Figure1_Gallup_Trust.csv data.
Figure 2: Data for Figure 2 comes from a content analysis of The New York Times coverage of the U.S. Supreme Court. To produce those results, we used the New York Times API, we first downloaded the corpus of news articles from 1 January 2008 – 20 June 2023 (the last date available at the time we performed our content analysis) that included the words “Supreme Court” as a keyword for the article. This proceeds in two steps. The API does not return the full text but does return the full-text URL, so we used the API to generate the set of URLs, and then used the URLs to scrape the full text of the articles. Finally, we randomly selected samples of 100 to check for false positives. After removing a large subset of articles concerning the Supreme Court of India, we were left with approximately 88-92 percent accuracy (i.e., approximately 9 in 10 articles were actually about the U.S. Supreme Court). Given the large number of articles (8,497), the trends are either conservative or likely unaffected by the small portion of false positives. False negatives are extremely unlikely here: it is hard to imagine the Times writing many articles (or even any articles) about the Supreme Court without listing it as a keyword.
In the body of the paper, we discussed various trends in coverage of the Court: coverage of key cases and controversies, ideological/partisan coverage, and discussion of the Court’s legitimacy.
This figure can be replicated using the Figure2_NYTimes_Content.R script and the Figure2_NYTimes_Coverage.csv data.
Figures 3 & 4 and Table 1: These figures and tables rely on Data from the AIOD Panel Study. The AIOD Panel Study is an ongoing survey of voters in Michigan, Pennsylvania, and Wisconsin. Subjects were first recruited to join the panel in late 2019 and early 2020 via address- based sampling, and as such, this data is a random sample of voters in each state; we use post-stratification weights in our analyses to ensure our data are reflective of each state’s population. Throughout the study, panelists haven taken our surveys via a custom web portal hosted by the survey firm SSRS. To ensure that the survey does not miss important groups with low rates of Internet usage, especially those in more rural areas and those with less education, respondents are allowed to take the study via the telephone; 2.3 percent of the sample did so in the most recent wave (this figure has been relatively constant across time). Full details on the recruitment of these panelists, response rates, construction of our post-stratification weights, and so forth, please see reference number 60. Informed consent was obtained for all survey subjects included in this project. The data collection for the AIOD study, as well as the APPC data used later in the paper, was approved by the University of Pennsylvania Institutional Review Board.
During the 2020 election, panelists were interviewed quite regularly—nearly every month. After Trump’s second impeachment trial in February 2021, AIOD reduced the frequency to one survey per quarter, and reduced the sample to approximately one-third of our original size. To avoid biasing the sample, when AIOD reduced the panel size, they did so by randomly sampling from our existing sample, hence the data are a random sample of a random sample.
In the 2021 data used in the paper, respondents were interviewed between October 6th and 12th (replicate 1), November 3rd and 9th (replicate 2), and December 1st and 7th (replicate 3); the dates of the 2022 wave are provided in the body of the paper. Note that replicate 1 is the pre-leak replicate, replicate 2 is the post-leak replicate, and replicate 3 is the post-decision replicate. For more details on the methodology of the panel, see the Appendix of Democracy Amid Crises (2023).
These figures can be replicated using the Figure3_Specific_Support.R and Figure4_Importance_Nominations.R scripts and the Figure3a_SC_Favorability.csv, Figure3b_SC_Trust.csv, and Figure4_Importance_Nominations.csv data files. This table can be replicated using the Table1_Change_Legitimacy_Reform.R script and Table1_Change_Legitimacy_Reform.csv data
Figures 5-7: In our over-time analysis, we draw on surveys conducted by the Annenberg Public Policy Center (APPC) from 2005 to 2023. The 2005 study was conducted by Princeton Survey Research Associates International for APPC. The telephone survey polled 1,504 adults aged eighteen years and older and was conducted between March 16 and April 18, 2005. Later surveys were conducted as part of the APPC’s annual Constitution Day Surveys. These surveys were conducted as part of the SSRS omnibus sample, a national, dual-frame bilingual telephone survey. The SSRS Omnibus sample is designed to represent the adult U.S. population (including Hawaii and Alaska). SSRS Omnibus uses a fully-replicated, single-stage, random-digit-dialing (RDD) sample of landline telephone households, and randomly generated cell phone numbers. Sample telephone numbers are computer generated and loaded into on-line sample files accessed directly by the computer-assisted telephone interviewing (CATI) system.
The 2022 APPC Constitution Day Telephone Survey was conducted for APPC via telephone (CATI) by SSRS, an independent research company. Interviews were conducted August 2 – August 13, 2022, among 1,113 U.S. adults, aged 18 and older. Respondents were drawn from a national probability sample in all 50 states. The dual frame sample included 889 cell phone respondents and 37 respondents who completed the survey in Spanish. Data were weighted to represent the target U.S. adult population. The adjusted margin of error for total respondents in the sample is +/-3.58% at the 95% confidence level. The\
response rate was 3.0% (AAPOR RR 3).
The 2023 APPC Online Survey was conducted for APPC via the SSRS Opinion Panel among U.S. adults ages 18 and older. Data collection was conducted from May 31 to June 7, 2023 among a sample of n=1,004 respondents. The survey was conducted via web (974) and telephone (30) in English (956) and Spanish (48). Data were weighted to represent the target population of U.S. adults ages 18 or older. The survey had a completion rate of 41.76% and a Survey RR3 of 42.33%. The cumulative response rate, accounting for response rate at recruitment, is 2.94%. The margin of sampling error for the complete set of weighted data is ± 3.9 percentage points at the 95% confidence level.
We supplement these data with waves of WashU's Weidenbaum Center's The American Panel Survey (TAPS). More information on the TAPS data can be found at: https://wc.wustl.edu/american-panel-survey
The overtime analysis can be replicated using the Figure5_...R, Figure6_...R, and Figure7_....R scripts and the accompanying Figure5_...csv, Figure6_...csv, and Figure7_...csv datasets.
For the pre-analysis plan, deviations from the plan, further details for both the panel and cross-sectional surveys including question wordings, additional analyses, and a description of the content analysis, please see Supplementary Materials SA Resubmission.pdf. All recodings are documented in the R scripts.
For any questions about the data, please contact Shawn Patterson (shawn.patterson@appc.upenn.edu)
Codebooks
The Supplementary Materials SA Resubmission.pdf provides a summary of the variables used in the analysis. The Sample_AIOD_Instrument.pdf provides an example of the instrument and raw variable labels (b22, k3f, etc.) for the panel. Variables generated for the analyses are annotated within the R script. For more information on the TAPS data and its codebooks, see: https://wc.wustl.edu/american-panel-survey
NA's represent missing data where respondents did not answer or skipped the survey item.
Code/Software
Data was analyzed in R. All code needed to replicate tables and figures are included.
R version 4.3.1 (2023-06-16) -- "Beagle Scouts"
Platform: aarch64-apple-darwin20 (64-bit)