Skip to main content

Roper Center Transparency and Acquisition Policy​​​​​​​

Introduction

This document articulates the Acquisition Policy of the Roper Center for Public Opinion Research at Cornell University within the context of the Center’s mission. The Acquisition Policy is designed to provide general guidance to the staff as they review potential acquisitions. The policy should be viewed not as a static document but one that must be reevaluated against current survey research methodologies, changing technology, and the Center’s overarching mission.

Mission and Purpose

The overall direction and goals of the acquisition program are governed by the Roper Center for Public Opinion Research mission, which is “to facilitate research on public opinion by faculty, staff, and students at many universities, researchers in the public and private sectors, journalists, policymakers, and private citizens.” A major goal of this mission is to acquire, maintain and make accessible to users the largest collection of polling and survey interview data of broad social, political, economic and cultural significance, potential historic or methodological interest.

Institutional Clientele

The Center provides a wide variety of information, tools and services to its public, commercial and academic users. User experience in the analysis and interpretation of survey results varies widely among these core constituents, a group that includes students, academic researchers, survey researchers, survey professionals, journalists, public policy analysists, and the general public. To fulfill its mission the Center provides introductory, aggregated data as well as more detailed information and tools

The diversity of Center users and uses directly affects the desired type (interpretive reports, marginal data), format (aggregate data, respondent-level data), and currency (new data for existing collections, older data collections not yet acquired) of Center acquisitions.

Acquisition Categories

Two broad categories are included in the scope of the Center’s acquisition efforts.

  1. A primary effort is made to acquire respondent-level data from surveys that may or may not have been released publicly by the original data producer or sponsoring organization. This constitutes the Roper Center’s core collection. Studies in this category offer the unrivaled potential for secondary analysis. The Center was founded, principally, on the acquisition and re-analysis of survey data in this format. This acquisition category consists of two collections based on methodological approaches: Longstanding Methods and Recently Developed Methods.
  2. A second category includes interpretive survey reports, marginal data, and news releases. These can include physical or digital materials.  These collections supplement the respondent-level data referred to above, and in some cases, provide the only preserved record of the survey. For surveys conducted using Longstanding Methods, these materials are entered into the online polling database regardless of whether respondent level data is acquired.  For Recently Developed Methods, these materials are only entered into the database if individual-level data are also provided.

While the Center was built upon the acquisition of respondent-level data, the importance of acquiring, preserving and making accessible marginal data cannot be underestimated within the context of the Center’s mission and the diversity of its clientele. Moreover, full marginal data constitute essential documentation of the respondent-level data.

Acquisition Criteria

In addition to the type and format of the material, potential acquisitions must be measured against a set of broad criteria. Factors to be considered include:

Subject Matter

Collections must be of current value or potential historical interest.

Sample Populations

Sample populations will act as a guide for both acquisition and processing efforts. Material is generally selected in the following descending order of priority:

  • United States national population probability samples
  • Representative subsamples of US national populations (e.g., women, teenagers)
  • US state exit polls or other state representative polls not in ‘national’ collections
  • Representative samples of other countries and regions

Basic Data Source and Methodology: Longstanding Methods Collection

Survey and poll data must be derived from interviews as the primary source. Interview data cannot be from exclusively self-selected respondents; sampling procedures must be probability-based. If surveys are conducted by telephone, trained live interviewers must be used.

Disclosure of Methodology: Longstanding Methods Collection

Polling methodology must meet the basic disclosure criteria:

  1. Information about who sponsored the survey, including all funding sources
  2. Field work provider, if outsourced
  3. The exact wording of questions asked, including the text of any preceding instruction or explanation to the interviewer or respondents that might reasonably be expected to affect the response.
  4. A definition of the population under study, and a description of the sampling criteria used to identify this population.
  5. A description of the sample selection procedure that gives a clear indication of whether or not the researcher selected the respondents or they were self-selected.
  6. The size of samples and, if applicable, completion rates and information on eligibility criteria and screening procedures. For the purpose of this policy statement response rate definition and calculation will be informed by the AAPOR report entitled “Standard Definitions.” The report defines standardized measures for response rates, cooperation rates, refusal rates and contact rates. Each of these rates requires a count of the disposition of all units selected in the sample. The various disposition codes are standardized in the report.
  7. There should be a discussion of the precision of the findings, including, if appropriate, estimates of sampling error, and a description of any weighting or estimating procedures that were used.
  8. A listing of which results are based on parts of the sample, rather than on the total sample.
  9. The mode, location, and dates of data collection.

Basic Data Source and Methodology: Recently Developed Methods Collection

The Recently Developed Methods Collection can include surveys conducted using nonprobability methods or other approaches that are not classified as Longstanding Methods. The studies must still be intended to represent a broader population beyond the individuals who participated.

Disclosure of Methodology: Recently Developed Methods Collection

Polling methodology must meet the basic disclosure criteria:

  1. Information about who sponsored the survey, including all funding sources
  2. Field work provider, if outsourced
  3. The population of which the results are said to be representative, and the justification for this research claim
  4. Full survey questionnaire with all instructions, prompts, visual aids
  5. Full survey dataset with sample weighting variable(s)
  6. Universe
  7. Sample frame (i.e., description of the universe from which the sample was drawn)
  8. Respondent Selection Criteria
  9. Sampling method: e.g., Probability, non-probability, or hybrid
  10. Mode: RDD telephone, IVR; listed-sample telephone with live interviewers; listed-sample telephone via IVR; other telephone (describe); opt-in online panel; other online (e.g., river samples, mobile apps; hybrid or other (describe))
  11. Unweighted sample size
  12. Interview dates
  13. Survey language(s)
  14. Response Rate/Participation Rate: Response rate calculated to AAPOR standards, or sample disposition data adequate for the calculation of AAPOR-standard response rates. When AAPOR-standard response rates or sample disposition data cannot be calculated or provided, completion or participation rates shall be provided using another method that is fully disclosed
  15. Description of all sample weights and sources of weighting targets

Disclosure of the following items also is encouraged:

  • The proportion of that universe that had a nonzero chance of participation 
  • Use of incentives
  • Breakoff rate (i.e., the percent of respondents who start the survey but do not finish it)
  • Use of survey routers or chains
  • Details of quality control checks (e.g., for logic, speeding, straightlining), including how they were performed and results of those checks, including percent of completed interviews excluded or dropped from the analysis
  • Sample provider(s), and, if multiple, the share of the sample from each provider

Center staff will review submissions of data collected through recently developed methods on three criteria:

  • Whether the donor claims that the sample is representative (required for consideration)
  • The face validity of this and other research claims
  • Compliance with disclosure requirements

Submissions of data collected using recently developed methods that do not pass these disclosure criteria will not be made available to any users, but can be archived with the Roper Center for permanent preservation.  The Longstanding Methods and Recently Developed Methods Collections will be presented to users as two separate databases, following display recommendations made by the Transparency and Data Acquisitions Committee of the Roper Center for Public Opinion Research Board of Directors.

Additional Acquisitions Considerations

Risk of Data Loss

Material may be acquired to mitigate the risk of data loss.

Availability of Data Elsewhere

Data may be acquired because the information is not available from another reputable, accessible data archive.

Technical Criteria that May Affect Acquiring Material Include:

  • The general condition of the study/collection and whether it requires costly methods to preserve or make it accessible
  • Whether there are any specific restrictions that would not allow the material to be freely accessible to researchers (however, the Center will occasionally accept “embargoes” on certain questions for limited periods of time, and it will accept a restriction that, for a limited period of time, where data may be disseminated only with the academic membership of the Center)
  • Whether the cost of processing materials is higher than normally expected

What Will Not Be Acquired

There are no firm rules concerning the kinds of survey material that will not be acquired. However, the following list represents a compilation of current practices at the Center and may serve as a guide to the staff and to the Committee as it implements this Acquisition Policy. There have been and will again be exceptions to some or all of these customary practices.

  • Surveys without content that is salient to the Center’s Mission and Purpose, such as many marketing studies of single products
  • Surveys with sample sizes that are too small to represent any population
  • Surveys which are not intended to represent a broader population beyond the individuals who participated.
  • Surveys where release of the respondent-level data would pose an unreasonable and uncontrollable risk of violation of respondents’ privacy and confidentiality
  • Data resulting from “push polls” and other uses of survey methodologies to gather data for purposes other than legitimate survey research
  • Surveys for which the data collector will not provide the full questionnaire or required methodological information.

Roles and Responsibilities

The Acquisition Policy recognizes that professional judgment is central to making decisions about new acquisitions. The broad criteria described above will serve as a guide for archive staff in their continuing efforts to build unique and valuable data collections. The Center’s Executive Director will provide final approval of all new acquisition efforts to oversee appropriate interpretation of these guidelines and safeguard against over-committing Center resources. Beyond this document, the Transparency and Data Acquisitions Committee will serve in an advisory capacity to Center staff. To facilitate this role, archive staff will provide a periodic report on recent acquisitions and rejections. In specific cases where there are questions regarding the implementation of this Policy, Center staff will call upon the Acquisitions Committee for its advice and recommendations.

Review

Given the rapid evolution of technology and its implications for survey research methodologies, the Acquisitions Committee and Center staff will review this policy annually to ensure that it continues to serve the Center’s needs.

 

Approved by the Roper Center for Public Opinion Research Board of Directors June 22, 2018