Panel Effects: Do the Reports of Panel Respondents Get Better or Worse over Time?

It is well established that taking part in earlier rounds of a panel survey can affect how respondents answer questions in later rounds. It is less clear, however, whether panel participation affects the quality of the data that respondents provide. We examined two panels to investigate how participation affects several indicators of data quality—including straightlining, item missing data, scale reliabilities, and differences in item functioning over time—and to test the hypotheses that it is less educated and older respondents who mainly account for any panel effects. The two panels were the GfK Knowledge Panel, in which some respondents completed up to four rounds measuring their attitudes toward terrorism and ways to counter terrorism, and the General Social Survey (GSS), in which respondents completed up to three rounds with an omnibus set of questions. The two panels differ sharply in terms of response rates and the level of prior survey experience of the respondents. Most of our comparisons are within-respondent, comparing the answers panel members gave in earlier rounds with those they gave in later rounds, but we also confirm the main results using between-subject comparisons. We find little evidence that respondents gave either better or worse data over time in either panel and little support for either the education or age hypotheses.