The difficulty of measuring support for democracy in a changing society

Evidence from Russia

Bibliographic Data

ID6254387
AuthorsEllen Carnaghan (Saint Louis University, corresponding author)
Year2011
Volume18
Issue3
Pages682-706
Publication date2011-06-01
Peer ReviewedYes
Open AccessNo
TypeARTICLE
VenueDemocratization (JOURNAL)
Journal identifiersISSN: 1351-0347 • E-ISSN: 1743-890X
PublisherInforma UK Limited (PUBLISHER • GB)
DOI10.1080/13510347.2011.563113
OpenAlexW2055893683
LanguageEN
Citations received13
References cited60

Measuring support for democracy in societies where democratic institutions do not exist or do not function well is a challenge faced by many researchers around the world. In societies moving either toward or away from democracy, the very meaning of 'democracy' is often in question and institutions and practices that go by the label of 'democratic' may vary widely from accepted norms. As a result, respondents are likely to interpret survey questions on democratic concepts in unpredictable ways. This article examines some of the ways respondents in non-democratic or imperfectly democratic countries may misinterpret the meaning of survey questions and consequently how their answers may mislead researchers. Previous research has focused on problems with abstract concepts like 'democracy'. Evidence presented here – from interviews with Russians – shows that the problem is broader and covers more kinds of questions than previously thought. A strong potential for miscommunication also exists with more concrete questions about institutions and values, forced choices that encourage respondents to change the meaning of questions, and questions about trust in institutions. Keywords: Russiademocracysurveyspublic opiniontransitional regimesmeasurementtrust Notes The assumption is that, with a democratically-inclined public, there should be less of a chance that citizens will challenge efforts to build democratic institutions or support candidates who would undermine democracy, and more of a chance that citizens will tolerate short-term problems with new government capacity or resist a reversal back to authoritarianism. See, for example, Inglehart and Welzel, Modernization, Cultural Change; Linz and Stepan, 'Toward Consolidated Democracies'; Welzel, 'Levels of Democracy'. Bishop, Illusion of Public Opinion; Payne, Art of Asking Questions; Schaeffer and Presser, 'Science of Asking Questions'. Funding for this project was provided by a Fulbright-Hays Faculty Research Abroad grant, 1998 (with funds from the United States Department of Education); the American Political Science Association, 1997; and the International Research and Exchanges Board, 1998, 2000 (with funds from the National Endowment for the Humanities, the United States Department of State, and the United States Information Agency). Inglehart, 'How Solid'; Mishler and Rose, 'Political Support'. For example, Bartels, 'Question Order'; Javeline, 'Response Effects'; Kampen, 'Impact of Survey Methodology'. King et al., 'Enhancing the Validity'. But see Canache, Mondak, and Seligson, 'Meaning and Measurement'; Schedler and Sarsfield, 'Democrats with Adjectives'. Schober and Conrad, 'Conversational Interviewing'. Brenner, 'Response-effects', 138–41. For a more detailed list of sources of miscommunication, see Belson, Design and Understanding, 370–87. DeLamater, 'Response-effects', 24–5; Converse and Presser, Survey Questions, 10–25. Tourangeau, Rips, and Rasinski, Psychology of Survey Response, 23–61. Conrad and Schober, 'Clarifying Question Meaning', 2; Sudman, Bradburn, and Schwarz, Thinking About Answers, 58–64. Clark and Schober, 'Asking Questions', 27–8. Bishop, Illusion of Public Opinion, 15. Zaller and Feldman, 'Simple Theory'; Tourangeau, Rips, and Rasinski, Psychology of Survey Response, 165–96. Schuman and Presser, Questions and Answers. Alvarez and Brehm, Hard Choices; Sniderman et al., Clash of Rights, 65; Gibson, 'Sober Second Thought', 820. The challenges of communicating accurately are exacerbated if researchers are working in languages and cultures other than their own, but that is a different issue than the one being examined here. See Braun, 'Communication and Social Cognition'; Harkness, 'Questionnaire Translation', 42; Harkness, van de Vijver, and Mohler, 'Comparative Research', 14; Schwarz, 'Culture-Sensitive Context Effects'; Smith, 'Developing and Evaluating'. Schedler and Sarsfield, 'Democrats with Adjective', 639. Fails, 'Substantive Democratization', 856. Presser et al., Methods for Testing. For example, Bartels, 'Question Order'; Javeline, 'Response Effects'; Kampen, 'Impact of Survey Methodology'; King et al., 'Enhancing the Validity'. Canache, Mondak, and Seligson, 'Meaning and Measurement'. Norris, 'Introduction', 10–11; similarly, Thomassen, 'Support for Democratic Values', 383. Dalton, Shin, and Jou, 'Understanding Democracy'; Dryzek and Holmes, Post-communist Democratization; Miller, Hesli, and Reisinger, 'Conceptions of Democracy'; Schaffer, Democracy in Translation. Schedler and Sarsfield, 'Democrats with Adjectives', 640. Beatty, 'Dynamics of Cognitive Interviewing'. O'Muircheartaigh, 'CASM', 54. Geertz, Interpretation of Cultures; Gerber, 'View from Anthropology', 219; Spradley, Ethnographic Interview. Walsh, 'Applying Norton's Challenge', 354. I conducted nearly all of the interviews myself, in Russian. Four interviews were conducted by a Russian colleague, Ol'ga Iastrebova. Interviews ranged in length from a minimum of about forty-five minutes to over three hours. The interviews were taped, with the permission of the respondent. I also took notes, both as a backup in case the tape recorder failed and to distract the respondent from the tape recorder. The tapes were transcribed in Russian by a native speaker; translations are my own. Most interviewees were offered a small stipend. Although there were a couple of exceptions, most interviewees seemed happy to talk at length and did not appear constrained by the context of the interview, by the tape recorder, or by the fact that they were talking to an American researcher. Blee and Taylor, 'Semi-Structured Interviewing', 100. These are characteristics that have been shown to influence political attitudes in Russia. See Berinsky and Tucker, '"Don't Knows"'; Hough, Davidheiser, and Lehmann, 1996 Russian Presidential Election, 56–7; Colton, 'Determinants of the Party Vote', 87. Interviews were conducted according to a quota sample, based on desired combinations of age, education, gender, and location within Russia. Many of the local researchers who helped me find volunteers were part of the network of survey researchers and trained interviewers developed by Polina Kozyreva and Mikhail Kosolapov of the Institute of Sociology, Russian Academy of Sciences. I am particularly grateful for the assistance of Polina Kozyreva, Tat'iana Bogomolova, Marina Chizhova, Arbakhan Magomedov, Natasha Peshkova, Nina Rostegaeva, Andrei Shashenkov, and Ol'ga Shcheglova. Gel'man, 'Out of the Frying Pan'; Gill, 'A New Turn'; Lipman, 'How Russia is Not Ukraine'; Way, 'Authoritarian State Building'. Payne, Art of Asking Questions, 235. Canache, Mondak, and Seligson, 'Meaning and Measurement'; Schedler and Sarsfield, 'Democrats with Adjectives'. Among the major surveys including questions with the word 'democracy' are the Latinobarometer, the Afrobarometer, and the Asian Barometer. These questions include, 'Democracy is always preferable to any other kind of government'; 'Under some circumstances, an authoritarian government can be preferable to a democratic one'; and 'For people like me, it does not matter whether we have a democratic or a nondemocratic regime.' See Chu et al., 'Public Opinion and Democratic Legitimacy, 77; Weitz-Shapiro, 'Local Connection'; Bratton and Mattes, 'Support for Democracy', 460; Smith, 'Legitimate Grievances'. The World Values Survey includes questions on the political performance of democracy: 'Democracies are not good at maintaining order', and 'Democracies are indecisive and have too much quibbling'. See Sing, 'Public Support for Democracy', 247. Bratton and Mattes, 'Support for Democracy'; Dalton, Shin, and Jou, 'Understanding Democracy'; Miller, Hesli, and Reisinger, 'Conceptions of Democracy'. The World Values Survey includes questions about confidence in parliament and a set of other major institutions. See Newton, 'Trust, Social Capital', 208; also, Gibson, 'Political and Economic Markets'; Miller, Hesli, Reisinger, 'Conceptions of Democracy', 190; Wang, 'Public Support for Democracy'; Schedler and Sarsfield, 'Democrats with Adjectives'. Relying on multiple items has the added advantage of permitting the construction of multi-item scales. With multiple items, it is possible to test the relationship between questions included in the scale, thereby reducing the likelihood that questions that capture something other than what was intended will be included. Rohrschneider, Learning Democracy, 80, 176. Rose and Shin, 'Democratization Backwards'; Colton and McFaul, 'Are Russians Undemocratic?', 11. For more on this point, see Carnaghan, 'Thinking About Democracy'. A similar process arose in terms of attitudes toward capitalism and private property. For some of my respondents, their comments drew from how market institutions were supposed to work. For others, how those institutions actually worked in Russia. See Carnaghan, Out of Order, chap. 5. See Finkel, Sigelman, and Humphries, 'Democratic Values'. Colton and McFaul, 'Are Russians Undemocratic?', 10; Dalton, Shin, and Jou, 'Understanding Democracy', 142; Inglehart, 'How Solid is Mass Support', 52. Whitefield and Evans, 'Support for Democracy'. VTsIOM, 'Informatsiia'; Lapkin and Pantin, 'Russkii poriadok'; Miller, Hesli, Reisinger, 'Reassessing Mass Support', 406. Colton, Transitional Citizens, 248. For instance, see Javeline, 'Response Effects'. Another forced-choice question that elicited a particularly large number of alternative responses was the survey classic of whether the country was moving in the right direction or not. Some respondents thought that Russia was standing still in need of a push, moving in many directions at the same time, or zigging and zagging through history. One of the more thoughtful respondents wondered whether anyone could tell in the present what the right direction was. Mishler and Rose, 'Origins of Political Trust'; Norris, 'Introduction'. These questions are: 'How much of the time do you think you can trust the government in Washington to do what is right – just about always, most of the time, or only some of the time?', 'Do you think that people in government waste a lot of the money we pay in taxes, waste some of it, or don't waste very much of it?', 'Would you say the government is pretty much run by a few big interests looking out for themselves or that it is run for the benefit of all people?', 'Do you think that quite a few of the people running the government are crooked, not very many are, or do you think hardly any of them are crooked?'. Some of the studies that use variants of these questions are Alwin et al., 'Comparative Referential Structures'; Miller, 'In Search of Regime Legitimacy'; Klingemann and Fuchs, Citizens and the State. George Bishop (Illusion of Public Opinion, 15) identifies the first of these questions as the sort of vague question that means different things to different people. Payne (Art of Asking Questions, 165) suggests that 'government' is a word that may mean different things to different people. See Miller, 'Political Issues and Trust'; Citrin, 'Comment'; Citrin and Luks, 'Political Trust Revisited'; Levi and Stoker, 'Political Trust and Trusworthiness'. Rose, Mishler, and Munro, Russia Transformed, 139. Similarly, Russell Hardin ('Do We Want Trust in Government') notes that citizens do not know enough about officeholders or their incentives to be able to interpret their actions; consequently, citizens do not know enough to trust officeholders. Some respondents still hesitated to apply the word 'trust' to a group of people, only some of which they knew. One young man in Smolensk said, 'There are people whom I know that I trust. There are those whom I know that I don't trust. Trust only arises as a result of personal acquaintance'. For example, Mishler and Rose, 'Trust, Distrust and Skepticism; Mishler and Rose, 'Origins of Political Trust'; Miller and Listhaug, 'Political Performance'; Citrin and Muste, 'Trust in Government'. Newton, 'Trust, Social Capital'; Sarsfield and Echegaray, 'Opening the Black Box'; Mattes and Bratton, 'Learning about Democracy'; Mishler and Rose, 'Origins of Political Trust'. Fails ('Substantive Democratization') argues that citizens who judge the regime to be more successful in substantive democratization are less likely to be committed democrats, an unexpected finding that he attributes in part to the effect of living in imperfectly democratic regimes

Democracy · Law and economics · Political economy · Political science · Politics · Sociology · Electoral Systems and Political Participation · Law · Social Capital and Networks · Social Media and Politics

Tools

Open DOISci-Hub