Rank Scholarship

Previous articleNext article FreeEditorialRank ScholarshipDavid PostDavid PostPDFPDF PLUSFull Text Add to favoritesDownload CitationTrack CitationsPermissionsReprints Share onFacebookTwitterLinked InRedditEmailQR Code SectionsMoreTake each man's censure, but reserve thy judgment.Costly thy habit as thy purse can buy,But not express'd in fancy; rich, not gaudy;For the apparel oft proclaims the man,And they in France of the best rank and stationAre most select and generous, chief in that.(Hamlet, 1.3)O, my offence is rank, it smells to heaven;It hath the primal eldest curse upon't,A brother's murther! Pray can I not,Though inclination be as sharp as will.(Hamlet, 3.2)Along with an immortal theater, Shakespeare's other eternal achievement was to fix our language for the ages when movable type set his blank verse, so propagating 30,000 unique English words.1 One way the Bard did this was through homonyms, as illustrated in the two meanings of "rank," from act 1 and act 3 of Hamlet. Both meanings persist. It is true that today's tenure committees and acquisition librarians, not to mention the contributors to this journal, are more familiar with "rank" as it was conceived in act 1. But Shakespeare's conception of "rank" in act 3 also is becoming less archaic, thanks to Thomson Reuters ISI's propagation of impact factors and the collusion of four movements that we wish to discuss in this year's editorial column: the (irreversible?) rationalization of expertise, as a taken-for-granted feature of bureaucratic authority; the politics of higher education regulation and control, as manifest in the new managerialism and associated research assessment exercises; the pricing and finance of commercial scholarly publishing, which takes advantage of the preceding developments by socking it to university libraries, soaking up most of their budgets; and the increasingly staged drama that editors and their journals are expected to produce, even when they see themselves as hosts of conviviality and thought, as opposed to line employees in CV manufactories. After touching on these four movements, we situate the Comparative Education Review (CER), consider the options, and then immodestly suggest ways to promote a more vital and engaged educational research community than what looks to be its fate from some publications, where the titles of many articles are destined primarily for authors' resumes. We specifically suggest a means to judge the quality of scholarly journals that could be used as an alternative, or supplement, to the metric of the impact factor alone, by considering articles as the by-products of scholarly communication. We advocate that journals and readers attend to the intrinsic value of that communication as the most fundamental product. Finally, we turn our attention to the contributions that scholars have made to our field through their printed, published-on-line, and in-process communications, in this and in forthcoming CER issues.One sign of rank and weedy scholarship is the lengthening reference lists found in most journals over the years. Obviously, there are reasons for references, but we have begun to suggest that citations not be sprinkled about to convey encyclopedic familiarity with an entire field. Accordingly, many authors who publish in the CER are advised to link the author whose work they cite to their point in mentioning the reference. Often we ask authors to specify a page or an argument from the source cited. Most authors take this advice when revising, but one recent exchange was breathtaking in its honesty. In response to the complaint (in this case by an external reviewer, not the editor) that the authors had engaged in name-dropping, the authors wrote back with their revision: "As citation is becoming increasingly important especially in those higher education systems which face research assessment exercises of some form or other, we decided to cite rather than ignore contributions by key authors associated with particular points." This response may reflect a broader sentiment, and so it prompts us now to consider how the new audience of research assessors has changed journal production. The backdrop, perhaps even the stage for these changes, is the long-term restructuring of authority and the need for legitimated expertise. Research assessment exercises, impact factor analysis, and journal ranking are enacted on this stage, not only by governments and universities but also by entrepreneurs and commercial publishers. This drama leads to a struggle by libraries to buy books with their diminishing acquisition budgets, rather than to pay ever-higher prices for site licenses to ever-more journals, which are now controlled by a few key businesses. Moving amid these opportunities and contested demands are journals and their editors.Certified Expertise: The Modern ResearcherMax Weber held that rational bureaucratic structures of domination advance independent of the knowledge areas over which they hold authority. Weber foresaw that the dynamic growth in the certification of expertise would be independent of the growth in the knowledge over which bureaucratic authority had been extended. "When we hear from all sides the demand for an introduction of regular curricula and special examinations, the reason behind it is, of course, not a suddenly awakened 'thirst for education' but the desire for restricting the supply of these positions and their monopolization by the owners of educational certificates" (Weber 1958, 240–41). "Gate keeping," as a concept, if not a term, originated out of the theory of bureaucratic authority.Other theorists from different traditions excavated the underpinnings of supposedly "universal" knowledge and also found institutional constructions. Knowledge about clinical medicine did not eventuate in the creation of medical clinics in nineteenth-century France, according to Michel Foucault ([1963] 1994). To the contrary, widespread and taken-for-granted facts about human anatomy were by-products of the rise of clinics. "The clinic is both a new 'carving up' of things and the principle of their verbalization in the form which we have been accustomed to recognizing as the language of 'positive science'" (xviii).2 Peter Berger and Thomas Luckmann (1966) similarly theorized the ways that reality becomes taken for granted as it is institutionalized. As we get closer to the emergent field of comparative education, beginning in the 1960s John Meyer proposed that schools create "graduates" possessing socially legitimate "diplomas," partly because of an implicit and informal charter that societies assign to these institutions. He subsequently elaborated on this theory and distinguished his argument from a purely credentialist interpretation of Weber.3 Public recognition of the expertise of economists and psychiatrists, Meyer argued, results partly from the expansion of the university institutions and the invention of authoritative knowledge and degrees in these fields. There are clear implications for the understanding of scholarly journals. If personnel are not only sorted but also granted new expertise by educational institutions, then "top" journals form parts of systems to define the knowledge they stamp out in the name of the experts who review or edit their publication. Through journals, discoveries can be "authorized" in the Weberian sense of authority. Even where there are distinctive systems of noncommensurable knowledge—think of Hong Kong's complementary university faculties of Chinese and Western medicine—institutionalized knowledge is hierarchical by nature. Thus, within each medical system, there are perceptions of "rank."New ManagerialismThe advent of quality assessment in higher education is an aspect of the "new" managerialism, a movement that applied to public enterprises, and to the production of public goods, the management techniques previously used in private enterprise.4 The global diffusion of ideas about the regulation of schools and universities has been widely discussed (see Welch 1998). Within this larger movement in higher education, the most relevant feature for scholarly journals, and for this discussion, is that government financial support has incorporated explicit guidelines and funding formulas to reward research productivity.5 "Productivity" has been measured not only by the sheer number of experiments conducted and investigations completed but also by the publication of the results of this research, now conceived as the "product." Publication has further been defined in various categories, including peer-reviewed publications such as the CER. There are cases—for example, Hong Kong, according to Ka Ho Mok (2000, 160)—in which the "publish or perish" syndrome affects not only individual employees but also their employers because institutional funding is tied to research productivity.Peer-reviewed publication has been further categorized according to journal ranking, under the theory that publication in better journals indicates that the product itself is of better quality. For example, last year the Australian Research Council ranked 22,000 journals into "grades," with top journals given a ranking of A* and lesser journals graded from A down to D. This ranking—since withdrawn after some well-placed shots—would have had immediate consequences for researchers and their funding (Howard 2011). Of course, one way for us to read the Australian list would be with the smug satisfaction of seeing CER (for the moment) in exalted company. But there would have been a price: such lists commoditize scholarship as they rationalize and quantify, on a global scale, the products of university and individual energies.While the rationalization of expertise under a bureaucratic authority provides the stage for the enactment of new management strategies, it is individual governments and individual academics—with names and biographies—who act as agents of their own self-interests in this process. Eugene Garfield, the man who invented the impact factor as a means to track the diffusion of US-government-funded medical research in 1955, looked back 50 years later at the unintended consequences of its use in evaluating not only research but also the journals publishing that research, the individual researchers, and (we might extrapolate) their institutions. He commented (Garfield 2005, 20) that "a better evaluation system would involve actually reading each article for quality. But ... when it comes time to evaluating faculty, most people do not have or care to take the time to read the articles any more! Even if they did, their judgment surely would be tempered by observing the comments of those who have cited the work."6 It is one thing to describe the dilemma of peer review but quite another to celebrate disengagement or the devices we use to avoid connection, especially if the profitability of one's company depends on people not having "time to read the articles any more." Even a discerning fan of Garfield, Robert K. Merton (2000, 438), saw that citation indexes could become a motivating extrinsic reward that "at its dysfunctional extreme, displaced ... the motivating intrinsic reward" of research.Extending a point made originally by Keith Hoskin (1996) about auditing cultures, and then applied by Marilyn Strathern (1997) to higher education in England,7 David Bridges (2011, 33) reminds us that "when something shifts from being a measure to a target, then it ceases to be a measure. The trouble is that what start off as perhaps empirically grounded (extrinsic) indicators of quality rapidly become targets that people seek to achieve—and this distorts behaviour in a way which invalidates the original evidence of an association or at least the grounds for believing that the extrinsic indicator has a probabilistic relationship with intrinsic features of quality." Bridges takes inspiration from Martha Nussbaum's (1990) insistence on the noncommensurability of valuable things, writing that in the face of assessment schemes and impact factors and journal rankings, we must be clear that "quality in research is not reducible to a single set of values, nor representable by a single set of measures on a scale. In making qualitative judgments we have to find a way to hold a plurality of values in our minds at once and to discover such as are appropriate in the object under scrutiny" (Bridges 2009, 513).8The Full Costs of Commercial Scholarly PublishingAt one time, individual subscriptions underwrote most costs of scholarly journals; now, institutions pay ever-larger portions of the costs. Today relatively few individuals subscribe to journals unless—as with the CER—the journal is part of a membership organization (and even in such cases, individuals pay less than do institutions).9 Because libraries have less money to buy books, academic publishers encourage fewer monographs. Commercial publishers solicit ad hoc collections of articles that are, sometimes, thoughtfully edited but which, in many cases, are compiled from conference presentations. The prices of these volumes are typically higher per page than most scholarly journals. The system "works" because, in many countries, rewards to individual scholars and their institutions are being indexed to extrinsic measures of productivity. When the intrinsic value of research is difficult to gauge, extrinsic measures are being used as presumptive proxies for the intrinsic value. One of the major indicators of research productivity and the value of the product has been said to be publication. Since this criterion began to be adopted, pressures on institutions and individual scholars have increased such that the ostensible measure of productivity became a target and ceased to measure an underlying intrinsic quality. Accordingly, research assessment exercises have rationalized the types of indicators needed and awarded actual points, in many cases, for publication in a particular journal or particular types of journals.10Today's commercial publishers of scholarly journals dwarf the largest university presses, and they are doing very well indeed. For example, with almost 2,000 titles, Elsevier earned a 36 percent profit last year (US$1.1 billion on revenues of US$3.15 billion). Even at large US research libraries, the exponential growth in the numbers of scholarly publications combined with subscription price increases far outpacing inflation have created an unsustainable burden at a time of dramatic cuts in government support to higher education. At Penn State University, for example, libraries must now spend over three-quarters of their acquisition budgets to these serials, squeezing out purchases of monographs.11 The height of irony is that the (terrific) previously cited 2011 article by Professor David Bridges, of Cambridge University, is unavailable at his own institution because his library cannot afford to subscribe to the electronic journal where it appeared.The Editor as Dramaturge/Makeup ArtistIn the midst of this drama are journals themselves and their editors. Like cosmetologists, editors see quite a bit. They read some embarrassing first drafts (which they often return to the author without external review). In each particular subfield, editors occasionally work backstage with their particular prima donnas and Don Juans, as these sit indolent in dressing gowns, before makeup can white out the traces of verbosity. Editors can help the stars either massage highlights into unruly data dumps or grow out underspecified regression models; they repaste the dangling modifiers of the headline acts (and disappear their mixed metaphors). But, more than cosmetics, editing the academic journal today has become as much a dramaturgical process as it is about the product or final presentation. In this case, the dramaturgy is the running commentary between reviewers and authors over 1 or 2 years.Apart from the CER's open file of reviews, discussed below, another way to understand this dramaturgy can be seen thanks to a careful history Andrew Abbott has written about another University of Chicago Press journal. Abbott (1999) detailed the transformation the American Journal of Sociology (AJS) after the editorial term of Everett Hughes in the 1950s. Although Hughes was disinclined to permit anonymous reviewing of manuscripts (as the practice was then being initiated at the breakaway American Sociological Review), the successor AJS editors embraced anonymous reviewing even while recognizing that it would dissuade senior scholars from submitting their work.12 The number of external ad hoc reviewers expanded most rapidly under the AJS editorship of C. Arnold Anderson (who previously had institutionalized comparative education as a graduate degree program at Chicago). By the 1960s, the proliferation of US universities, and the boom in their social science departments, had produced a tremendous in the numbers of which the of for As numbers of social tenure editors and began to from and about evidence of journal was to the quality of the articles types of the from the later to our journal, there is CER history to review process became something more than a means of to this not only in this journal but also in many are the about expertise behind review and the of this the ways that review for research from most scholarly journal The of the review is according to because for and must be while a review of a can be made is for editors at journals where review became the but in the editorial the AJS or CER in the did not use that system discussed as a the and of each reviewers more careful (and more with public than they might be when judgments can be through the own in the from reading in the of and from with of leads to a different one more in with on his on a editorial the judgments of an individual ... in peer and are to partly public review social of because they are made even much of what takes behind In what we on the of to suggest a feature of peer review as an institution for and This feature an of the intrinsic quality of an academic for on a the who the of scholarship in when they the of and from a of in which are becoming closer to a than to a We would not the But it is nor to the of editors who would take a a of publication. for example, the case of a a of and one of these publishers a large of the scholarship in the field of comparative education. The that has been a major key indicator of the and has dramatic the first of the The of the most has to more and more to and support the demand of ... our academic has a number of titles in this in both and 2011 editorial in one important journal, Comparative a key editors who would scholarship as a and it also the mixed to authors from where the of higher education is the new The first the of the impact factor and ranking system and then as we do journal have become part of a taken-for-granted measure of scholarly quality. this very is how these become Even people whose academic is social and of can be found the language of and it is a The what readers first see on the Comparative Education two the of the journal itself and before to the is the one line the journal the ranking of the journal on the of that this is a the is by are the The of the are widely As and have an edit a ... Today scholars are the institutions of the as they have for even becoming aspect of scholarly is being even more being scholars of are their association and their own on and are being compiled from A commercial of this more to authors of books rather than to scholarly for an to any publishing company and electronic publications that could be relatively on devices as the one now being and by comes The of scholarly publishing a but we the can be without into the institutions of the bureaucratic rise to much of the we as scholarship today and that are from and in this from membership and universities, could a community of who their value or on Knowledge a in the is not a when a private of the could be found in the In our own one has edited the Education and has some of the authors who in the CER we There are also of electronic journals, such as the in Comparative Of the we describe have been in other education. The journal in in in response to the rapidly subscription costs of But, for the this is more the that that as other electronic journals, originally open now libraries of In some cases, prices up the original of the journal. For example, the by Robert and of University of is to fix the academic and the library subscriptions to journals in from to The reasons for the of journals are but they do to not in a different for education research may be a of open and In where the public has once to out research in our own researchers could be as a of to articles they publish in journals also into open In US medical research, for example, a recent that of the of that all by the or have for to the of an electronic of their peer-reviewed manuscripts for to be made later than after the of on the of university presses, there is a that is with the and not an to data It is our and our with the CER. The scholarly communication as the as opposed to a only on the production of articles per A recent on a by the of American University in which the University of Chicago Press was a key advised the academic publishing community that being be as scholarly communication. new may a or aspect of this system but it only if it our the in the academic community faculty, libraries, presses, scholarly government and the Review and for in Comparative Education Review not so very in an when communication to a and community was the reason for writing and also the of and the were in large because to and exchange with people who from the this become as as the can such as and The of to readers under many today the from a Comparative Education and has its in the we in the CER to our by not to the through scholarly journals promote scholarship much as the movement and The is more and relevant and less than the CER down the to and to a about their and this work in we that our with peer review is with by who saw that even anonymous peer review is in some way public and not so different from the evaluation of by review by the we first read with an to and we also have been We have been about of the in recent years for a external review by in and return to within an number of two of our editorial take for the external and an response to the Since only two have been after this first but many are to with the reviewers by and their of and are to the entire set of reviewers (and about of the of articles are for public at the this takes find that many CER reviewers as much for one another as for the author because they their be This can down the process because reviewers (and

Combinatorics · Political science · Rank (graph theory · Scholarship · Sociology · Contemporary Sociological Theory and Practice · Data Analysis and Archiving · Law · Mathematics · scientometrics and bibliometrics research

• The Global Ranking Regime and the Reconfiguration of Higher Education

  Open Access•Jun Li•Higher Education Policy•2016

  Add to list

• The Paradox of Autonomy

  Open Access•Mayumi Ishikawa, Chengzhi Sun•Higher Education Policy•2016

  Add to list

• Among the comparativists

  Michele Schweisfurth•Comparative Education•2014

  Add to list

• The History and Meaning of the Journal Impact Factor

  Eugene Garfield•JAMA•2006

  Add to list

• Department and Discipline

  Andrew Abbott•Department and Discipline•1999

  Add to list

• Love's Knowledge

  Martha Nussbaum•Love's Knowledge•1992

  Add to list

• Research Quality Assessment

  Open Access•David Bridges•Power and Education•2011

  Add to list

• How did we get here? Unintended consequences in education

  Michele Schweisfurth•Comparative Education•2011

  Add to list

• The Cult of Efficiency in Education

  Anthony Rogers Welch•Comparative Education•1998

  Add to list

• Research quality assessment in education

  Open Access•David Bridges•British Educational Research…•2009

  Add to list

• The social construction of reality

  Peter L Berger, Peter L Luckmann Thomas A Berger et al.•The social construction of reality•1967

  Add to list

• The Effects of Education as an Institution

  J W Meyer•American Journal of Sociology•1977

  Add to list