Old news. I refer to the article “Who’s Afraid of Peer Review?” by John Bohannon published today in Science. The article is described as “A spoof paper concocted by Science reveals little or no scrutiny at many open-access journals.” That statement is true. However, it is also true that traditional publishers have been found to lack scrutiny as well.
Bohannon did confer with a small group of scientists. He reports that some say “that the open-access model itself is not to blame for the poor quality control revealed by Science‘s investigation.” For me, that is the salient point. Whether articles are open access or not is independent of quality peer review. The benefits of making research available for other researchers to read and build upon outweigh the rogue publishers, or instances of shoddy peer review. Traditional methods of publishing were developed to make science more accessible. Open access extends that goal. The issue now, for us in academia, is to teach others how to evaluate critically the quality of research in an article, however delivered.
Tenure-track faculty aspiring to achieve the highest academic ranks are often hesitant to explore creative and innovative modes of scholarship. While there are several possible reasons for their reluctance, one may be the fact that alternative scholarship still does not carry as much weight during promotion and tenure reviews as traditional scholarship does.
Stephen Nichols, professor of medieval French literature at Johns Hopkins University, highlights this concern in Digital Scholarship in the University Tenure and Promotion Process: A Report on the Sixth Scholarly Communication Symposium at Georgetown University Library:
“… disincentives are so powerful as to discourage experimentation. Young scholars are counseled that they need solid print dossiers before they attempt digital scholarship and that, even then, they are still at some risk.”
A University of California, Berkeley report entitled The Influence of Academic Values on Scholarly Publication and Communication Practices acknowledges that while faculty members realize that it is important to experiment with using alternative methods of scholarly communication, the challenge may be the inability to have those works properly evaluated by tenure review committees:
“There is presently a somewhat dichotomous situation in which electronic forms of print publications are used heavily, even nearly exclusively, by performers of research in many fields, but perceptions and realities of the reward system keep a strong adherence to conventional, high-stature print publications as the means of record for reporting research and having it evaluated institutionally.”
Review committees have a difficult time understanding the significance of digital scholarship, let alone knowing how to assess its impact. Review committees understand citation indexes and journal impact factors or book reviews as the evidence of impact of traditional scholarship. How does one determine the impact of a blog post, a web site, a YouTube video, or a slide deck posted on Slideshare? As a result, emerging forms of digital scholarship are often not defined in criteria documents and therefore not fully valued the faculty rewards system.
Jason Priem spoke at The Ohio State University Libraries on March 5th, 2013. His talk focused around three areas. A link to the complete presentation is located at the end of the post.
Setting up the System
He gave a quick rundown of the history of publishing and noted that when scholars shared data/information it was done through the letter. It made the best of available technology. However, in 1665 there was a revolution with the scholarly journal and with this change new standards were created. Articles developed structures.
With the web, it’s now time for a second revolution. Publication is nearly free and we’re no longer using the web to its full potential. We are still using the best technology of 1665. His contention is that science of the future will be in the analysis NOT in the collection of data. Open data allows researchers to replicate results exactly OR do a mashup of data and add your own analysis.
There are a couple of data repositories where data can be deposited for future analysis by any number of research groups. These include Dryad and FigShare.
Another interesting change in the realm of scholarly publishing relates to the conversation around research. In building from the invisible college, researchers are able to share answers and, more importantly, share questions. Math Overflow is an example, similar to Stack Exchange, where tough questions can be examined, debated, and solved by experts or novices. These type of resources, like Wikipedia, allow for experts to emerge based on their area of expertise, not on their institutional affiliation. The crowd will self-patrol and promote resources of high value.
Currently people are confusing form with purpose. Publishing used to mean “the act of making public.” Therefore a tweet, a blog post, a journal article, would all be forms of making information available publically. Information can be made public very quickly. Then it is imperative to set up good filtering systems. Today academics spend about the same amount of time reading as they did in the 1970s; however, more is being read because more is being published. Jason argues that if you do the same thing only faster and more of it that the system is broken. His solution? Set up your own personal journal. If the article can be decoupled from the journal, you are able to filter to only get the information that is relevant to you and your research interests. One way is to set up a Tweetdeck. This will allow for great peer review of articles of interest. You’re also able to target your own research for feedback to an audience that is extremely interested in your output.