This is the second of a three post series focused on transparency in peer review hosted by Publishing and Repository Services and the Research Commons to celebrate the 3rd annual Peer Review Week. The first post reflected on editing the open peer review journal, Empirical Musicology Review. Our second in the series is a guest post from Theresa Delgadillo, the lead editor of Latinx Talk. Published by The Ohio State University Libraries, Latinx Talk is an interdisciplinary, moderated, and peer-reviewed forum for ideas, research, and creativity that fosters critical dialogues.
Peer Review in an Online Publication
By: Theresa Delgadillo
As the lead editor of an online academic forum for most of the past six years, I have been involved in an ongoing discussion about the meaning, function, and process of peer review with scholars who are also working in new formats. The publication that I have been involved in growing is Mujeres Talk, online from 2011 to 2017 and now transitioned to our new online academic forum Latinx Talk, which will debut in September 2017. Mujeres Talk was, and Latinx Talk will be, published by The Ohio State University Libraries. Latinx Talk will carry forward key aspects of the goals of Mujeres Talk, which includes publishing peer-reviewed short-form research, informed commentary, creative work, reports from the field, and guidance on academic professionalization.
For its first year of existence, Mujeres Talk was strictly an editor-reviewed site, and I provided review and requested revisions for all submissions. In 2012, I recruited a group of women to join me in an editorial collective, and in the process of sharing the work of reviewing submissions, we recognized that we were actually engaged in a form of peer review. Over the next few years, we worked to codify our review process into what we called a modified form of peer review, and that model, with some tweaks, became our standard. But we didn’t get there without some controversy, primarily from scholars involved in print journals.
When we began referring to our site as one that offered a modified form of peer review, I heard from a number of scholars who were not shy about how they viewed that claim. “Peer review is blind. How can you say you do peer review?” “Peer review only applies to journals.” “By saying that you do peer review, you’re going to mislead junior scholars into thinking that what they write for your site counts in promotion and tenure when it doesn’t.” What did we mean by “modified form of peer review”? Because two priorities of our site were, and remain, to publish short articles on current research and commentary informed by research, we felt it important to ensure that submissions in these categories sufficiently engaged with relevant and current research. Every submission to the site is reviewed by two members of our editorial collective, and reviewer’s comments are returned to authors with suggestions for revision. Our guidelines for review are published on our web site and include assessing whether a submission offers an original and unique perspective, sufficiently cites and discusses relevant scholarship, is appropriate for our site, and meets our length requirements. Because we are an online forum, we are constantly working to provide feedback to authors on how to write for an audience that includes both academics and non-academics. But our review process was not blind. Reviewers were aware of author names and authors knew who had reviewed their piece when they received our comments. We adopted what might be better termed “open peer review,” in part, because we found this the most economical and efficient way to review for the length, forms, and context in which we published – quick reviews, more advice on how to write in this format in the earlier years, and no staff. We often asked scholars to engage with or cite relevant scholarship. We rarely accepted submissions that did not require revisions, but key exceptions were invited essays from experienced senior scholars.
Print journals maintain expansive Editorial Boards so that there are enough potential reviewers from as wide a range as possible of expertise to review any submissions a journal might receive. In our first six years, we maintained what might be better described as a working group of editors whose research and teaching was centered in our interdisciplinary field, and so we managed to offer strong reviews to submissions from a variety of fields while also pinch hitting on moderating comments, or managing the site. I am not sure how our practice of providing non-anonymous reviews to authors mattered to them, but my impression, based on the professionalism of authors who responded to reviews and revised, is that authors seemed less likely to take a review personally when they knew who the reviewers were. I can think of one or two notable exceptions, but it’s also likely that those exceptions would have taken it personally in any case. This is strictly an impression, and even if I asked a few authors, I couldn’t offer more than anecdotal evidence in support of this impression, but both are based on six years of experience and interactions with authors on over 140 submissions. As a contributor to the forum, I have valued and appreciated my colleague’s edits, corrections, directions, and suggestions on anything that I’ve written for Mujeres Talk.
Authors who published on Mujeres Talk gained a wider audience for their work (between 200 and 1200 page views per post) and gained publication experience, while also contributing to building a public and national network of scholars in Chicana/o and Latina/o Studies. We encouraged authors to include this digital publication on their CVs, while also cautioning them that a short form publication would weigh less in promotion and tenure than a longer critical essay. In these six years, I have received one inquiry from a Chair of a Department asking for clarification on whether we were really a peer-reviewed publication. I explained our open peer review process, which has also always been described on our site. But that inquiry was a sign of what’s at stake.
While our editorial process and policies departed from those of print journals, they did so because we were working in a distinct context, yet this didn’t obviate the need for peer review. I believe that some of that early critique was driven by a number of factors: the perception of a short-form digital format as a threat rather than a complement to print journals, the belief that Latina/o Studies couldn’t sustain more than two or three journals, and competitiveness, but an underlying part of that critique was also the shared experience of difficulty in getting recognition for the scholarship and research of Chicanas and Latinas, and the fear of departing from well-known and respected forms. I have experienced that difficulty quite intimately in my years in the academy, and have witnessed how easily we recognize the innovation of our peers when they are white and male, yet how difficult we find it to recognize the innovation of our peers when they are women of color. While our work on Mujeres Talk and Latinx Talk gets categorized as “outreach,” similar work by others is categorized as “public humanities” or “digital humanities scholarship,” with the attendant disparities in funding and institutionalization for our respective projects. Nonetheless, we have made a contribution to new forms of digital scholarly publications, widening the possibilities for where Latina/o Studies work has impact, and working alongside peers on similar sites. And we are working on gaining funding and resources.
About two years ago, Susy Zepeda, one of our editorial board members proposed that we return to a practice we had intermittently followed in earlier years by sharing our reviewer’s comments with each other as a way of developing a shared editing ethos and of training newer members to the group in how to do peer review. We adopted this practice anew, and we all benefitted from it, especially since the composition of the editorial group had changed with time as some cycled off and new members joined. In an attempt to streamline our editorial process, we tried to review in succession so that the second reviewer would receive both the essay and the first reviewer comments, but this turned out to take too much coordination in the short turn-around times and lack of resources that we work with on Mujeres Talk and now Latinx Talk. However, just this past spring, I was asked to provide a blind peer review for a print journal, and a short while after completing the task, I received a copy of the other reviewer’s comments, though the second reviewer was not identified. This had never happened before. I inquired and the journal editor explained that this was their practice, to share reviewer comments with each other. Though I appreciated the other reviewer’s comments, that reviewer had had access to my comments before doing their own review, and this is how I was able to recognize the drawbacks of trying to do reviews in succession. It may be a good way of helping to teach people how to do reviews, but it lessens the independence of the second review, and thereby deprives the author of greater insight. In our case, a better way to teach new editors how to peer review is to share examples of competent and useful peer reviews, which we do when new editors join our Editorial Board.
Peer reviewing articles and manuscripts is not something anyone ever taught me, but a task I learned in fits and starts when I joined the profession. I learned, slowly, how to do this by reading reviews of my own work; by listening to scholars commenting on reviews they received, reviews they gave, reviews they heard about — sometimes with an eye toward critiquing someone they suspected of giving them a bad review (not always an accurate assumption). Each of our editors brought their own academic experiences of the review process to our shared work, yet because we publish work in shorter formats, work with quicker turn-around times from submission to publication, and meet regularly to coordinate this work, we find ourselves building more of a shared ethos and practice in peer review and editing than might exist at print publications.
Most recently, and as we prepare to launch our new site, we experimented with re-instituting blind review by removing the names of authors from some reviews and removing the names of reviewers in comments to authors. My initial impression was that this didn’t make much difference. I felt that our reviewers were as attentive to rigor as they were to supporting authors who submit to our site as they had always been, but our Editorial Board has expressed a preference for not knowing author names and a desire to institute partial blind review as a way of adhering to academic norms that would make the work of our site, and those who publish on it, more legible in academia.
I confess that my experiment was driven in part by the nagging sense that we might gain greater academic standing if we could claim that we were a publication governed by blind peer review. Those earlier voices of opposition – “peer review must be blind to be peer review” — and more recent experiences with how my work is viewed within my own department – where this work has been viewed as “service” rather than “research” (and “service” weighs less in evaluating merit raises, not to mention all kinds of other ways) both combined to lead me to experiment with blind peer review in this format. I thought that this transition period from Mujeres Talk to Latinx Talk might be a useful time for the experiment. My own view is that for this forum, for this format, we gain more by employing an open peer review process in which we enact, with authors, a shared commitment to make more Latina/o Studies knowledge known among wider audiences – a commitment that calls us all to act fairly with respect to our work and the work of others – as well as constant practice in diplomatically and fairly responding to the ideas of others. I believe our new review process, though partially blind, continues to embrace an ethos of shared respect, commitment, generosity, and responsibility while recognizing that this ethos only exists to the degree that we attentively enact it every day.
For more on Peer Review Week, including available webinars, visit the Peer Review Week activities page: https://peerreviewweek.wordpress.com/activities/
To learn more about the Libraries Publishing Program, visit our website: go.osu.edu/librarypublishing