Arguably the biggest source of issues with peer review quality is the lack of incentives to invest more work in invisible service labor, and improving the incentives for that was one of the missions of ACL’23.
With support of ACL exec, we are pleased to announce the inaugural round of ACL reviewer awards, which will be comparable in scope to the best paper awards policy (1-1.5% of the pool of reviewers and chairs). Besides serving as a token of community recognition, and they come with a special award: a fee waiver for any *ACL virtual conference, to be used within a year of the award. Since most academic institutions cover attendance for the conferences where their affiliated authors have accepted papers—but not other conferences—-we hope that this will allow the awardees to attend more conferences than they would otherwise, and the benefit of the award will thus be received by them rather than by their host institutions.
The reviewers are nominated for awards by their area chairs, whom we asked to recognize the reviewers providing extra helpful reviews or high-quality emergency reviews, “champion’’ reviewers, and reviewers who were particularly active in the discussion phase, and demonstrated exceptional open-mindedness or expertise. We also asked the Senior Area chairs to nominate exceptional area chairs, and finally we as PCs also nominated the SACs of the track that was the most on-time with its tasks, provided the most helpful feedback, and followed our instructions the most closely. In total, we have 67 awards.
Here’s a few of our favorite reviewer nomination pitches:
Although the paper was neither sound nor exciting, they took the time to write very specific and detailed comments along with constructive feedback and suggestions.
They wrote good enthusiastic reviews and actively participated in the discussion, acknowledging and taking into account other reviewers, as well as clarifying their own stance.
This reviewer not only submitted thorough and helpful review, but also followed up on the authors’ response and provided convincing arguments for why the paper should be accepted.
They responded very promptly, humbly, and with an open mind to my request to fix some review policy issues before the author response. As a result, they revised their review to be very balanced while keeping a friendly and encouraging tone.
This reviewer did a thorough emergency review in one day, she is a hero :)
They volunteered to do an emergency review and wrote detailed reviews for both papers they reviewed in my batch. Additionally, they proactively started discussing with the other reviewers after the rebuttal, pushing back on some of the other reviewers’ criticism that was unjustified also in my opinion.
And AC nomination pitches:
… had read at least two papers herself to make a more informed decision. In one case this led to noticing reviewers’ misunderstanding/mistakes and saved the submission.
Thorough summaries, engagement in the reviewers’ discussions, clear stands on the papers.
This area chair proactively completed all assigned duties well in time, moderated discussions promptly, and even maintained a log of interactions with late reviewers to chase them for completing their tasks in time. He also declared potential COIs upfront, so as to avoid later conflicts.
On behalf of ourselves and the authors - thanks for your help!
Outstanding Senior Area Chairs
- Carolin Lawrence (NEC Laboratories Europe, Germany)
- Ana Marasovic (University of Utah, United States)
- Chenhao Tan (University of Chicago, United States)
Outstanding Area Chairs
- Jasmijn Bastings (Google, Netherlands)
- Tommaso Caselli (Rijksuniversiteit Groningen, Netherlands)
- Lea Frermann (Melbourne University, Australia)
- Atsushi Fujita (National Institute of Information and Communications Technology, Japan)
- Christophe Gravier (University Jean Monnet, France)
- Peter Hase (University of North Carolina at Chapel Hill, United States)
- Germán Kruszewski (Naver Labs Europe, France)
- R. Thomas McCoy (Princeton University, United States)
- Jakob Prange (Hong Kong Polytechnic University, Hong Kong Special Administrative Region of China)
- Grusha Prasad (Colgate University, United States)
- Abulhair Saparov (New York University, United States)
- Mihai Surdeanu (University of Arizona, United States)
- Sarah Wiegreffe (Allen Institute for Artificial Intelligence, United States)
- Xi Ai (Chongqing University, China)
- Alan Akbik (Humboldt-Universität zu Berlin, Germany). Secondary reviewers: Susanna Rücker, Elena Merdjanovska, Ansar Aynetdinov and Max Ploner
- Tariq Alhindi (Columbia University, United States)
- Long Bai (School of Computer Science and Technology, University of Chinese Academy of Sciences; CAS Key Laboratory of Network Data Science and Technology, Institute of Computing Technology, Chinese Academy of Sciences, China)
- Roy Bar-Haim (IBM Research, Israel)
- Barend Beekhuizen (University of Toronto, Canada)
- Rajarshi Bhowmik (Bloomberg L.P., United States)
- Logan Born (Simon Fraser University, Canada)
- Marie Candito (LLF (Université Paris Cité / CNRS), France)
- Zeming Chen (Swiss Federal Institute of Technology Lausanne (EPFL), Switzerland)
- Yang Chen (Georgia Institute of Technology, United States)
- Yangyi Chen (UIUC, United States)
- Philipp Cimiano (Univ. Bielefeld, Germany)
- Mathias Creutz (University of Helsinki, Finland)
- Verna Dankers (University of Edinburgh, United Kingdom)
- Dipankar Das (Jadavpur University, India)
- Maud Ehrmann (École Polytechnique Fédérale de Lausanne, Switzerland)
- Cristina España-Bonet (DFKI GmbH, Germany)
- James Fan (Tomato AI, Inc, United States)
- Sergey Feldman (Allen Institute for Artificial Intelligence, United States)
- Lukas Galke (Max Planck Institute for Psycholingiustics, Netherlands)
- Shaoru Guo (Institute of Automation,Chinese Academy of Sciences, China)
- Jiaqi Guo (Microsoft, China)
- Pere-Lluís Huguet Cabot (Sapienza University of Rome, Italy)
- Joseph Marvin Imperial (University of Bath, United Kingdom)
- Vivek Iyer (The University of Edinburgh, United Kingdom)
- Tom Kenter (Google UK, United Kingdom)
- Gene Kim (University of South Florida, United States)
- Fei-Tzin Lee (Computer Science Department, Columbia University, United States)
- Abhay Lokesh Kashyap (Walmart, United States)
- Chao Lou (ShanghaiTech University, China)
- Jinliang Lu (National Laboratory of Pattern Recognition, CASIA, Beijing, China, China)
- Avinash Madasu (University of North Carolina Chapel Hill, United States)
- Brielen Madureira (University of Potsdam, Germany)
- Márton Makrai (MTA Research Centre for Natural Sciences, Hungary)
- Abelardo Carlos Martínez Lorenzo (Sapienza University, Italy)
- Evgeny Matusov (AppTek, Germany)
- Matthew Mulholland (Educational Testing Service, United States)
- Mathias Müller (University of Zurich, Switzerland)
- Vivi Nastase (University of Geneva, Switzerland)
- Matan Orbach (IBM Research, Israel)
- Hao Peng (Beihang University, China)
- Manikandan Ravikiran (Georgia Institute of Technology, Hitachi India Pvt Ltd, India)
- Kyle Richardson (the Allen Institute for Artificial Intelligence, United States)
- Maria Ryskina (Massachusetts Institute of Technology, United States)
- Michael Saxon (University of California, Santa Barbara, United States)
- Tom Sherborne (University of Edinburgh, United Kingdom)
- José G. C. de Souza (Unbabel, Portugal)
- Keyang Xu (Columbia University, United States)
- Mert Yuksekgonul (Stanford University, United States)
- Eric Zelikman (Stanford University, United States)