Quantum Decision Making
This is an Editorial.
By Nicolai Friis (Atominstitut, Technische Universität Wien, Stadionallee 2, 1020 Vienna, Austria).
| Published: | 2023-02-07, volume 7, page 72 |
| Doi: | https://doi.org/10.22331/qv-2023-02-07-72 |
| Citation: | Quantum Views 7, 72 (2023) |
No, what you are about to read is not about the latest quantum-research band waggon to jump on. Instead, this editorial is intended to give an inside view on the editorial process at Quantum from the perspective of an editor at the journal.
Since taking on this responsibility in the fall of 2021 I have spoken to numerous colleagues about the journal, its purpose and its mechanisms, and I have heard why researchers like the journal, but also discussed some critical comments. My impression during these discussions was that, while many researchers are endorsing Quantum’s philosophy of community-driven open-access publishing as opposed to traditional for-profit publishing, not everyone is aware just how manuscripts are processed and how decisions are made at the journal, and by whom; what authors can generally expect, and what they should not expect. So here I hope to be able to give some insight into the inner workings at Quantum to help set expectations for authors, but also to convey the realities and challenges faced by the staff and volunteers that help to run the journal.
The life cycle of a manuscript at Quantum
Who is handling your manuscript
Before talking about what happens to your manuscripts, it is useful to understand a little bit about the organizational structure of Quantum, and the people behind it.
At Quantum, the non-profit organization supporting the journal (an NGO called Verein zur Förderung des Open Access Publizierens in den Quantenwissenschaften”, registered in Austria) and the editorial process are entirely separated, connected only by the fact that the (currently two) management assistants, the only paid staff at Quantum, are involved both in the NGO (taking care of admin, accounting, and publishing manuscripts) and in the editorial process (as we shall see in a moment). However, the editorial decisions are entirely in the hands of the editorial board, consisting of unpaid volunteer editors.
Can you handle the tr… manuscript?
When a manuscript is first submitted to Quantum, both the management assistants and the (current) editors in principle have access to the manuscript. However, in my own experience, editors (at least, I) hardly ever have the time to engage with manuscripts before being prompted directly. This is where the management assistants come in: they diligently go through every submission and make a first basic check, in particular, to see if the manuscript is on the arXiv (quant-ph, or at least cross-listed with quant-ph is a requirement for publishing in Quantum) and if there are any potential conflicts of interest with members of the editorial board. If a conflict of interest is identified, e.g., because an editor is also an author of a paper, then the respective editor has their access to the manuscript revoked and can thus not access any information pertaining to this manuscript going forward.
If all is in order, the management assistants then try to identify a suitable handling editor according to their respective topics of expertise and availability, taking into account the topic (title, abstract) of the submission and any suggestions for handling editors made by the authors themselves. Once a potential handling editor is identified, the editor is queried to check their availability and willingness to take over the manuscript in question. With the current submission numbers, the number of available editors, and editor response times, this can take some time (more on this below), but it also means that every manuscript is handled by an expert in the field; active researchers and authors themselves, who agree to take on manuscripts on a case-by-case basis, with no quotas to meet in terms of numbers of handled manuscripts, rejections, or acceptances. Ideally this means that editors can really focus on the science.
To review, or not to review?
After taking on a manuscript as handling editor, it is then the editor’s responsibility to read the paper and make a first assessment. In particular, the first decision to be made is whether to send the paper out to reviewers or not.
Now, I understand that there are conflicting opinions on this point around, so let me present my take on this. Of course every author hopes that when they submit their
manuscript to a journal, the editor will send it out to reviewers. At some journals, all papers are sent to reviewers, but for many journals having a manuscript sent to review means that the paper was convincing enough in its presentation and claims to warrant in-depth scrutiny by experts in the field. However, at most journals, this call is made by an editor who may have a different background, and even if they are from the same field as the research in the manuscript, they could be from a potentially very different subfield. This basically puts much of the responsibility on the reviewers.
At Quantum the situation is different. Handling editors are usually researchers who are intimately familiar with the field and their initial assessment may already be as detailed as a (good) review at some other journal. In particular, editors will have a good idea (and will be able to articulate it to other editors) what a manuscript brings to the table and whether or not a manuscript is likely to pass Quantum’s selection criteria (as perceived by other editors and reviewers). That is, papers in Quantum should offer a significant conceptual and/or significant technical contribution beyond the state of the art. Clearly, deciding if this is (even potentially) the case for a given manuscript is a judgement call and hard to press into an objectively verifiable or universally applicable standardized mold. Some manuscripts are extremely well written, clear, and convey diligent research on a relevant topic that carefully analyze their results. Such papers form the backbone of scientific progress. Yet, not all `good’ papers are seen as representing significant advances. When looking through a manuscript, the editor’s gaze is thus scanning for new concepts and techniques, ideas that combine existing approaches in a surprising or creative way, or come to a new insight based on them. A rule of thumb for an editor is to ask oneself “Is this a paper that I would recommend to colleagues or students?”
When an editor has doubts that a manuscript passes this bar, they will usually consult other editors to reach a consensus. I myself tend to involve 2-5 other editors in such a discussion, including at least one more experienced so-called coordinating editor, and try to briefly summarize the paper and its results before presenting my opinion on what it has to offer and making a suggestion for how to proceed with a manuscript. The most common courses of action are:
- Desk rejection: If the consensus among the editors is that they do not see a reasonable path towards eventual acceptance of the manuscript, then it is desk rejected. Such a decision cannot be appealed and is thus final. Although such a decision may come as a disappointment to the authors, it has an obvious advantage with respect to a (likely as disappointing) peer-review process: it saves the authors (and reviewers) much time. If the editors can relatively quickly identify reasons why the paper might not be recommended for publication, then it is to be expected that the expert reviewers will come to similar conclusions, at the expense of the time of everyone involved. A (relatively) quick desk rejection is not fun (also not for the editors), but it is often the better alternative.
- `Soft’ desk rejection: In many cases the editors see no path towards acceptance for the manuscript as is but it stands to reason that some (potentially substantial) revisions/additions/modifications of the manuscript might change this perception or that there is at least some doubt; editors are not infallible after all. In such cases the editors have the option of rejecting a manuscript but officially choosing a decision to `revise and resubmit’ without the paper having been sent to external reviewers. In such cases the respective decision letters outline the reasoning behind the decision and potential changes that would have to be made or points that would have to be addressed for the paper to be further considered. Such a decision gives the authors a chance to cut their losses, consider the feedback from
editors, and revise their manuscript for submission to another venue. Or, if they really feel that they can counter the arguments of the editors, to try to go forward with Quantum and resubmit a revised manuscript. Based on my (limited) personal experience (I handled 2 manuscripts that were resubmitted after soft desk rejection), I
have not seen this work out in favour of the eventually resubmitted manuscript though. - Send to review: An initial editorial discussion does not mean that a paper is necessarily desk rejected. In some cases the discussions provide input that help the handling editors to get a new perspective and the manuscript is sent to review after all. Aside from the manuscripts where the editors have some doubts initially, there are also many manuscripts where the initial impression is (strongly) leaning towards the positive. In such cases reviewers are sought immediately, but in any case no paper (that I have handled) is rejected without having been seen/commented on by at least two (usually more) editors.
Who wants to be Reviewer 2?
When manuscripts go out to review, one might naively assume that the editor’s job becomes much simpler going forward; now the reviewers have to do some heavy lifting. Right? Well, far from it.
First, identifying and finding available reviewers can be very tedious and frustrating. Different editors have different strategies to go about this, but my personal strategy is to try to ideally find 2-5 people that are senior enough to have a good overview of a field and its literature, but not so senior that their institutional/teaching/supervision responsibilities will prevent them from writing reviews; with the aim of ultimately obtaining at least two reviews within a reasonable time. I usually ask people to please get back to me within a week to let me know if they can (in a time frame that works for them) provide a report.
Many invited referees do respond reasonably fast, and some never respond. In some cases this may be tied to out-of-date email addresses stored in the system or emails ending up in spam filters, sometimes people are simply too busy or have other things on their mind/plate. We have all been there and we all understand that volunteering to review papers or even responding to review requests is sometimes not high up on our list of priorities, especially in the face of pandemics and other global, local, or personal crises. As volunteers ourselves, we editors nevertheless appreciate when we do get reksponses, even if they are to let us know that a reviewer invitation is declined. And I am happy to say that we also get many enthusiastic responses from academics who are happy to contribute to Quantum’s mission, and this feedback is of course particularly uplifting.
In selecting reviewers I do take into account (and in indeed very much encourage) suggestions by the authors, but for obvious reasons I try to avoid inviting very recent coauthors of the authors as reviewers. When too many of the initially invited reviewers decline to review or are unresponsive for too long (meaning, also personal follow-up
messages to the automatic reminders by the Scholastica system do not lead to a response), then I invite additional reviewers. In my experience authors can really help the review process along if they think about potential reviewers ahead of time and make suggestions upon submission. Oftentimes the authors are the people who know
their own field the best, they know who is active, who has the expertise, who might already be familiar with their work, but also who might have a conflict of interest, so these suggestions can speed up things. Editors will take these suggestions into account. Ironically, some authors believe that the people they suggest as reviewers will write particularly favourable reviews, but from what I have seen so far, reviewers suggested by authors are at times the most critical ones.
When enough reviews are obtained (usually two is the target) editors carefully read them and try to distill a coherent picture from them. In many cases this leads to further editorial discussions, queries to previous reviewers, or having to invite new reviewers, until a clear decision can be made (rejection, revise & resubmit, acceptance). Decisions at Quantum are not made based on a majority vote or some star ratings; the comments of the reviewers are carefully considered, and the quality and level of detail of a report is taken into account. A very superficial report advocating rejection or acceptance does not automatically lead to one or the other, but is weighed against the feedback of other reviewers and the opinions of various editors. So whichever way the decision goes, authors can be sure that their manuscripts were given due consideration.
Some statistics
To illustrate the process outlined above, let me go through some numbers. While Scholastica does collect some statistics on the submitted manuscripts, I did not find the statistics the platform offers to be particularly useful, so the following numbers are based on data I collected (and processed) myself for the papers I handled. So it should be understood that these numbers may not accurately reflect the overall statistics for all papers submitted to Quantum.
In particular, it is crucial to note that biases may be introduced not just through the way that I myself handle submissions, but also in terms of which submissions are assigned to me in the first place. So the statistics might vary for different topics and for different editors in the same topics. For instance, editors might already introduce some bias in the publication statistics in how they select which handling requests they choose to accept: an editor that is more likely to take on assignments of papers whose titles they find particularly interesting might have a higher rate of acceptance, whereas I myself tend to base my decision for accepting assignments purely on my available time (provided that the topic of the submission is within my area of expertise). So keeping this mind, here are the numbers.
Decisions
In the 16 months since I started I took on 44 submissions as handling editor, roughly speaking 2-3 papers every month. For two of these 44 papers no initial decision has been made yet, so I will only consider the remaining 42 papers for the following statistics (and all percentages of total numbers refer to these 42):
- 14 manuscripts (33%) have been desk rejected (with no option to appeal/resubmit);
- 11 manuscripts (26%) hav
e been subject to a `soft’ desk rejection and the respective authors could thus technically challenge the decision and resubmit. Two out of these 11 manuscripts were resubmitted and sent to reviewers, but were ultimately not accepted based on the reviewer reports; This means a total of 23 manuscripts (55%) were not sent to reviewers. - Of the 19 papers that were eventually sent to reviewers (including the two initially `soft’ desk-rejected papers), 3 papers were rejected (no option to resubmit, but technically with the option to appeal), and for 9 manuscripts a formal decision to revise and resubmit was issued but, similar to the soft desk rejection, it was made clear that the manuscript would only be reconsidered if substantial revisions would be made and/or strong arguments would be supplied to counter criticism by the reviewers (and/or editors). For one of the latter 9 manuscripts, the authors did respond to the reviewer comments, revised their manuscript and eventually the manuscript was indeed accepted (and published) along with 6 other manuscripts.
- A total of 7 manuscripts (17%) has been accepted (and published). Out of these 6 were accepted after the second round of review, one after the third round. Finally, 1 manuscript is currently with reviewers and still under consideration (likely to be accepted, provided that the reviewers agree that the minor revisions they suggested have been appropriately implemented). Being optimistic about this manuscript (and pessimistically assuming none of the soft desk-rejections successfully resubmit) would bring the total number of (eventually) accepted papers to 8 out of 42, an acceptance rate of 19%.
The wheel of time
Although some readers and authors of Quantum might be fans of lengthy fantasy epics and thus be acquainted with waiting for years, or even decades, for the publication of the latest entry in a series, most authors might not be as patient when it comes to the publication of their manuscript. So what are the typical time frames that authors have to expect at Quantum? In the following I will take a look at the characteristic time scales for the 42 papers discussed above.
Assignment
For the 42 papers in question, the average time from submission until the paper was assigned to an editor (me) was 25.7 $\pm $ 16.4 days. This seems fairly long, but as the large standard deviation of 16.4 days shows, it varies strongly from manuscript to manuscript. For 7 manuscripts it took a week or less, for 4 manuscripts it took more than 50 days (for two it was two months), but for 27 manuscripts (64%) it took less than a month.
To understand these times, one has to take several aspects into account. First, high (and in particular, steadily increasing) submission rates create a backlog which means that it can take a few days before the management assistants get around to taking a first look at a submission. However, the second, and usually bigger factor influencing the time until assignment is the response time of editors and the number of editors that have to be queried before a handling editor is identified. The management assistants diligently identify editors with suitable expertise and check against records how many manuscripts they are handling and when they have last taken on new manuscripts. But once editors are queried it may take some time until they are able to respond, and when they do it can turn out that they are simply not available. As other researchers in the community, editors at Quantum have students, teaching, grant deadlines, papers to write, and, rumour has it, occasionally also a life outside academia. But jokes aside, work-life balance and mental health are serious matters and also editors at Quantum need to prioritize their wellbeing before taking on more editorial duties.
These factors are then combined with certain seasonal bottlenecks: usually spikes in the number of paper submissions around certain holiday periods are correlated with decreased availability of editors, which can sometimes lead to delays. A third aspect that comes into play is the specific topic of the paper. Although the editorial board of Quantum has already accumulated a very broad set of backgrounds across various fields of quantum research, there are sometimes submissions whose topics do not directly fall into the area of expertise of the available editors, which can prolong the search for a suitable handling editor.
Initial decision
Once the papers are assigned to an editor, the time until an initial decision (desk rejection or sending out for review) is made clocks in at an average of 9.9$\pm $6.5 days. This time includes reading and thinking about the manuscript, but also editorial discussions and communicating the decision to authors in case the paper is desk rejected or querying first potential reviewers (more on that shortly).
The total processing time from submission until the initial decision then amounts to 37.6$\pm$16.0 days.
Note from Quantum’s executive board: Nico is one of Quantum’s most responsive editors. The time from submission to the initial decision is on average (much) larger, with high variance across the editorial board.
Inviting reviewers.
When it comes to inviting reviewers, which was done for 19 out of the 42 discussed manuscripts, an average of 5.6$\pm $2.6 people were invited to obtain an average of 1.9 reports in the first round of review. For 14 manuscripts 2 reports were obtained. For the five remaining manuscripts, two obtained 3 reports (but were both rejected eventually), and three only received a single report (despite the fact that 6, 7, and even 12 reviewers had been invited, respectively) in the first round of review. Since some manuscripts go through a second round of review and initial reviewers are sometimes not able to provide a second report, or because additional opinions are sought in the second round, the total number of distinct reviewers per reviewed manuscript is 2.3$\pm $0.9 on average.
Decision time
When the dust of the review process has finally settled, when all $\hbar $’s have been crossed and all $i$’s and $\dot {\psi }$’s have been dotted, and a final decision has been made the average manuscript will have spent a total (throughout all rounds of review) of 16.3$\pm $13.2 days with the editors and 62.6$\pm $31.6 days with reviewers, for a combined total processing time of 70.5$\pm $42.0 days on average. This figure does not include the times that manuscripts spend with authors between decisions and resubmissions.
Keeping this in mind, the total processing time (that the manuscripts spend with editors or reviewers) adds up to an average of 59.9$\pm$38.7 days until rejection versus 112.1$\pm $20.2 until acceptance (not including the paper that is currently with reviewers).
Although it is not surprising that the average time until acceptance is larger due to the fact that all accepted manuscripts underwent (at least) two rounds of review it is notable that the difference is quite substantial. This is owed to the small sample size of 7 accepted manuscripts, which means that a single outlier can skew the statistics quite substantially. In this case, one manuscript was with reviewers for 71 days in the second round, another manuscript for 52 days, much longer than the times it took to obtain reviews in the second round for the other 5 accepted manuscripts (whose average overall processing time is 100.4$\pm $3.8 days).
How to contribute
So what is our take-away message from these numbers? Well, first of all, there is room to improve. True. The journal will have to continue to adjust to increasing submission numbers and find an equilibrium in its number of editors. Yet, I think these numbers do not compare so unfavourably with my personal experience as an author with many for-profit journals who pay editors and copy-editors, financed through (in some cases) exorbitant open-access fees or subscription models.
I for one feel that the processing times offered by Quantum are competitive, the reports are (in my experience as editor but also as author at Quantum) of very high quality and generally fair and constructive in their criticism. So mainly, I think the challenge will be to keep this up, and to keep the community engaged and sold on the idea of fair and cheap community-driven open-access publishing.
So how can you as readers contribute to this effort? Authors can contribute by continuing to put their trust and high expectations into Quantum. Send the manuscripts that you feel represent your best work, be open for criticism, but also communicate to the volunteers at Quantum how they might perhaps be able to improve your experience.
You can also contribute by reviewing for Quantum. Many of you probably already have done so, and I hope many more will be willing to accept review requests in the future. So far, I can only say that the reviewers that I have put my trust in as editor have done an outstanding job, many thanks for your time and effort!
And finally, some readers might consider the option of contributing as editors themselves. Currently the call for editors is open, the deadline for applications is the 16th of February 2023.
► BibTeX data
Cited by
On Crossref's cited-by service no data on citing works was found (last attempt 2024-05-10 17:05:55). On SAO/NASA ADS no data on citing works was found (last attempt 2024-05-10 17:05:55).
This View is published in Quantum Views under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. Copyright remains with the original copyright holders such as the authors or their institutions.
Pingback: Call for editors 2023 – Quantum
Pingback: Call for editors 2024 – Quantum