Comment by zipy124
6 days ago
The main problem is the incentives are off. Publishers are now rewarded for publishing more papers, as opposed to having more readers. When it was more readers, you were rewarded for the quality of the publication thus more people wanted to read it. By switching the profit incentive to number of publications, we have chosen quantity over quality.
Needless to say I prefer open access since those outside institutions can then read science, but the incentive model is heavily broken, and I'm not sure it's a good price to pay for the reward.
I disagree. We haven't chosen quantity over quality, we have decided that journals should not be the arbiters of quality. I think these new incentives are exactly what we want:
1. Journals want to publish lots of articles, so they are incentivised to provide a better publishing experience to authors (i.e. better tech, post-PDF science, etc) - Good.
2. Journals will stop prioritising quality, which means they will relinquish their "prestige" factor and potentially end the reign of glam-journals - Good.
3. Journals will stop prioritising quality, which means we can move to post-publication peer-review unimpeded - Good.
> We haven't chosen quantity over quality, we have decided that journals should not be the arbiters of quality.
In CS, this is definitely not the case at all.
If you remove the "quality badge" factor, journals are totally useless. Everyone in my field knows how to use LaTeX, produce a decent-looking PDF and upload it to arXiv. This saves you from paying APC's, has actually better discoverability (everyone checks arXiv as a one-stop shop for papers, almost no one goes to check the dozens of different journals) and much less hassle (no need to fiddle with arcane templates, idiosyncratic paper structures forced by each journal, idiosyncratic submission systems that look straight from the 90s, typesetters that introduce more errors than they fix, etc.).
I am pretty sure that journals, at least in my field, subsist precisely as arbiters of quality, they don't provide any other value at all.
Maybe it's time to do a Eurovision style thing for the quality badge. Everyone uploads to Arxiv. Every who's in the field votes on the worthiest papers (not allowed to vote for anyone you actually collaborated with).
Winners get to put a shiny sticker on their papers.
I assume uploading to arXiv doesn't count as having published a peer reviewed journal article, which is a problem for professionals.
For example, for me to progress in my current job I either need a doctorate or to have published a number of peer-reviewed articles in recognised journals as first author. I have written two IETF RFCs and these count for nothing.
I am not a scientist, I am a software developer. I am not employed as a scientist, I am employed as a software developer. But the rules of the organisation are thus.
2 replies →
In my field, journals subsist precisely as targets for a PhD. 3 journal publications and you can become a doc.
Yes exactly. Right now they are arbiters of quality but they shouldn't be, and the move towards Open Access is changing their role.
semanticscholar is a better one stop shop than arxiv
1 reply →
> journals should not be the arbiters of quality
It is the editorial board, i.e. academic peers, not the publisher, that are (?were) the arbiters. As far as I can see, the primary non-degenerate function of journals is to provide a quality control mechanism that is not provided by "publishing" on your own webpage or arxiv.org. If journals really are going to abandon this quality control role (personally I doubt it) then I fail to see their relevance to science and academic discourse at large.
Indeed, they are irrelevant. Right now they maintain an administrative monopoly over the peer review process, that makes them de-facto arbiters even if it's peers doing the work.
Journals should either become tech companies offering (and charging for) new and exciting ways to present scientific research, or simply stop existing.
I agree, and...
Completely off topic, but thanks for creating AudioMulch, I don't use it actively anymore but it totally revolutionized how I approach working with sound!
At the end of the day, I expect a journal that I pay for to be better than arXiv and that means quality control. Few people have the time to self-vet everything they read to the extent that it should be in absence of other eyes
If publishing the most papers is the goal why do we even need journals? We have blogs and preprint servers if Volume is your goal.
Journals should absolutely play a role in maintaining quality and curating what they publish.
> If publishing the most papers is the goal why do we even need journals?
For discoverability. Someone's trivial finding may be someone else's key to a major breakthrough, but little good it does if it can't be easily found
5 replies →
Peer review success is not the rule of the owner of a company but the acceptance you get from peers.
I can tell you for a fact that points 2 and 3 usually do not hold simply because publishing fees are directly correlated with the "prestige" perception of the journal.
> we have decided that journals should not be the arbiters of quality.
That's literally all I want them to do. I would love if they dwindled away to simply being monthly blog entries with magnet links to the articles, maybe with an introductory editorial.
We refuse to do this, because we have deeply integrated journals into a system of compensation for everyone involved. They're just magazines; "journal" is the beginning of the pomposity.
You could already publish a "fusion" journal where you link to the best articles in your field, and publish reviews of them - or even go back and forth with authors who want to be listed in your journal for a paper that they're about to publish. Outside of salaries, it would cost as much as a wordpress/patreon blog, or really, just a monthly twitter thread. The reason this doesn't happen is because it doesn't integrate with the academic financial system.
The only thing worthwhile about the journals is their brands, and the major ones in a lot of their fields (especially medicine) have ground their brands into dust through low quality. They continue through inertia: once anyone has ever made money doing something in the West, it will be preserved by any means necessary, because it's worth giving up part of that cash in order not to lose all of it. Scams are only ever defeated by bigger scams.
Nobody who is only important because they published in The Lancet will ever tolerate the devaluing of the idea of publishing in The Lancet, unless you give him a stipend for being involved in the next thing. Consequently, you're not going to be able to get a job from being published in Bob's Blog, no matter the quality of the peer review. Hence $1500 open access fees.
> we have decided that journals should not be the arbiters of quality.
At that point why even have a journal, let's just put everything as a Reddit post and be done with it. We will get comment abilities for free.
Maintaining quality standards is a good service, the journal system isn't perfect but its the only real check we have left.
> At that point why even have a journal
Great question.
> the journal system isn't perfect but its the only real check we have left.
I wish I could agree but Nature et al continually publish bad, attention-grabbing science, while holding back the good science because it threatens the research programmes that gave the editorial board successful careers.
"Isn't perfect" is a massive understatement.
These are all valid points. I think we agree we are just looking at different things, I argued if journals maintained their arbiter quality then the system is bad, but you rightly point out that this could finally grip this quality out of their hands, and so it could be good for science overall actually. I think these are fair points :)
Haha yes I jumped off in a very different direction. The points you raised are very much valid in the short-term. But longer term, I think journals charging authors for some kind of enhanced research presentation service is actually quite valuable, so the short-term negative effects might lead to a good outcome for the industry down the line - we hope.
So what service to the journals provide to the people who are paying them?
You pay them, they give your work a stamp of prestige that is mostly unrelated to the quality of your work.
I definitely want journals to be arbiters of quality. I have very limited time and want to read the best, and at the same time I don't want to read misinformation or disinformation.
They seem well-positioned to be such arbiters. Who else do you suggest and why are they better?
Nobody can possibly read every article and few have the expertise to decide. There is no reason to think the 'wisdom of the crowds' is reliable - and lots of experience and research showing it is not, and easily manipulated by nonsense. I don't want Reddit or Twitter.
Maybe we should pay the ones that put in the work and leverage their experience to judge the quality which would be the reviewers. In this age of disintermediation journals add little value in providing infrastructure or paying (if at all) reviewers and that money is in any case mostly public money.
> Who else do you suggest and why are they better?
The arbiters are just our colleagues, at the end of the day. The journal is just the organisational mechanism, one of many possible mechanisms.
For example, I follow a weekly reading list (https://superlab.ca) published by a group of motor control labs at Western University. Those people are my arbiters of quality.
I want to continue having arbiters, and I want it to be the same people (broadly speaking). I just don't want them to be organised around journals because journals are toxic and lead to concentrated power over scientific narratives.
1 reply →
A different way to not require journals to be the arbiters of quality is to let the truth itself be the arbiter of quality instead of designate gatekeepers.
1. Open peer-review to anyone interested instead of only select few. HN is an example of this phenomenon but not for novelty specifically.
2. Permit publication of papers that are shorter for results to spread faster. AI papers are a good example of this phenomenon.
> Publishers are now rewarded for publishing more papers, as opposed to having more readers.
That's the first order effect, but you have to look beyond it. If authors have to pony up $1500, they will only do so for journals that have readers. The journals that are able to charge will be those that focus on their readership.
> If authors have to pony up $1500, they will only do so for journals that have readers
On the other hand predatory journals make a killing from APCs so there is some market for journals with no readers.
My university had made it mandatory for students to publish atleast 1 paper to graduate from their bachelors degree, and would pay all the associated fees.
Most kids unfortunately did end up paying to publish.
Authors don't pay for that personally though. Nobody bats an eye at the $1500 publishing fee for a mediocre paper, that could have been a blog post, because the institution is happy to bolster its publication count.
Heck, nobody even bats an eye if that publication is to be presented at a conference with a few thousand bucks in travel costs.
This would probably depend heavily on how tenure decisions handles publishing. If it is heavily biased towards quantity of publishing, then that won't matter as much as you can "pay to win your paycheck".
If the tenure process focuses on quality of work, then it should work better.
The whole publication model is broken, not just the incentives. It used to be researchers eager to share their new findings with the few hundred people that could understand them, now it's throngs of PhD students grinding their way to degrees and postdocs trying to secure tenure. The journals are flooded with nonsense and actual researchers resort to word of mouth point out valuable papers to each other.
This is accurate and known to anyone actually in the area.
The people that pay are the Institutions (Universities mainly). Not the readers. The publications are sold to them as bundles even if the Institution does not want all the journals.
Yes you are correct, however I don't understand how this relates to my point, do you mind clarifying? I'd also caveat that the library (the purchaser of these bundles at most universities) often buys bundles based on requests from academics (more specifically research groups/departments) at the university, thus the readers do have some sway over which are purchased.
I do not know specifics of bundling agreements (shocker that I admit not knowing something:). I do know that libraries at some Institutions have started to provide funds to their researchers to pay the APCs. The library then goes to the Open Access publisher and negotiates bulk APC deals if they commit to a certain number of publications. Sort of a win win grant wise. This does not necessarily guaranty publication but if it does not get published you don't pay (processing submissions is an expense Open Access publishers incurs).
I am certain that that no system is perfect. My belief is that the Closed Access publishers have had free reign for so long that the largest ones abuse the system and competitive models are useful to restore some balance. The model also restricts access to information.
I would argue that one downside to Open Access is that incentives volume over quality (as others have said) but I would judge that on a per publisher basis just as I would any publisher. Closed Access models might also provide publication in areas of research that don't get tons of attention and research money.
I would also argue that there are other problems within research such as lack of reproducible results in many papers that is a far more pressing issue. Just my 2 cents. Thank you for the honest discussion.
1 reply →
Many if not most of the readers are grad students. Arguably they're the people who pay that indirectly in increased tuition fees.
> Publishers are now rewarded for publishing more papers
Publishers have a finite capacity based on the number of credible peer reviewers. In the past, it felt very exploitative as an academic doing peer review for the economic benefit of publishing houses. I'd much rather have "public good" publishers with open access -- at least I feel like the "free" labor is aligned with the desired outcome.
Is it a fee for publication or a fee for reviewing?
Found,
> Once your paper has been accepted, we will confirm your eligibility automatically through the eRights system, and you’ll get to choose your Creative Commons license (CC BY or CC BY-NC-ND).
It is on acceptance almost universally. This is why more selective journals have higher APCs. The overhead of reviewing and processing more papers when less ultimately convert costs money.
Upon publication almost exclusively.
It still wouldn't be perfect, but I'd like to see a system that rewarded publishers and authors for coming up with work that was a load bearing citation for other work (by different authors on different publishers, i.e. ones with no ulterior motive for having chosen it as a source).
Like some escrow account that the universities pay into and the publisher payouts go to whoever best enables their authors to do the most useful work... as determined by the other authors.
You know, we briefly had this with the h-index, and now h-index manipulation is so rife that it is no longer highly correlated with successful academic careers
I see, I hadn't seen the h-index before. I guess that's Goodhart's law for you.
There's got to be ways to improve things though.
I built something like this but it didn't get users. Replying to an author for the valuable info they posted would pay the author and it also accepted public payments.
An AI or search engine that identified the value of a contribution and paid the author directly from advertising money based on query traffic could be a way to solve this.
I can imagine that adoption was hard to achieve gradually. I figure you'd need a bunch of universities to get together and all at once say to the publishers:
> The only way we'll pay you ever again is through {the protocol}, deal with it.
If people just sought out and participated in better incentive alignment under the expectation that things would be better if only everybody did so... Well then things would already be better and we wouldn't be dreaming these dreams in the first place.
You had the quantity argument as well when it was about accumulation of subscribers. As a bigger variety of content also attracts a bigger variety of people.
The incentives are alright. Publishers who now start publishing too much low quality slop will lose readers (who has time to read all those low quality publications). Less readers leads to less citations, which will drag dawn their impact factor resulting in less authors willing to pay a high publication fee.
For those fields with an existing market, meaning there is more than one high quality journal, the market will provide the right incentives for those publishers.
I doubt that this is true except maybe for the top journals. Mid and low tier journals cater to scientists whose main incentive is to publish no matter how while moderately optimizing for impact factor (i.e. readers and citations). This lower quality market is huge. The fact that even top tier publishers have created low-ranking journals that address this market segment using APC-based open-access models shows the alignment between publisher and author interests will not necessarily lead to increasing quality, rather the opposite.
Does anyone actually read articles from those low tier journals? Many of those articles are illegible fluff pieces.
That top tier publishers create new low-tier journals for this market shows that they are very well aware of these incentives and risks. They are not flooding their top journals with low quality OA "pay to publish" articles, which was the argument from OP.
For academia's sake I hope you are correct, but my experience of the system leads me to suspect otherwise, though only time will tell.
One hope might be that it incentivises institutions away from the publish or perish mind set and starts to discourage salami slicing and other such practices, allowing researchers to focus on putting out less work of a higher quality, but I suspect the fees would need to be larger to start seeing this sort of change.
Disagree. The journals are now acting like a paid certification. If they admit any old slop, who would pay to submit their papers?
The service they are providing is peer review and applying a reputable quality bar to submissions.
Think of it this way, if you have a good paper why would you publish on Arxiv instead of Nature? And then if you are Nature, why would you throw away this edge to become a free-to-publish (non-revenue-accruing) publication?
Small correction to your point: they perhaps provide a reason for peer review to happen, but it's scientists themselves who coordinate and provide the actual peer review.
That is, unless ACM and Nature have a different approach to organizing peer review, in which case my correction is wrong. But I believe my point stands for many conferences and journals.
Reputable quality bar isn't the right metric. Quality is a better metric. To the extent it can be estimated, impact is another. Neither of these require journals specifically.
A different way to look at this is to question what "old slop" actually means.
The reason not to publish in Nature is that it might take a long time to get everything right in the paper to publish, to the point it takes years to get it read. Publishing fewer results faster spreads the results faster.
Processing != Publishing (at least I hope not).
What about a better deal: Scientific knowledge shouldn't be a for-profit venture to pursue.
Do you work for free?