The Scholarly Kitchen

Not specific to Wikipedia:

https://aarontay.medium.com/3-new-tools-to-try-for-literature-mapping-connected-papers-inciteful-and-litmaps-a399f27622a

https://archive.is/Ul13s

Specific to Wikipedia:

Wikipedia Citations: Reproducible Citation Extraction from Multilingual Wikipedia [2024]

https://arxiv.org/abs/2406.19291v1

https://doi.org/10.48550/arXiv.2406.19291

> Wikipedia is an essential component of the open science ecosystem, yet it is poorly integrated with academic open science initiatives. Wikipedia Citations is a project that focuses on extracting and releasing comprehensive datasets of citations from Wikipedia. A total of 29.3 million citations were extracted from English Wikipedia in May 2020. Following this one-off research project, we designed a reproducible pipeline that can process any given Wikipedia dump in the cloud-based settings. To demonstrate its usability, we extracted 40.6 million citations in February 2023 and 44.7 million citations in February 2024. Furthermore, we equipped the pipeline with an adapted Wikipedia citation template translation module to process multilingual Wikipedia articles in 15 European languages so that they are parsed and mapped into a generic structured citation template. This paper presents our open-source software pipeline to retrieve, classify, and disambiguate citations on demand from a given Wikipedia dump.

Prior work referenced in above abstract with some team overlap:

Wikipedia citations: A comprehensive data set of…

A reporter's tips for understanding science journalism:

http://www.cjr.org/analysis/understanding_science_journalism.php

A dialog between two scholars about how make scientific research more trustworthy:

http://www.washingtonpost.com/blogs/monkey-cage/wp/2015/03/13/how-to-make-scientific-research-more-trustworthy/

A blog post, "How Can We Improve the Article Review and Submission Process?" with many suggestions for improving scientific publishing:

http://scholarlykitchen.sspnet.org/2015/03/26/ask-the-chefs-how-can-we-improve-the-article-review-and-submission-process/

In the end, we have to count on incremental improvements in publishing, and incremental improvements in reading by all of us, to prompt medical studies and all kinds of scientific studies to become more accurate and informative about important issues. We especially have to improve our own tendency to seize a hopeful answer whether or not it's a true answer as we grapple with life's problems. (When I write "we" here I am including all of us, including myself, whether we are producers or consumers of scientific studies.) Medical studies specifically and scientific studies in general have helped humankind gain a lot of useful knowledge, but it has been a slow, unsteady process with lots of back and forth.

P.S. All the links I've posted here, including the one opening the thread, are from the "Weekend Reads" weekly post on the Retraction Watch group blog, which I think many Hacker News participants will find…

Scholarly Kitchen is the 'house' blog of the publishing industry, they've run very similar articles

>https://scholarlykitchen.sspnet.org/2018/09/18/guest-post-think-sci-hub-is-just-downloading-pdfs-think-again/

>Let me be clear: Sci-Hub is not just stealing PDFs. They’re phishing, they’re spamming, they’re hacking, they’re password-cracking, and basically doing anything to find personal credentials to get into academic institutions. While illegal access to published content is the most obvious target, this is just the tip of an iceberg concealing underlying efforts to steal multiple streams of personal and research data from the world’s academic institutions.

Comment 1:
>The notion that Sci-Hub is not involved in hacking is laughable. The founder of Sci-Hub, when asked about this, disingenuously replies that Sci-Hub itself does not engage in hacking and phishing, without disputing that it relies on these to operate. More evidence that there is something big and powerful behind Sci-Hub: the Russian mafia.

Or this howler, a blog-post on Russian information warfare that then goes into Sci-Hub:

>In the scholarly information community, some individuals apparently sympathetic with the open information calls of Sci-Hub and LibGen actively shared authentication information, inadvertently providing institutional credentials to cybercriminals and cyberwarriors, who are probably still sitting on the usernames and passwords or, more likely, the information they grabbed before the…

So, there was an earlier attempt at a preservation storage system with a "pay up front to fund perpetual storage", called the "Digital Preservation Network".

It came out of the library and archive sector, and focused on institutional customers -- libraries and archives with long-term archival/preservation needs. It was also non-profit. It of course seems attractive, if you really want to ensure long-term preservation, to set up a financial model that can somehow do this independent of the risk of the business continuity and budgets of the original depositors. And for cultural heritage institutions -- in the business of preserving incrementally more stuff every year -- it's a big advantage to budgeting if you can budget per GB you add to preservation storage up front, and never have to pay for it again -- so your yearly budget for preservation storage is just based on number of NEW TB's you are adding, instead of constantly going up with total number of TB's preserved, and thus this portion of your budget stays pretty constant instead of ever-increasing.

However, the DPN folded in Dec 2018, after fewer than 5 years of operation. It did not attain financial sustainability. (So all those up-front fees did not end up paying for perpetual storage after all).

You can read more about it's shutdown here:

https://duraspace.org/the-digital-preservation-network-dpn-to-cease-operations/

https://scholarlykitchen.sspnet.org/2018/12/13/digital-preservation-network-disband/

This new…

2) why the market failure?

Because the scientific reputation (and career prospects) are tied to publications in reputable journals, to certain titles. And the titles belong to the publishers. There has been considerable consolidation over time, the big 4 or 5 have just bought up smaller publishers over time, and it's now run not by idealistic academics, but (by all appearances) by cold-blooded money guys trying to squeeze out as much as they can.

Competitors can't just enter - there's a signalling problem: how do you distinguish scientists that don't publish in the top journal because they're idealistically opposed to their closed-access practices, from those that cannot get published there because their work is not up to standards?

1) Open Access Journals: the editors do some editing and vetting, and, crucially, peer review. If you put an article on your website, it doesn't have the same standing, "it doesn't count" as a real article or for your publication list.

So, in terms of scientific reputation, for now:

Traditional journal > OA journal > self-published.

The challenge is to get OA >= traditional, and that takes a lot of time, or, rather, coordination:

Don Knuth sent a letter in 2003 to the editors of the Journal of Algorithms to persuade them to collectively resign and move to a different journal. People of that stature are needed to break the power of despicable academic publishers (Elsevier is particularly revolting,…

Full disclosure: I'm a founder of a company called Scholastica that provides software that helps journals peer-review and publish open-access content online. One of our journal clients, Discrete Analysis, is linked to in the NYT article.

It is incredibly obvious that journal content shouldn't cost as much as it does.

- Scholars write the content for free

- Scholars do the peer-review for free

- All the legacy publishers do is take the content and paywall PDF files

Can you believe it? Paywalling. PDFs. For billions.

Of course the publishers say they create immense value by typesetting said PDFs, but as technologists, we can clearly see that this is bunk.

There's a comment in this thread that mentions the manual work involved in taking Word files and getting them into PDFs, XML, etc. While that is an issue, which you could consider a technology problem, it definitely doesn't justify the incredible cost of journal content that has been created and peer-reviewed at no cost. Keep in mind that journal prices have risen much faster than the consumer price index since the 80s (1).

The future is very clear, academics do the work as they've always done and share the content with the public at a very low cost via the internet.

PS. If you want a peek into how the publishers see the whole Sci-Hub kerfuffle, check out this post from one of their industry blogs - the comment section is a doozy:…