Internet Archive Blog

This has been playing out for many years. And it's all because Brewster Kahle decided that an overly broad interpretation of the Internet Archive's mission trumped the rights of authors and publishers, and the laws of the United States.

When IA was asked to stop CDL - many times - he continued. The National Writers Union tried to open a dialogue as early as 2010 but was ignored:

The Internet Archive says it would rather talk with writers individually than talk to the NWU or other writers’ organizations. But requests by NWU members to talk to or meet with the Internet Archive have been ignored or rebuffed.

https://nwu.org/nwu-denounces-cdl/

When the requests to abandon CDL turned into demands, Kahle dug in his heels. When the inevitable lawsuits followed, and IA lost, he insisted that he was still in the right and plowed ahead with appeals.

He also opened a new front in the court of public opinion. In his blog posts and interviews with U.S. media, Kahle portrays the court cases and legal judgements as a crusade against the Internet Archive and all librarians (see https://blog.archive.org/2023/12/15/brewster-kahle-appeal-statement/). It's not. It's the logical outcome of one man's seemingly fanatical conviction against the law and the people who work very hard to bring new books into being.

In addition, there has been real collateral damage to the many noble aspects of the Internet Archive. Legal fees and judgements have diverted resources away from the Wayback Machine,…

Before building an app that depends on the Wayback Machine (or other Archive infrastructure) it's good to keep in mind this post from their blog: <https://blog.archive.org/2023/05/29/let-us-serve-you-but-dont-bring-us-down/>

One of my favorite tricks when coming across a blog with a longtail of past posts is to verify that it's hosted on WordPress and then to ingest the archives into my feedreader.

Once you have the WordPress feed URL, you can slurp it all in by appending `?paged=n` (or `&paged=n`) for the nth page of the feed. (This is a little tedious in Thunderbird; up till now I've generated a list of URLs and dragged and dropped each one into the subscribe-to-feed dialog. The whole process is amenable to scripting by bookmarklet, though—gesture at a blog with the appropriate metadata, and then get a file that's one big RSS/Atom container with every blog post.)

I agree that Moby-Dick [1] is “far funnier, weirder, and gripping than you likely imagine.” But that is in contrast not only to contemporary novels that might—or might not—have been influenced by television. The vast majority of English-language novels published in the 19th century were also driven by plot and character and lacked long digressions about the author’s obsessions. Thousands of such novels, most of them no longer read, are now available at the Internet Archive [2]. Some of those might be worth reading, too.

[1] There’s a hyphen on the title page of the first edition but not in the body text or page heads:

https://archive.org/details/mobydickorwhale01melv/page/n7/mode/2up

https://archive.org/details/mobydickorwhale01melv/page/196/mode/2up?q=Moby

[2] https://blog.archive.org/2021/07/14/forgotten-novels-of-the-19th-century/

Interestingly, only one comment [1] in one thread from all the ones 'dang pulled up [2] mentions Carl (his son) at all. Despite the article calling out that Richard Feynman's involvement with the Connection Machine was to follow along w/ Carl's involvement:

> I got to know Richard through his son. I was a graduate student at the MIT Artificial Intelligence Lab and Carl was one of the undergraduates helping me with my thesis project.

Also of note is that Thinking Machines is where the Long Now and Internet Archive folks had worked together (see the "group photo" from [3]).

Definitely a unique group of folks came out of Thinking Machines!

[1] https://news.ycombinator.com/item?id=8683903

[2] https://news.ycombinator.com/item?id=28982737

[3] https://blog.archive.org/2021/07/21/reflections-as-the-internet-archive-turns-25/

http://blog.archive.org/tag/distributed-web/

My personal opinion is that they’re going about this the wrong way, with IPFS, anything blockchain, etc. They can support this entire model with a DHT (supporting PKI for identity) and torrent advertising, the latest version of BitTorrent (which has great support for cross swarm file serving by hash), and RSS or other feeds for advertising the corpus or chunks of it.

I maintain a metadata copy in R2 of all IA items and their metadata, and would love to work on the ArchiveTeam equivalent of a Warrior appliance: appliance or containers starts, consumes an RSS feed to request the least seeded items, download them up to disk space allowed, and then serve downloaded item files to the swarm.

> When you find a page [...] and you want to make sure it's available to you later, what do you do?

Instead of doing a bad and lossy job of archiving the page myself, I notify† our friendly neighbourhood archivists at the Internet Archive of the page; and they then do the best, most lossless job of preserving the page that they're able, given their cumulative experience.

† http://blog.archive.org/2017/01/25/see-something-save-something/

As a side-benefit, they also then take care of keeping the archive they've made around and available online in perpetuity, with no additional marginal effort on my part. The same can't be said for something in my own "private collection."