Wikimedia's engineering team describes how Wikipedia's infrastructure works, including its CDN, datacenters, HTTP performance, and maps serving.
HN commenters link it for posts on PHP HTTP connection pooling, the mobile and desktop domain merge, and HHVM history.
>I had a major eyesight scare this year, and it has gotten me thinking a lot more about accessibility.
Thankfully it was just a scare. My family has some eyesight issues, so I've always been curious on how visually impaired users navigate the web.
It helps that in my high school there was a boy that was born blind. He used a braille machine during class, until he got a thinkpad configured with a screen reader. Later he joined a bank working in IT, and I remember he already knew Linux very well during high school (early 00's).
>Can anyone point me to the best resources which might help me understand how visually impaired users navigate the internet today?
I use the Wave accessibility tool [1] on my websites, and whenever I am reviewing pull requests for $work, or Apache, Jenkins (it&plugins do very poorly), and other Open Source sites (latest one was the CWL Conf 2024 [2]).
Another thing I use to test accessibility is try to navigate the site with Orca (with/without looking at the screen). You can install that on any Linux machine, completely free. Microsoft screen reader is better than Orca, or at least was the last time I checked it. But I am a Linux user -- and plan to continue being even if my eyesight gets really bad.
It is relatively easy to learn Orca, you just need to learn a few shortcuts, and the commands to enable/disable it. And testing web sites will help you both learn to use the screen reader, while also helping other visually impaired users if you submit…
> I knew from hard experience with Darwin that internally siloed open source projects cannot thrive (HHVM was a repeat lesson)
I'm glad HHVM happened, and also glad it stalled. I don't think PHP 7 and 8 would've made the improvements they did without HHVM kicking their ass, and I think there would've been a fork based on HHVM rather than PHP 8 if HHVM hadn't lost that public momentum.
I remember Wikimedia's testing/partial implementation of HHVM[1] being a turning point, at least in the circles I was in at the time. It showed PHP performance could actually be improved, and by quite a lot. Without that proof of concept at that scale _in the open_, HHVM devs could've ran benchmarks from here to eternity and people still would've said, "yeah, sure, _if you're Facebook_"
1: https://techblog.wikimedia.org/2014/12/29/how-we-made-editing-wikipedia-twice-as-fast/
This is indeed correct. Wikimedia overall uses less than 2000 bare-metal servers, so yes the infrastructure is tiny compared to those.
What can be interesting, I think, is that you have a completely open infrastructure that has to solve problems on a global traffic scale.
If people are interested in knowing more, I suggest you also take a peek at the wikimedia techblog, specifically to the SRE category https://techblog.wikimedia.org/category/site-reliability-engineering/ and the performance one https://techblog.wikimedia.org/category/performance/
This may finally be a solution for scraping wikipedia and turning it into structured data. (Or do we even need structured data in the post-AI age?)
Mediawiki is notorious for being hard to parse:
* https://github.com/spencermountain/wtf_wikipedia#ok-first- - why it's hard
* https://techblog.wikimedia.org/2022/04/26/what-it-takes-to-parse-mediawiki-page-titles-in-rust/ - an entire article about parsing page TITLES
* https://osr.cs.fau.de/wp-content/uploads/2017/09/wikitext-parser.pdf - a paper published about a wikitext parser
Here is one that looks promising: http://techblog.wikimedia.org/2010/09/video-labs-p2p-next-community-cdn-for-video-distribution/
It's a CDN that uses P2P technology to help distribute video. This would come in very handy for viral videos...if 30 of your coworkers have already watched the video, then you could stream it from them rather than fetching it from an edge server (in the case of a traditional CDN).
This is indeed a downside but there are workarounds -- The wikimedia foundation put out a good blog post talking about getting HTTP connection pooling for PHP
https://techblog.wikimedia.org/2020/10/26/impact-of-using-http-connection-pooling-for-php-applications-at-scale/
TLDR they used envoy proxy as a sidecar container. So to your point its a pain but still achievable