Linux Plumbers Conference
6 HN recommenders · recommended in 15 months, 2013–2024
34:00 · 41K views · Oct 15, 2013
Paul Turner of Google presents kernel interfaces for user-level scheduling of real threads, aiming at a uniform synchronous programming model across C++, Java, Go and Python. It introduces the switchto-style system calls behind Google's fibers.
HN cites it as the key explanation of cooperative, user-scheduled kernel threads and the prior art behind later async and coroutine debates.
20 comments from Hacker News
Paul Turner explained this really well at this year's Linux Plumbers Conference. The whole talk is fantastic, but the explanation of what pron is describing in particular (and how it could be improved) starts around 8:39: this video
Google needs to get off its ass and open source Fibers (not to be confused with Google Fiber). Fibers sparked plenty of joy in me back when I was at Google. It's just a more dignified way of doing things with threads, IMO. Here's the talk which, unfortunately, does not show the higher level APIs I'm talking about: this video . If this were released today everyone would slap their foreheads and say "of course that's how it should be done". Google folks have already removed the obstacle in that direction by adding userspace switching to the kernel in recent kernel releases. Just go all the way now, and release the kraken. Last update I've seen on this is this article from June of this year: [LINK] Imagine how excruciating it must have been to all the Google people on the C++ Standard committee to work on this stuff that comes nowhere close to what they've been using internally for years, and not be able to talk about it in any kind of detail.
Oh OK, I definitely could have written that better. From what I can tell, M:N threading has been thoroughly abandoned by Linux, but it might be worth revisiting given the horrendous I/O scaling of Linux kernel threads. I would imagine scheduling user threads cooperatively simplifies things a lot. Google has a very interesting talk about cooperative native thread scheduling, but they haven't upstreamed their code: this video And here's what linux devs are actually doing (it's underwhelming, something like 50% of what the hardware is capable of): [LINK] In the meantime, people who actually need the performance (HPC and even some enterprise servers) have to bypass the kernel entirely. It looks like that's going to be possible even in virtualized environments, thanks to hardware support: [LINK]
If you're interested in this area you might like this talk from the Linux plumbers' conference: "Over the past year we've been looking at how a uniform, performant, synchronous programming model can be implemented across all of Google's core languages (C++, Java, Go, Python), while maintaining compatibility with existing code." this video
Allow me to third that. It's one of dangers assuming that what you learned at university 15-20 years ago still applies. Another interesting talk: [0] (and even that is 5 years old) [0] this video