Tenthousandmeters

> Async Python is still confusing af - when do I need it,

It's needed when you're spending a lot of time waiting for an I/O request to complete (network/HTTP requests, disk reads/writes, database reads/writes)

> what happens under the hood,

https://tenthousandmeters.com/blog/python-behind-the-scenes-12-how-asyncawait-works-in-python/#:~:text=likely%20to%20use.-,asyncio,-asyncio%20came%20to

(Please read the entire blog post - It goes through the necessary concepts like generators, event loops, & coroutines)

> does it actually help with performance,

Refer to the first answer: You'll see improved performances if your workloads are mainly comprised of waiting for other stuff to complete. If you're compute-heavy, it'll be better to use the 'multiprocessing' library instead.

> sometimes the GIL comes into play and sometimes it doesn't,

The GIL comes into play when you have a lot of compute-heavy tasks: Otherwise, you'll rarely encounter it.

It's only when you have that many compute-heavy tasks that you start to use the 'multiprocessing' library.

> why do we ever use threads at all if there's a GIL,

Threads exist because it was there before asyncio & event loops came into Python.

> why is it called asyncio if we can use it for anything.

Its name came from PEP 3156, proposing the asyncio library back in 2012.

https://peps.python.org/pep-3156/

As for why, asynchronous I/O stands in contrast to synchronous I/O, where the program/thread had to wait for the I/O request to…

Python, for example, has arbitrary precision integers. That means that it is theoretically possible to represent any whole number in Python, at least assuming your computer has enough memory to support it. Under the hood, the `int` object can have several different implementations depending on how large the number is. So small numbers will be represented one way, and larger numbers might be implemented as 64-bit integers, but very large numbers are implemented as an array of other integers that can grow arbitrarily large. You can think of the array as being like base-10 representation (so 17,537 might be represented as [1, 7, 5, 3, 7]), although in practice much larger bases are used to make the calculations quicker.

Obviously maths with the smaller representations will be quicker than with this array representation, so the interpreter does some work to try and use smaller representations where possible. But if you tried to, say, add two 64-bit signed ints together, and the result would overflow, then the interpreter will transparently convert the integers into the array representation for you, so that the overflow doesn't happen.

So the first poster said that the default merge sort implementation on Wikipedia was buggy, because it doesn't protect against overflows (assuming that the implementation used fixed-sized integers). The second poster pointed out that if the implementation used these arbitrary precision integers, then there is no chance of overflow, and the code…

I highly recommend Victor Skvortsov's exhaustive, 12-part series, "Python Behind the Scenes," a deep dive into the implementation details of just about every Python language feature one could possibly take for granted. There isn't a single one I haven't learned something from, but some of my personal favorites include:

* Part 2, How the CPython compiler works: https://tenthousandmeters.com/blog/python-behind-the-scenes-2-how-the-cpython-compiler-works/

* Part 6, How Python's object system works: https://tenthousandmeters.com/blog/python-behind-the-scenes-6-how-python-object-system-works/

* Part 10, How Python dictionaries work: https://tenthousandmeters.com/blog/python-behind-the-scenes-10-how-python-dictionaries-work/

* Part 13: The GIL and its effects on Python multithreading: https://tenthousandmeters.com/blog/python-behind-the-scenes-13-the-gil-and-its-effects-on-python-multithreading/

The whole series can be found here: https://tenthousandmeters.com/tag/python-behind-the-scenes/

For a slightly higher-level (i.e., no macabre, low-level C code), but still incredibly enlightening dive into the "dark side" of Python, in particular various abuses and hexes for manipulating control flow, have a look at David Beazley's smorgasboard of keynote presentations like "Generators: The Final Frontier" (https://www.dabeaz.com/finalgenerator/), "Python Concurrency From The Ground Up" (https://www.youtube.com/watch?v=MCs5OvhV9S4), and "Build Your Own Async"…

Hi! I'm the author of this article. Thanks for posting it.

The GIL is an old topic, but I was surprised to learn recently that it's much more nuanced than I thought. So, here is my little research.

This article is a part of my series that dives deep into various aspects of the Python language and the CPython interpreter. The topics include: the VM; the compiler; the implementation of built-in types; the import system; async/await. Check out the series if you liked this post: https://tenthousandmeters.com/tag/python-behind-the-scenes

I welcome your feedback and questions, thanks!

Pre-python 3.2, the format used for representing `str` objects in memory depended on if you used a "narrow" (UTF-16) or "wide" (UTF-32) build of Python.

Fortunately, wide and narrow builds were abandonned in Python 3.2, with a new way of representing strings : current Python will use ASCII if there's no non-ASCII char, UCS-2 –UTF-16 without surrogate pairs — if there is no codepoint higher than U+FFFF, and UTF-32 else.

See this article for a good overview of the history of strings in Python : https://tenthousandmeters.com/blog/python-behind-the-scenes-9-how-python-strings-work/

EDIT:

Paste this hacky snippet into your browser's console to save all of the links into your clipboard:

(() =>
{
let result = [...document.querySelectorAll('td a')]
.map((link) =>
link &&
!link.href.includes('ycombinator') &&
!link.href.includes('javascript:void')
? link.href
: ''
)
.filter(Boolean);
result = [...new Set(result)];
result.sort();
copy(result.join('\n\n'));
})();

Just to help, here's a compiled-list of all of the links shared here so…