Itamar Turner-Trauring writes about Python performance, Docker packaging, and data processing, aimed at developers shipping Python in production.
HN commenters call it an excellent resource for Python in containers and data science, and cite its posts on Alpine being slow for Python images.
Just to clarify for anyone who isn't aware, the "compiling issues", at least historically, have been that that Alpine uses musl, and PyPI's manylinux wheels are built against old glibc versions. So stuff like numpy that would trivially and quickly install from whl on glibc distros (like a bare-bones Ubuntu image) trigger compilations and the installation of build-only dependencies on Alpine.
That said, it looks like as of late-2021, at least some projects are offering musllinux wheels as well, per the discussion here: https://github.com/pypa/manylinux/issues/37 (not numpy, though: https://pypi.org/project/numpy/#files)
Here's a longer version of the above, originally published in 2020, but with an update regarding PEP 656: https://pythonspeed.com/articles/alpine-docker-python/
Sorry, but we did all of that 20 years ago, and those approaches remain a lot simpler for basic use cases. You know, like serving 100k dynamic pages/hour database backed site. (That's what I was doing 20 years ago, how about you?)
See https://pythonspeed.com/articles/dont-need-kubernetes/ for some of the reasons NOT to use Kubernetes for this.
With, of course, a giant exception for two cases. The first is, as with the author, if the purpose of using Kubernetes is to learn how to use Kubernetes. The second is if you are aware of the tradeoffs and have specific reason to say that Kubernetes really is appropriate for your circumstance.
And if you think that Kubernetes is always the right approach, then you clearly are NOT aware of the tradeoffs.
imho:
A false positive CVE list is an issue elsewhere as well, and it's important to understand that there's not much a Docker Postgres maintainer can do if the problem lies in the Debian or Ubuntu package and isn't getting fixed for some reason.
https://github.com/docker-library/faq#why-does-my-security-scanner-show-that-an-image-has-cves
It's also advisable not to use the default settings:
https://pythonspeed.com/articles/docker-security-scanner/ "trivy --ignore-unfixed <image>"
Of course, it is advisable for the image maintainer to rebuild the Docker image weekly or bi-weekly to ensure all recent patches are included.
However, for those who prioritize security, it is best to build the image themselves to guarantee up-to-date packages.
Using alpine with Python in production is usually a mistake. Python is considerably slower with musl libc than with glibc and on top of that no one is providing built wheels for musl-based platforms so you need to drag in the entire buildchain to package most projects. Who wants to build numpy from scratch as part of their build pipeline? And having a C compiler present is not great as it allows exploits to escalate their seriousness considerably.
Anyway most of these security scanners are bunkum:
https://pythonspeed.com/articles/docker-security-scanner/
There are not, in fact, a huge number of known and unpatched vulnerabilities in debian stable. If your tool is finding numerous problems, it is time to look more closely at your tool.
> offering almost a Python like experience
If you do scientific computing, the fortran experience is much better than python's. You never feel out of place. Multi-dimensional arrays of floats are a native type, loops are fast, and the system is extremely stable.
What does python offer you? Lists, dictionaries, strings (none of which is of any use to scientific computing) and a plethora of slightly incompatible external libraries for dealing with matrices. Worse, there's no hope [0] that your algorithms based in numpy/scipy/numba/tensorflow/torch/jax will run unchanged in a decade, not to say in 50 years.
[0] https://pythonspeed.com/articles/numpy-2/
I think the point is that, from a compiler's perspective, it's not obvious how much you should be allowed to optimise code at the cost of changing the outcomes of floating points maths - do you allow 1e-10, or 1e-6, or 1e-4 level changes? Does your compiler have to run some test calcs to bound the scale of the change introduced by rewriting fp maths? Some compilers will let you opt in to rewriting floating point maths, but that's opt in so users understand that their numeric outputs might change between optimisation levels.
For more, there's a good post on this kind of flag in Rust: https://pythonspeed.com/articles/faster-float-math-rust/