The Cilium project's company and community blog, covering releases and eBPF-based networking, observability, and security for Kubernetes.
Commenters cite Cilium as a good example of new BPF packet filtering capabilities and as a Kubernetes networking choice in home clusters; Isovalent also posts job ads.
Cilium's container networking and security product [0]; Facebook's Katran [1]; Netflix's flowsrus (not public yet, but see my tcplife tool in BCC[2]). That's just the beginning. It's not just for performance, it's also security, as while BPF programs run in kernel-mode they have a limited and secured API for interacting with the system (BPF helpers).
Since extended BPF is in the Linux kernel (and will be in other kernels in the future), everyone is getting it, and we'll see more use cases over time. In some ways it's like the birth of JavaScript for the browser, and all the new applications it made possible. But it goes further than that: we could still analyze and debug JavaScript applications using traditional tools. But BPF programs are neither process-space or kernel routines, and are outside the view of everything. No visibility in ps(1), top(1), or lsmod(8). We're having to create new tools to even see what's running on the CPUs. Every performance monitoring product that shows a process table with CPU consumption will now need a BPF program table as well.
[0] https://cilium.io/
[1] https://github.com/facebookincubator/katran
[2] http://www.brendangregg.com/blog/2016-11-30/linux-bcc-tcplife.html
q3k has mentioned metallb+bgp, which is basically in-cluster implementation of LoadBalancer Service type (bgp speakers are running on k8s nodes and announce /32 routes to nodes based on configuration), but it doesn't provide an answer for "stabilizing" ecmp connections when there are changes to backends.
There has to be something "behind" metallb[1] that will handle not only stable hashing for connections, but keep forwarding "in-flight" flows (like established tcp sessions) to correct backends, even if packets arrive on different ingress nodes. It seems cilium has some solution for that[2] (by both bundling metallb, and having maglev-based loadbalancer implementation) but I haven't had time to dig into it, so I was curious if someone else has solved it and would be willing to share stories from the front. This is one of those rough edges around kubernetes deployments in bare metal environments and I'd love to see what can be done to make it more robust.
[1] metallb only really announces IPs so that "behind" is probably just CNI that actually handles traffic
[2] https://cilium.io/blog/2020/11/10/cilium-19#maglev
One of the linked blog posts has a decent overview of the what & why of BPF: https://cilium.io/blog/2018/04/17/why-is-the-kernel-community-replacing-iptables/
The issue with microservices at scale vis-a-vis microservices is linear scaling of routing tables, very short-lived IP connections, and large update times for massive service maps.
I think networking is complex in general, but the promise of this kind of integration (from my limited understanding), would be a clear reflection of services at the routing level instead of a hodgepodge of ports and IP addresses that provide little context or meaning. Having lost a few half-days learning iptable nuances in our Kubernetes cluster I can see the benefits in having an integrated stack with less impedance between architecture and routing.
Generally we don't eliminate complexity, we just move it from one place to another. I imagine this is a case where a more complex service implementation could provide a simpler user experience.
Isovalent | Multiple roles | Mountain View (US), Zürich (CH), or Remote
We're the company behind the open source Cilium project (https://cilium.io) (11K stars on GitHub) providing eBPF-based networking, observability, and security for container workloads and clusters.
We have an amazing and in-demand product using revolutionary technology and are looking for top talent to help us build and explore all of its possibilities.
We're remote-first, mainly in the EU and US timezones.
If you're interested please apply through our careers site https://isovalent.com/careers and mention Hacker News in your application.
Keywords for searchers: open source, Go/Golang, eBPF, C, C++, Kubernetes, networking, OpenShift, Linux kernel, performance, CI, SRE, technical writing, marketing, community advocate
hi,
what do you think about the approaches of CiliumProject ? (https://opensource.googleblog.com/2016/11/cilium-networking-and-security.html). They do have a Docker plugin (but does not work seamlessly with Swarm), but they claim to use BPF for superior performance https://cilium.io/blog/2018/04/17/why-is-the-kernel-community-replacing-iptables/
I think the Kubernetes and Swarm devs have generally leveraged IPVS as the next step from iptables. Any thoughts on going straight to BPF ?
There are still people working on packet filtering! The BPF enhancements are benefiting lots of different areas of the Linux world at the same time.
Cilium [1] is a great example of how the new capabilities of BPF can be used for more complex packet filtering, such as understanding higher level protocols.
[1] https://cilium.io/