Twistlock Blog

From a production Swarm operator without much experience with k8s:

When you create a new Swarm cluster with `docker swarm init`, Swarmkit creates PKI that is used for everything onward: gRPC cluster state communication, encrypted overlay networks, secrets, and so on. Keys are automatically rolled every 12 hours by default. This is done automatically and transparently to the operator, without any effort on their part. Adding workers or managers after that is as done with `docker swarm join --token <token>` on the new node, that's all.

I believe operation ease for starting a cluster and maintaining it for k8s is something that's going to get a lot easier soon from Moby and k8s joining forces.

Secrets. Swarm got encrypted secrets in January, and you create one with `docker secret create <secret-name> <file/stdin>` or the API. It looks like k8s got it as an alpha feature at the end of June and it doesn't look easy: https://www.twistlock.com/2017/08/02/kubernetes-secrets-encryption/
They are making secrets functionality plug-able, to match the flexibiliy in networks, volumes, and logging.

As far as development power goes, 9k contributors and 9k pull requests in the last year is nothing to scoff at. That's also a misrepresentation because of the effort put forth for CNI and container standards that make a lot of the work interchangeable.

k8s has a lot of nice stuff and a great, evolving ecosystem, but I think you are being a bit harsh on Swarm. I think Moby (and derivatives)…

We use Twistlock (https://www.twistlock.com/) as it does the CVE scanning but you can setup rules for compliance, binary monitoring and a whole plethora of other security/auditing type things. It also has a jenkins plugin so you can fail builds if a certain threshold of CVEs/compliance failures are introduced by developers (the only way to actually get the team to care about security).

Our security folks haven't really decided what to do with containers although some people are just using RHEL7 base images since its "enterprise-y". Our group personally uses alpine base images. If we have something like a java service hosted by Tomcat, we build alpine then build tomcat and then build our "service" container. While most people are fine pulling from Dockerhub, we do work in closed-loop environments and have a private docker registry where we host our "chain" of docker images which are versioned and updated regularly.

Not sure if this is what grandparent is referring to, but DNS rebinding[0] is what springs to mind.

It's simultaneously kind of smart and also really stupid. Basically, you give a valid 3rd-party domain multiple IPs, one of them normal and one of them resolving to a local IP. Then you cut off the normal one and the browser just allows you to make calls to whatever local interface you want.

There was a good defcon video about this a while back[1]. It's a much bigger problem than most people realize. This is why it's good practice to have at least some security around devices even if they're only connected to your LAN.

[0]: https://www.twistlock.com/2018/02/28/dear-developers-beware-dns-rebinding/

[1]: https://www.youtube.com/watch?v=stnJiPBIM6o

Excellent point :) So. First of all, you'd not have to worry about following, if you don't use containers.

If your app, running in Docker container, is compromised, like PHP webshell, it might try to escape the container. What capabilities you granted for your containers? CAP_SYS_ADMIN, CAP_NET_ADMIN or you have no idea? This is just one example of escaping https://www.twistlock.com/2017/12/27/escaping-docker-container-using-waitid-cve-2017-5123 and lets talk about namespaces, like user namespace. Is root inside container is a root user on host system?

Does it feels like production ready system?

Containers (and virtualisation) are helpful but. not bulletproof. If your threat model is already malicious code with ill intent, you probably want higher grades of privsep / sandboxing.

E.g.,

https://www.twistlock.com/labs-blog/escaping-docker-container-using-waitid-cve-2017-5123/

https://www.exploit-db.com/exploits/46978

Full virtualisation does far better, though I believe there've been exploits there as well. Yes, from 2015:

https://threatpost.com/xen-patches-7-year-old-vm-escape-hypervisor-vulnerability/115206/