Docker Blog

There are a great many factual errors in this article and I'm sure someone will write a proper response, but I thought someone should address the most glaring:

- The repeated assertion that you can only use one process container is false. As an example, the most popular non-official base image (phusion/baseimage) provides a full system including SSHd and Cron

- No integration points

Docker provides an extensive event stream[0], as do their tools (such as Registry[1])

- "ps, ls, find [...] are useless here"

This depends on your system setup; on a standard Debian setup, all container processes are visible to the actual system. ps, top, netstat, etc will all work just fine

- "You can not easily and directly access your durable non-volatile data storage"

One of the most basic concepts in Docker is that of a "volume"[2]; essentially a directory that bypasses Docker's abstraction layers, for performance and durability. Additionally, using your normal storage is absolutely trivial[3] - if Linux can see it, you can use it.

- No authentication for images

This one's true to a certain extent. While most images aren't currently signed (that's a relatively new feature[4]), it's disingenuous to say that there's no authentication at all. Docker enforces the use of HTTPS to talk to registries, and requires HTTP authentication for registry accounts. It's not signing, but it's something.

Finally, just as it's a personal bugbear of mine, I would note that the userland proxy is a very…

Docker is a couple different things in one, so in that respect, it works in different ways.

The main thing Docker does is run arbitrary commands with (or like) LXC, which is basically a way of running a program inside a chroot environment, plus providing a more low-level separation for various parts of the kernel. The end result is your host is kept insulated from the program being run. Here's a great HOWTO on setting up LXC: http://lxc.teegra.net/ And here's a great StackOverflow answer on what Docker/LXC does: http://stackoverflow.com/questions/16047306/how-is-docker-io-different-from-a-normal-virtual-machine

This page from Docker's blog discusses not only the security aspects of a container, but a comparison to traditional virtual machines: http://blog.docker.io/2013/08/containers-docker-how-secure-are-they/

And the best technical summary from Docker's documentation on what a container is:

"Once you start a process in Docker from an Image, Docker fetches the image and its Parent Image, and repeats the process until it reaches the Base Image. Then the Union File System adds a read-write layer on top. That read-write layer, plus the information about its Parent Image and some additional information like its unique id, networking configuration, and resource limits is called a container."

On top of that it adds a Registry, an Index, and a Remote API. So really, it has remote deployment features, it builds chroot environments, and it combines them and runs applications in…

I think most of the observations of that talk still stand.

> System software became about managing large numbers of machines.

Yes, but is academic research on these topics relevant? Most of the progress on infrastructure software has come from the attempts of internet companies to cope with their big data problems. The people who implemented these systems have all received their phds from academic CS institutions and they used bits and pieces of existing distributed systems research, but they did their innovative work in the context of commercial companies, not academic CS departments.

> Good new languages were developed.

And what are they? A quick look at https://tiobe.com/tiobe-index/ (for the lack of a better resource) corroborates the thesis that the bulk of software is developed using the same old boring languages. Yes, they have evolved, but not much. Golang is, well, not exactly innovative. Rust still has a long way to go to be called a mainstream language.

> Things running in containers on in VMs need far less of an OS, and a lot less OS state.

Unikernels exist but is there a compelling non-niche use case for them? Docker acquired Unikernel systems and that's what we got so far: https://blog.docker.com/2016/05/docker-unikernels-open-source/

These are all great examples confirming the thesis that producing good relevant systems research is hard. On the one hand we are piling up gross inefficiencies on top of decades old technology, so improving the current…

>Who is the author of aufs or overlayfs? Should these projects work with no recognition while VC funded companies with marketing funds swoop down and extract market value without giving anything back. How has Docker contributed back to all the projects it is critically dependent on?

Here's an exert from a docker blog post from 2013[0]

>>Under the hood Docker makes heavy use of AUFS by Junjiro R. Okajima as a copy-on-write storage mechanism. AUFS is an amazing piece of software and at this point it’s safe to say that it has safely copied billions of containers over the last few years, a great many of them in critical production environments. Unfortunately, AUFS is not part of the standard linux kernel and it’s unclear when it will be merged. This has prevented docker from being available on all Linux systems. Docker 0.7 solves this problem by introducing a storage driver API, and shipping with several drivers. Currently 3 drivers are available: AUFS, VFS (which uses simple directories and copy) and DEVICEMAPPER, developed in collaboration with Alex Larsson and the talented team at Red Hat, which uses an advanced variation of LVM snapshots to implement copy-on-write. An experimental BTRFS driver is also being developed, with even more coming soon: ZFS, Gluster, Ceph, etc.

I think the real nature of your complaint is that you want the recognition to be in the form of cash.

[0] https://blog.docker.com/2013/11/docker-0-7-docker-now-runs-on-any-linux-distribution/

This is a very good tutorial! Love the end-to-end examples. Great teaching method as it gives an overview of what you want to achieve without looking at things in isolation. Wanted to add one bit of info and give a heads up about the Docker for Mac & Windows beta at https://beta.docker.com/.

> If you are on Mac or Windows, consider using a Virtual Machine. I use Parallels on Mac OS X to run an Ubuntu machine for most development activities. Being able to take snapshots, break things and then revert back is very handy when experimenting.

The beta will make is super easy to use Docker on Mac & Windows without running a heavy Virtual Machine. Here's a little video about it @ https://blog.docker.com/2016/03/docker-for-mac-windows-beta/. Basically, it feels like running Docker natively, has upgrade features build it, so you can choose to always be on the latest release (super nice). I've been using it for a few months and it is awesome to just type "docker run" on my macbook and have it work seamlessly (vs using third party virtual machine software and env bits). The idea is that you can develop on Mac & Windows using the same images you will eventually deploy. It just works (although I'm biased).

Disclaimer: Work for Docker.

Through all the noise, this comment is exactly what I care about. For a new project, which tech's should I choose?

If I were Docker, I would organize the ecosystem for the end user on a nice clean website that explicitly lists the specific layers and technology options, some of which are from Docker, and some of which are from third-parties. Yes, they'll give mention to competitors, but at least it creates clarity and puts some meat behind their claims of "Open".

At present, I'm thinking of the following:

* CONTAINER: Stick w/ Docker since until a competing standard like Rocket is mature, everyone has to support Docker, even CoreOS

* CONTAINER MANAGEMENT: Use CoreOS. Per above, they have to support Docker until Rocket is fully baked. Then Rocket and CoreOS will probably have synergies you can't get with Docker / CoreOS. Also, from [1]:

These three [Docker] orchestration services are targeted
for general availability in Q2, 2015.

I honestly don't know what the other layers and decisions are yet. Perspective and clarification is welcome.

[1] http://blog.docker.com/2014/12/docker-announces-orchestration-for-multi-container-distributed-apps/