Mesosphere Blog

Disclosure: CoreOS employee; Kubernetes contributor.

After reading the title I was a bit tempted to call BS since I think Kubernetes should have the rights to "The first Datacenter OS" tagline[1]. However, after reading up on the project details at https://mesosphere.com/learn I can see how Mesosphere came to the conclusion of being a DCOS, if not necessarily the first. Mesosphere goes a bit further than Kubernetes and offers a solution to the storage problem and attempts to address other "userland" concerns by shipping Apache Spark, Cassandra, Kafka, and Hadoop. So maybe it would be more accurate to call this a datacenter distro on top of the Kubernetes kernel?

Regardless, I think the concept of a datacenter OS will be the key to commoditizing IaaS providers and leveling the playing field in terms of features and usability for those who have not given up on the dream of running a "private" cloud.

Why will the DCOS work where others have failed?

Current solutions aimed at taming the datacenter operate at the machine/VM level, which exposes the OS for each machine, and completely punts on the application. Guess who gets to stitch it all back together? A DCOS is designed to manage applications directly, commonly via application containers, which means we can treat the OS running on the underlying machines like firmware and limit our interactions to basic updates and minimal configuration -- think CoreOS.

What about PaaS?

That's a topic worthy of a lengthy discussion, but…

Why is this awesome?

Production ready Mesos hosted in the cloud.

Now I dont have to run a $500M company to cut server costs and save dev ops time. Apache Mesos is a system predominatly used by large companies, to more efficiently utilize servers in their data centers.

The traditional way of organizing your servers requires a certain number of servers per cluster. For example: 50 webservers for your web app in one cluster, 5 servers for your database in another cluster, and 10 servers for redis, etc. On average, you'd utilize 50% of each cluster, but have the extra capacity for spikes.

Mesos makes your servers way more efficient by treating them as raw power, allowing any server to run any kind of app/task. It chops a server up into many linux containers and can shuffle around your tasks, so that a web app, redis server, and database server could all be running in different areas of the same "physical" server. I read that one company saved ~40% of their server costs and served the same load just by switching to Mesos.

Marathon is made by Mesosphere and acts as the brains or controller for your servers. You can allocate what percentage of resources go to certain apps/tasks, and it also handles deployment, scaling, failover and restarting.

Mesos supports Docker, so converting an existing app to run this on DO with Mesosphere is simple. Some applications are built for Mesos already, like Spark (MapReduce replacement), and will be inherently faster without the Docker…

It's worth noting that Kubernetes can also run on top of Mesos (https://github.com/GoogleCloudPlatform/kubernetes/tree/master/contrib/mesos) and DCOS (https://mesosphere.com/learn/).

So if you invest in Kubernetes and later want to take advantage of Mesos you can lift up k8s and put Mesos underneath.

Mesos is definitely more mature, and you can use Marathon for container orchestration, which is a little more mature than Kubernetes, but it's also a bit simpler. Marathon doesn't have the service, secrets, or pod abstractions, for example.

DCOS adds some additional benefits on top of Mesos, like the dcos-ui and dcos-cli. One of the more compelling features is one-step cluster package installation (ex: dcos package install kubernetes). Currently, the community edition is available on AWS (https://mesosphere.com/amazon/).

I do agree that the kube-up scripts in k8s are a bit of a mess. There's really hard to read and reverse engineer, and each provider has their own divergent deployment methods. That's one of the things DCOS is trying to standardize, to have a consistent deployment pattern for Mesos frameworks that deploys their core components inside Marathon, giving them a more battle-tested platform to live within, and granting automatic resurrection in case of failure.

Mesosphere (https://mesosphere.com) | Remote | Software Engineers | Full-time

Mesosphere is heavily involved in the development of Apache Mesos with the vision to build an infrastructure to utilize your entire datacenter or cloud instances like one big computer.

In this role there will be a large variety of features to work on, including resource allocation, job scheduling, containerization, provisioning, metrics, logging, task preemption, checkpointing, fault-tolerance, auto-scaling, modularization, testing, and much more. All of these involve modern C++ libraries that facilitate asynchronous I/O handling for distributed computing and make concurrency and manual memory management fun again. As an added bonus, majority of your work will be upstreamed into Apache Mesos project giving your work global visibility and provide an avenue to become a committer of a popular OSS project!

We are hiring in SF, Hamburg (Germany) and Remote.

https://mesosphere.com/careers/?gh_jid=1265717
https://mesosphere.com/careers/?gh_jid=1265690
https://mesosphere.com/careers/?gh_jid=1229864

I'm the hiring manager and would love to hear from you.

Mesosphere is a company, not a piece of software. I think you're referring to Mesosphere's dcos[1].

DCOS is some really nice packaging for marathon, chronos, etc, with a nice cli tool for downloading and installing new frameworks onto mesos. Personally, I find using Aurora a lot nicer.

That being said, I do think the kubernetes-mesos gives you far and above the best of both worlds. You get the developer story of kubernetes, with the ops story of mesos.

From an ops perspective, k8s is a bit clunky. I was actually shocked when I found out after bringing a new kubelet (worker node) online, you have to also update a service on the master. This was in a power training class on Kubernetes at this year's Redhat Summit in Boston. Really underwhelmed with the complexity of k8s compared to mesos, but they aren't an apples to apples comparison.

[1] https://mesosphere.com/product/

Mesosphere, Inc. is hiring a variety of software engineers to help build the Datacenter Operating System, based upon Apache Mesos. If you're looking to work on distributed systems, large clusters at scale, containers & microservices, and big data frameworks, we'd love to hear from you. Languages we use include Scala, Java, Erlang, Go, C++, Python, & Javascript.

We start the process with an intro call, then a coding challenge + review call, followed by a full day of onsite interviews.

We are hiring for full-time roles in our SF, CA and Hamburg, Germany offices, and will sponsor visas wherever possible.
If it makes sense for you and works for our team, we are open to remote, home office working arrangements, in both the US and EMEA.
All openings:
https://mesosphere.com/careers/