App Container spec gains new support as a community-led effort [LWN.net]

App Container spec gains new support as a community-led effort

Posted May 6, 2015 18:29 UTC (Wed) by geertj (guest, #4116)
Parent article: App Container spec gains new support as a community-led effort

Post from Red Hat related to this announcement:

http://rhelblog.redhat.com/2015/05/05/rkt-appc-and-docker...

(Note: my employer, but not my blog post)

to post comments

App Container spec gains new support as a community-led effort

Posted May 6, 2015 21:28 UTC (Wed) by raven667 (subscriber, #5198) [Link] (8 responses)

That pretty much nails my feelings on the matter, especially the last two paragraphs, and makes a good comparison to the fundamental schism in the Linux Distro market caused by the arbitrary differences between apt/dpkg and yum/rpm which has split the ecosystem, Docker and CoreOS seem to be making the same mistakes as Debian and Red Hat in the past.

App Container spec gains new support as a community-led effort

Posted May 6, 2015 21:40 UTC (Wed) by dlang (guest, #313) [Link] (6 responses)

I remains to be seen what will happen with containers.

Getting into the container game with the format being completely controlled by your rival is a loosing game. If Docker can join in an open format definition (either on it's own for the good of the ecosystem, or by being forced to by enough people getting behind an alternate approach), this this would be a good thing.

On the other hand, the difference in the formats of different VMs doesn't seem to be a critical flaw (as annoying as it is sometimes)

App Container spec gains new support as a community-led effort

Posted May 6, 2015 23:28 UTC (Wed) by snajpa (subscriber, #73467) [Link] (5 responses)

Containers as a technology have almost nothing to do with what Docker and the others like it actually represent. Containers as a means to isolate processes from each other on a kernel level have been around long before Docker kids launched this "container" hype madness.

But the hype is not about containers as such, I like to think of Docker a "doctrine" for deploying applications in a predictable and massively reproducible way. Docker is just an implementation of that "doctrine", but the problem with Docker is, that it is primarily about the implementation itself, when it really should be about (preferably open) standards instead.

Please, stop confusing containers and application deployment methods, containers are a runtime for the deployed application, but even Docker does support more runtime back-ends (though their terminology might differ, I don't use Docker, so please excuse my ignorance).

App Container spec gains new support as a community-led effort

Posted May 6, 2015 23:41 UTC (Wed) by dlang (guest, #313) [Link] (4 responses)

> Containers as a technology have almost nothing to do with what Docker and the others like it actually represent.

technically yes, however that's not how people are using the term.

> But the hype is not about containers as such

it sure looks like it from my point of view. A few years ago the hype was "virtualization" and how that would save everything. Now people are talking about "containers" in the same way.

the real issue is trying to decouple application run environments from the OS that they are running on, without throwing away a lot of performance (i.e. money) in the process. A lot of the push is from people who have things running on RHEL5 (or earlier) and don't want to ever update anything, for fear that they will break something. They can get away with this for a while, but eventually the lack of security updates is going to bite them, but usually they will be off at a different job by then.

App Container spec gains new support as a community-led effort

Posted May 7, 2015 21:31 UTC (Thu) by kleptog (subscriber, #1183) [Link] (2 responses)

> the real issue is trying to decouple application run environments from the OS that they are running on, without throwing away a lot of performance (i.e. money) in the process

While this is a nice benefit, this isn't the reason I'm doing it. The problem with deploying applications that you want to know exactly what version is deployed where. Packaging is the first step but applications need to manage ports and run multiple programs which packaging alone can't do easily.

For me the real innovation with Docker is the *scripted deployment of applications* i.e. Dockerfiles. A developer can test on their local machine the exact environment as it will run in production. When a security update needs to be deployed, you kick the buildbot to generate a new image, you can test it and deploy it, replacing the old one. And rollback if it breaks.

So far I haven't found anything as nice for, for example, VM images. I at first thought Vagrantfiles could do it, but they are for controlling the VMs, there's nothing there to help with the creation of images.

So really, standardising the container format is completely uninteresting for me. Send me your Dockerfile and I'll build the image locally using whatever tool (though I guess only Docker understands Dockerfiles). I'm not at the point where I feel I can trust random images downloaded off the internet.

FWIW, I don't see anything about Rocket that resembles Dockerfiles, I hope they're not neglecting that aspect.

App Container spec gains new support as a community-led effort

Posted May 10, 2015 3:58 UTC (Sun) by misc (subscriber, #73730) [Link] (1 responses)

> Send me your Dockerfile and I'll build the image locally using whatever
> tool (though I guess only Docker understands Dockerfiles). I'm not at the
> point where I feel I can trust random images downloaded off the internet.

I think there is much more work needed than just getting the dockerfile to reproduce a docker container. As soon as it does something like "gem install foo" or npm, pip, or yum/apt, without exactly pinning the version, you have lost. Now, distributions at least try to keep compatibility in stable releases, but based on my own experience with ruby, this is not something all upstream or developers take in account.

The problem of course is that it work well enough for people to believe that, until this is deployed at scale. That's like race condition. Rather hard to track, work most of the time until suddenly, a mysterious failure happen.

> FWIW, I don't see anything about Rocket that resembles Dockerfiles, I
> hope they're not neglecting that aspect.

You could easily replace that with ansible or puppet ( or others ). The dockerfile format, while not totally horrible, still have some problems IMHO who would make it less than ideal for large scale usage.

For one, you cannot extend it ( no macros like rpm specs, for example ). While this permit to have uniform file accross deploy, it doesn't permit to encode the best practice in a easy way.

Another issue, you cannot parse it easily. Again, something like xml, yaml or json permit to create static analyzer who permit to verify and enforce best practice. Doing so for docker mean you would have to duplicate the parser, against a spec which is (afaik ) not written. Languages like python or specialized DSL like puppet have various tools, most reusing the AST of the compiler ( bandit is one example, puppet-lint is another ). And yes, I realize that it is hard to satisfy having a easy to parse language and being extensible. But here, none is done.

The syntax itself is inspired of one of the most annoying design decision of the rpm specs file, reusing the shell as the mechanism of construction of the system. Note that I said annoying, not bad, because I think you do not have the choice on this. But having specs files as shell scripts with macros make then hard to parse, prevent having nice things like "being able to fully revert a rpm" due to %post being in shell. And arguably make repeatable build harder.

Do not get me wrong, I am quite they didn't had the choice for that, given the legacy we have in term of build scripts all over the place. But like haskell do not have the choice to offer IO functions but explicitly tell this will cause issues, I wonder how much the requirement for running shell snippet could have been reduced.

App Container spec gains new support as a community-led effort

Posted May 12, 2015 7:32 UTC (Tue) by kleptog (subscriber, #1183) [Link]

> I think there is much more work needed than just getting the dockerfile to reproduce a docker container. As soon as it does something like "gem install foo" or npm, pip, or yum/apt, without exactly pinning the version, you have lost.

True. However, most such tools do provide the ability to fix versions and that does work. And you now have a sensible way of building an image against an own hosted PyPI or Apt repository. You still get the benefit of having a single build and being able test in multiple places.

> The dockerfile format, while not totally horrible, still have some problems IMHO who would make it less than ideal for large scale usage.

The Dockerfile format isn't great, it has some nasty corners but it doesn't require any help from the container. The Puppet DSL is nice, but the agent is enormously heavyweight (IMHO) and has versioning issues of its own. Ansible doesn't necessarily require an agent but I find its use of YAML makes things fairly hard to read (but that might get better with practice). There's definitely some innovation required in this area.

Still, I've managed to get several people to write Dockerfiles for various things while I've never really gotten people to write much in the way of Puppet files. So they're doing something right.

App Container spec gains new support as a community-led effort

Posted May 8, 2015 5:17 UTC (Fri) by drag (guest, #31333) [Link]

> the real issue is trying to decouple application run environments from the OS that they are running on, without throwing away a lot of performance (i.e. money) in the process.

The ultimate goal is to make running multiple applications manageable in Linux.

The thing that I always remember is that dealing with virtualization and containers is that you are seeing the same patterns that have been tried over and over and over again.

The reality is that people are just re-creating the Java VM. Although instead of using a C++-based environment designed from the ground up for then-modern object oriented paradigms in programming with optimized bytecode and simplified cpu architecture... now we are doing it with x86-64 machine code as the 'bytecode' and are using a existing GNU-style userspace as the runtime.

I figure that the sooner people start thinking these things as executable application run times and stop thinking them as 'containers' and 'virtual machines' the easier it's going to be to deal with all of this.

App Container spec gains new support as a community-led effort

Posted May 7, 2015 5:22 UTC (Thu) by misc (subscriber, #73730) [Link]

That's a simplification of the problem. Ubuntu packages are not made to be run on Debian and vice versa. Packaging formats are the tip of the iceberg when it come to the myriad of choice made by distribution, and the impact on a proper integration.

If the package was the only problematic part, upstream would distribute static binary and downstream users would jump on it. And the various attempts to fix the problem ( autopackage, etc ) would have worked fine.

It turn out that they didn't got any traction, so either the problem is more complicated than format, or that's not a problem big enough for anyone to justify the cost of changing.

So the same goes for docker images or anything, there is a lot more than superficial differences such as package format. Given that systemd nspawn is able to work with both docker and rocket, it should cause trouble.
Now, the way you compose image together matter ( ie, what overlay network do you use, how do you do discovery in a micro service architecture, what port you expose, etc ), the content of the image matter ( cf this often cited article : http://www.fewbytes.com/docker-selinux-and-the-myth-of-ke... ), how and who update the image matter a lot more than the format ( as summarized on http://www.vitavonni.de/blog/201503/2015031201-the-sad-st... ).

Heck, even the orchestrator will play a big role, from full featured ones such as openshift v3/kubernetes or mesos/aurora to more simplistic one such as the one shipped by docker inc.

So in the end, you will have incompatibilities. As long as people want to have the choice, and as long as it make sense from a resources point of view to focus on one single choice rather than supporting the matrix of combination, you run into that. package format, containers format, etc, it all go the same way since years and for years to come.

App Container spec gains new support as a community-led effort

Posted May 7, 2015 9:37 UTC (Thu) by ibukanov (subscriber, #3942) [Link] (2 responses)

The blog mentioned two incompatible specs. I wish there would be a spec for Docker! So presently there is only one spec but that is incompatible with the most popular container format.

App Container spec gains new support as a community-led effort

Posted May 7, 2015 15:50 UTC (Thu) by misc (subscriber, #73730) [Link] (1 responses)

I did see some work from docker to have a spec on the container format https://github.com/docker/docker/blob/master/image/spec/v...

But it is not clear if you speak of this or another area that need a spec.

App Container spec gains new support as a community-led effort

Posted May 7, 2015 16:39 UTC (Thu) by ibukanov (subscriber, #3942) [Link]

I was not aware of that spec. It could be that CoreOS efforts on app container specs pushed Docker to document at least their image format. The first commit is just 4 months old, about 2 months after CoreOS announcement.