what actually is orchestration? [LWN.net]

what actually is orchestration?

Posted Sep 3, 2022 17:38 UTC (Sat) by rra (subscriber, #99804)
In reply to: what actually is orchestration? by rorycl
Parent article: The container orchestrator landscape

> Based on my admittedly inexpert research it is difficult to see how the concept of devops orchestration brings together the idea of creating a performance from containers in a way that makes sense in different cloud environments and equally in one's own racks.

It's interesting that you would say this because this is exactly the problem that my job solves with Kubernetes.

Our mission is to provide a reusable platform for scientific astronomy, initially targeted at the needs of our specific project, but hopefully generalizable to similar problems. This is a complex set of interrelated services and, perhaps more importantly, underlying infrastructure that handles such things as authentication and authorization and makes it easy to deploy additional astronomy services. And, vitally, we have to be able to run copies of the entire platform both in the cloud and in private data centers. The team I'm part of currently maintains six separate deployments, half in the cloud and half on prem, in addition to developing the infrastructure for the platform as a whole, and the same underlying infrastructure is deployed in three other on-prem data centers by other groups.

We went all in for Kubernetes and it was the best decision we ever made and the only way in which any of this is possible. Kubernetes abstracts away the differences in hosting environments, so that we can develop the hosting platform targeting Kubernetes and anyone who can deploy Kubernetes can deploy a copy of it. It works exactly the same on a cloud Kubernetes environment as it does in a private data center, with only minor changes required to customize things like underlying storage methods. It gives us a fairly tight interface and set of requirements for any new hosting environment: we can just say "give us Kubernetes of at least this version" with a few other requirements, and then we know our entire platform will deploy and work. There is absolutely no way that we could have done this as quickly or consistently, with a very tiny team, while trying to deploy directly on Debian, or even using something like Terraform. We need the additional layer of abstraction and it saves us an absolutely IMMENSE amount of work and debugging.

I'm saying this as someone who has been in this industry for approaching 30 years now and has done just about every type of system administration from hand-compiled GNU software trees in shared file systems through hand-rolled configuration management systems, Puppet, Chef, AWS at scale, and proprietary container orchestration systems; I'm not some neophile who has no experience with other ways of doing things. Kubernetes has its problems to be sure, and sometimes can be quite frustrating, but that orchestration layer lets you define very complex ecosystems of related applications in declarative code and deploy it in a hosting-agnostic way and that solves a critical problem for us.

to post comments

what actually is orchestration?

Posted Sep 3, 2022 21:53 UTC (Sat) by rorycl (guest, #151214) [Link] (1 responses)

> Kubernetes abstracts away the differences in hosting environments, so that we can develop the hosting platform targeting Kubernetes and anyone who can deploy Kubernetes can deploy a copy of it. It works exactly the same on a cloud Kubernetes environment as it does in a private data center, with only minor changes required to customize things like underlying storage methods.

> [Kubernetes provides an] orchestration layer lets you define very complex ecosystems of related applications in declarative code and deploy it in a hosting-agnostic way...

Thank you for these very helpful descriptions of the benefits of Kubernetes, particularly its use across heterogenous environments at scale and your comments about the time it has saved your team.

I would be grateful to know how your team deals with local development and if it uses automated testing with Kubernetes, possibly as part of continuous integration workflows. It would also be great to know what reference material you and your team has found most useful in its implementation of Kubernetes, particularly from a conceptual perspective.

what actually is orchestration?

Posted Sep 3, 2022 22:37 UTC (Sat) by rra (subscriber, #99804) [Link]

We do some very basic automated pull-request testing that mostly amounts to installing the whole environment on minikube inside CI and making sure everything installs and there are no syntax errors. Ideally we would then run some end-to-end tests on that environment; right now, we don't, and instead rely on a cloud-based integration cluster and automated integration testing.

Mostly for development beyond the basic unit test sort of stuff we use a dev cluster in the cloud (on Google Kubernetes Engine to be precise). It's just easier and less fiddly than a local install, and GKE is rock-solid. That of course comes with a monetary cost, although IMO it's pretty small compared to the cost of developers. But not being able to test locally easily is a bit of a gap that does occasionally cause problems, and while minikube is in theory an answer to this, in practice it's tricky to get all the pieces working happily on a laptop for typical local development (particularly on modern macOS, which a lot of people like to use but which adds the wrinkle of not being x86-based).

In terms of reference material, honestly I mostly just read the Kubernetes reference and tutorial pages on kubernetes.io (and of course implementation-specific guidance for specific cloud providers), plus the Helm documentation. But I joined a team that was already doing Kubernetes, so a lot of my training was from watching what other people were doing and asking questions, so I'm maybe not the best person to ask about initial reference material.

We use Argo CD to automate our Kubernetes deployment and maintenance, and I cannot recommend it highly enough. It makes it so much easier to automate the deployment process end-to-end and then be able to easily see what the cluster is doing and debug problems (and upgrade things, which is very important since we have a very fast development pace and are usually updating five or more times a week). I'll fall back on kubectl for some specific problems, but the Argo CD interface is usually more useful, and I say this as someone who almost always prefers command lines to any graphical tools.