LWN: Comments on "Creating Kubernetes distributions"

Creating Kubernetes distributions

rahulsundaram — Mon, 16 Dec 2019 20:14:48 +0000

Not obvious to me. If a distribution of Kurbnetes passes all conformance tests, I would consider it compatible. If not, that would be a failure in the conformance tests by definition

Creating Kubernetes distributions

ofr — Mon, 16 Dec 2019 20:09:27 +0000

Reference for what exactly? It's obvious that it's not 100% compatible because many software packages for Kubernetes don't run unmodified on OpenShift. For the claim about the conformance test see https://github.com/openshift/origin/blob/master/test/extended/conformance-k8s.sh

Creating Kubernetes distributions

rahulsundaram — Sun, 15 Dec 2019 11:13:46 +0000

> It's also not 100% Kubernetes compatible even if it passes the conformance check by disabling security.

Can you provide a reference to this?

Creating Kubernetes distributions

ofr — Sun, 15 Dec 2019 10:31:35 +0000

It's also not 100% Kubernetes compatible even if it passes the conformance check by disabling security.

Typhoon is a free Kubernetes distribution

ofr — Sun, 15 Dec 2019 10:29:55 +0000

There's already a free (as in free software) Kubernetes distro. It is called Typhoon: https://github.com/poseidon/typhoon

"Typhoon distributes upstream Kubernetes, architectural conventions, and cluster addons, much like a GNU/Linux distribution provides the Linux kernel and userspace components."

So, there you go.

Python as a shell replacement

karkhaz — Sat, 14 Dec 2019 00:08:27 +0000

I occasionally use a combined shell script/makefile if I care about catching errors on each command:

#!/bin/sh
# vim:set syntax=make:set ft=make:

MAKEFILE_START_LINE=$(\
  grep -nre makefile_starts_here "$0" \
  | tail -n 1 \
  | awk -F: '{print $1}')

TMP=$(mktemp)
tail -n+${MAKEFILE_START_LINE} "$0" > "${TMP}"

make -f "${TMP}"
SUCCESS=$?

rm -f "$TMP"
exit "$SUCCESS"

makefile_starts_here:
	command-1
	command-2
	command-3

This prints out everything below and including "makefile_starts_here" to a Makefile and then runs make on it, executing the commands one at a time. This is especially nice if I want built-in parallelism etc, it's actually even better than just using the shell (just ensure to print out "MAKEFLAGS=-j" at the top of the file).

Python as a shell replacement

marcH — Fri, 13 Dec 2019 22:59:42 +0000

Yes, set -e is absolutely not a silver bullet. It does not catch all errors, so you must lower your expectations. It does catch many of them, which has saved me and many others a lot of time, routinely, for years.

Some influent and vocal experts seem to have decided that, short of catching "all errors", catching "no error" is better than "many errors". I've read all their essays and I still couldn't make sense of their logic https://mywiki.wooledge.org/BashFAQ/105

"Works for us".

PS: besides 105 and a couple others, https://mywiki.wooledge.org/BashFAQ is the best.

Python as a shell replacement

BenHutchings — Fri, 13 Dec 2019 17:23:46 +0000

I also use "set -e" by habit, but it doesn't do exactly what you probably want. When you check the result of a command, that completely suppresses its effect inside the command. For example:

set -e
f() {
    false
    echo "continued"
}
f || echo "failed"

prints:

continued

Python as a shell replacement

marcH — Sat, 07 Dec 2019 23:22:40 +0000

Right, error handling is not easy in shell either. There are a number of things you can do but nothing is bullet proof, agreed. As a start, every single script of mine starts with set -e. I also use a lot of: "do_the_work_func || die ...", "while do_work; do..." etc. (pro tip: you generally don't need ret=$?)

BTW C has similar error handling behaviors, most likely not a coincidence.

I repeat: any shell program longer than a few thousands lines or with some serious data structures is probably a mistake. This being out of the way, let me try to rephrase and clarify what I meant earlier:

1. Python-as-a-shell adds extra code and significant overhead; you can't start prototyping by just throwing your .bash_history into a file anymore.
2. Python has a built-in and pretty good exception system that you generally don't even have to think about.

So why did the migration overhead I paid in 1. didn't magically give me 2. for free? Why do I have to think so much about error handling when I use the subprocess module? In _short_ shell scripts good error handling is the only thing I was missing! So where did my migration money go?

The Python people are very smart, so I guess there must be good technical reasons for that, yet these excuses still don't make Python a desirable replacement for short shell scripts (unless you absolutely need to support Windows). Actually, I'm worried these justifications may not be Python specific and may preclude _any_ general purpose language as a shell replacement...

Insightful interview with Steve Bourne: https://www.arnnet.com.au/article/279011/a-z_programming_...

Creating Kubernetes distributions

Jandar — Sat, 07 Dec 2019 21:50:37 +0000

> It seems to require at least as much boilerplate as Python or Perl if you want to write | pipelines or do control flow while at the same time checking the exit status of each subprocess and perhaps checking whether anything was written on standard error too.

If you use #!/bin/bash "set -o pipefail" gives you checking of exit status of every part of a pipe.

Creating Kubernetes distributions

epa — Sat, 07 Dec 2019 18:48:49 +0000

Python's subprocess module has come a long way but still requires boilerplate and relatively complex error handling code

Can you give an example of how to do error checking and handling correctly in a shell script? It seems to require at least as much boilerplate as Python or Perl if you want to write | pipelines or do control flow while at the same time checking the exit status of each subprocess and perhaps checking whether anything was written on standard error too.

Creating Kubernetes distributions

IanKelling — Fri, 06 Dec 2019 23:27:47 +0000

I've been on the lookout for alternatives for years now and I don't see any. With shellcheck and proper error handling, bash can go pretty far.

Creating Kubernetes distributions

marcH — Thu, 05 Dec 2019 19:14:48 +0000

> Among the Kubernetes release scripts that date back to 2016 is anago, which is an 1,800-line bash script for releasing Kubernetes. Anago imports three separate libraries, each with another 500 lines of shell code. "It's time to not do that anymore," Augustus said.

Was any alternative suggested? Didn't find any in the Powerpoint.

A few thousands lines of shell script is too high but not crazy high IMHO.

Unix shell scripting shows its age but I haven't seen anything coming close for smaller programs (say a few hundred lines) interacting with files and gluing other programs together. Nothing as concise, dynamic and high-level. For instance using functions as parameters is trivial - not too bad!

One major drawback is incompatibility with Windows but hey, who implements release management and QA on that? ;-) Most Windows people I run into look like they haven't even heard of PowerShell yet. Click, click, click... or straight to WSL.

Python's subprocess module has come a long way but still requires boilerplate and relatively complex error handling code, people seem to get that wrong every time (error handling code being of course never tested).
Maybe Perl would have been a good candidate if it hadn't committed suicide by optimizing itself for "write-only" usage?

Is there anything else?

Creating Kubernetes distributions

SEJeff — Thu, 05 Dec 2019 17:54:16 +0000

It is also 100% open source.

Creating Kubernetes distributions

sml — Thu, 05 Dec 2019 15:13:57 +0000

Sure, but it's IBM's _commercial_ distro.

Creating Kubernetes distributions

rwmj — Thu, 05 Dec 2019 15:05:15 +0000

Isn't OpenShift supposed to be an "opinionated Kubernetes" distribution?