From 6b4e7d4bf64a447d50708c41d47e350eb91b1162 Mon Sep 17 00:00:00 2001 From: stephan48 Date: Sun, 13 Aug 2023 21:17:22 +0200 Subject: [PATCH] --- ...8-13-k3s-i-want-a-rootless-cluster-with-bgp-damn-it.mdwn | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/posts/2023-08-13-k3s-i-want-a-rootless-cluster-with-bgp-damn-it.mdwn b/posts/2023-08-13-k3s-i-want-a-rootless-cluster-with-bgp-damn-it.mdwn index 858f0b2..a8953be 100644 --- a/posts/2023-08-13-k3s-i-want-a-rootless-cluster-with-bgp-damn-it.mdwn +++ b/posts/2023-08-13-k3s-i-want-a-rootless-cluster-with-bgp-damn-it.mdwn @@ -1,8 +1,9 @@ This post should details the crazy journey on how i managed(or failed) to waste lots of my time and in the end got a rootless k3s cluster with bgp for loadbalancer IPs via metallb. -Your sanity may not survive this read. +Your sanity might not survive this read. Mine did not survive the project :P How does k3s setup the network: + * k3s server is started - calls into rootlesskit https://github.com/k3s-io/k3s/blob/38a0b91c1a917d2866aee265bc7815424af3e701/pkg/rootless/rootless.go#L37 * k3s server then forks itself(where?) to handle parent(outside netns) and child(inside netns) operations. * does not allow us to change from slirp4netns to something else like lxc-nic(easier to patch). @@ -12,6 +13,7 @@ How does k3s setup the network: * where does the tap0 device comes from? Wrote slirp4netns wrapper(crude & insecure, will need to harden): + * rootless wrapper: * will write environment infos as json for rootfull process to intercept and handle * will wait until marker file is there to wait for further startup @@ -25,11 +27,13 @@ Wrote slirp4netns wrapper(crude & insecure, will need to harden): * helper ns <-> cluster - uplink handling Learnings: + * lsns -t net is freaking awesome * you can address network namespaces by name(optional), pids of processed in them, network namespace id * unless you are root on the host you can't switch between network namespaces TODO: + * how to handle ipv6? * how to conjure all of this securely? -- 2.30.2