2024-08-29-k8s-cluster-step-ca-intermediate-with-existing-root

posts | 2024-04-05-ikiwiki-graphviz-test →

....

This is just a wip command by command play by play for now, i will add context and infos later(TM).

random comments:

root$ denotes stuff done as root
$ denotes stuff done as normal user
you totally can use a different easy-rsa instance to generate your sub-ca, it does not need to be done in the root-ca "pki" - i will try to mark the points where you need to divert at a later time, essentially you need two easy-rsa instances(go figure, one with a ca build and one without - and then move the CSR/Finished cert between them)

Relevant Domains:

kubecluster.example.org - subdomain your cluster should have
step-certificates.step-ca.svc.cluster.local - service and namespace under which your intermediate is reachable in the cluster
kubemaster.example.org - node fqdn
kubenode01.example.org - node fqdn
kubenode02.example.org - node fqdn
(add/remove more node fqdns)

Derived domains: - ca.kubecluster.example.org - fqdn for your ca - .kubecluster.example.org - subdomain for your cluster including "children"

You need a root ca managed by easy-rsa. Our first step is to create a template for our intermediate. This will need be run on the CA side.

$ cd $CA_PATH
$ cat > x509-types/ca.kubecluster.example.org <<EOF


# CA_PATH_LEN for CA path length limits. You could also do this here
# manually as in the following example in place of the existing line:
#
# basicConstraints = CA:TRUE, pathlen:0

subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer:always
keyUsage = cRLSign, keyCertSign

# this magic bit creates a subca which is not allowed to sign other CAs - nice fun :)
basicConstraints = critical, CA:TRUE, pathlen:0
keyUsage = critical, keyCertSign, cRLSign
subjectKeyIdentifier = hash
nameConstraints = critical,@nameconsts
subjectAltName = critical,@sans

[sans]
# configure all the namespaces & aliases here - atleast what you used for the commonname of the subca(to be created) and the CA alias in the cluster
DNS.0=ca.kubecluster.example.org
# helm-release-name.namespace.svc.cluster-local
DNS.1=step-certificates.step-ca.svc.cluster.local

[nameconsts]
# apparently ordered numbers are not needed :P
permitted;DNS.0=.kubecluster.example.org
permitted;DNS.2=kubecluster.example.org
# repeat the DNS.1 SAN here in the nameconstraint
permitted;DNS.3=step-certificates.step-ca.svc.cluster.local
# allow it to build certs for all nodenames in the cluster
permitted;DNS.4=kubemaster.example.org
permitted;DNS.5=kubenode01.example.org
permitted;DNS.6=kubenode02.example.org
EOF

The next commands need to run on the node preparing the step-ca config:

$ make-cadir cluster-step-ca
$ cd cluster-step-ca
$ ./easyrsa init-pki
$ pwgen -s 50 1 > ./intermediate-pw
$ EASYRSA_CN=ca.kubecluster.example.org  EASYRSA_ALGO=ec EASYRSA_PASSOUT=file:./intermediate-pw ./easyrsa gen-req ca.kubecluster.example.org batch
* Notice:
Using Easy-RSA configuration from: /home/k3s-rootless-experiments/dev/tmp/talos1-home-easy-rsa/vars

* WARNING:

  Move your vars file to your PKI folder, where it is safe!

* Notice:
Using SSL: openssl OpenSSL 3.0.13 30 Jan 2024 (Library: OpenSSL 3.0.13 30 Jan 2024)

-----
* Notice:

Keypair and certificate request completed. Your files are:
req: /home/k3s-rootless-experiments/dev/tmp/talos1-home-easy-rsa/pki/reqs/ca.kubecluster.example.org.req
key: /home/k3s-rootless-experiments/dev/tmp/talos1-home-easy-rsa/pki/private/ca.kubecluster.example.org.key

We now have a request we copy over to the CA, with which we can then generate the signed intermediate cert.

$ cd $CA_DIR
$ # copy the generated req to ca.kubecluster.example.org.req to this dir
$ ./easyrsa import-req ./ca.kubecluster.example.org.csr.pem ca.kubecluster.example.org
$ ./easyrsa sign-req ca.kubecluster.example.org ca.kubecluster.example.org
* No Easy-RSA 'vars' configuration file exists!

* Using SSL: openssl OpenSSL 3.0.13 30 Jan 2024 (Library: OpenSSL 3.0.13 30 Jan 2024)

You are about to sign the following certificate.
Please check over the details shown below for accuracy. Note that this request
has not been cryptographically verified. Please be sure it came from a trusted
source or that you have verified the request checksum with the sender.

Request subject, to be signed as a ca.kubecluster.home.stejau.vpn certificate for 825 days:

subject=
    commonName                = ca.kubecluster.example.org


Type the word 'yes' to continue, or any other input to abort.
  Confirm request details: yes

Using configuration from /root/vpnint-easy-rsa/easyrsa3/pki/98b8aa22/temp.5.x
Enter pass phrase for /root/vpnint-easy-rsa/easyrsa3/pki/private/ca.key:
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'ca.kubecluster.example.org'
Certificate is to be certified until Dec  4 22:53:17 2026 GMT (825 days)

Write out database with 1 new entries
Database updated

Notice
------
Certificate created at:
* /root/vpnint-easy-rsa/easyrsa3/pki/issued/ca.kubecluster.example.org.crt


$ openssl x509 -in /home/kind01/talos1-step-ca-autocert/pki/issued/ca.kubecluster.example.org.crt -noout -text
...

The generated cert should contain all the SANs/name constraints from the template.

We continue on the step-ca host:


$ # Copy the signed cert into pki/issued/ca.kubecluster.example.org.crt.
$ # Copy the root ca crt from pk/ca.crt into pki/ca.crt

$ bash finaggle-step-configs.sh ca.kubecluster.example.org step-certificates.step-ca.svc.cluster.local step-ca-issuer

Follow the helm repo part from https://github.com/smallstep/step-issuer and install the repo and the step-issuer via helm.

The files: - test2.yaml will contain your helm values to be fed to "helm upgrade --install --create-namespace --namespace step-ca --values test2.yaml step-certificates smallstep/step-certificates" - test-cluster.issuer-filled.yaml will contain the manifest for the cluster issuer, it can simply be applied.

If this works the cmd "kubectl get stepclusterissuers.certmanager.step.sm step-cluster-issuer -o yaml" should output "StepClusterIssuer verified and ready to sign certificates" or "Ready"/"true".

With this you can annotate your resources properly and will get certificates.

finaggle-step-configs.sh:

#!/bin/bash

EXTERNAL_DNS_NAME=$1
CLUSTER_DNS_NAME=$2
ISSUER_NAME=$3


# atleast the provisioner pw causes issues when it has a newline, this avoids that issue
echo -n "$(pwgen -s 50 1)" > provisioner-pw
echo -n "$(pwgen -s 50 1)" > useless-pw

rm test1.yaml
# we use the useless-pw here as if we use the actual files, step ca init will overwrite them
step ca init --password-file=useless-pw --provisioner-password-file=provisioner-pw --name=$EXTERNAL_DNS_NAME --deployment-type=standalone --dns $EXTERNAL_DNS_NAME --dns $CLUSTER_DNS_NAME --address=:9000 --provisioner=$ISSUER_NAME --helm > test1.yaml

rm useless-pw

# overwrite various fields with our own values, according to our needs
yq < test1.yaml | jq --arg fingerprint "$(openssl x509 -in pki/ca.crt -noout -fingerprint -sha256 | sed 's/.*=//; s/://g')" --rawfile root_ca_crt pki/ca.crt --rawfile intermediate_ca_pw ./intermediate-pw --rawfile intermediate_ca_key pki/private/${EXTERNAL_DNS_NAME}.key --rawfile intermediate_ca_crt pki/issued/${EXTERNAL_DNS_NAME}.crt  --rawfile provisioner_pw provisioner-pw '.inject.certificates.root_ca = $root_ca_crt | .inject.secrets.x509.root_ca_key = "" | .inject.certificates.intermediate_ca = $intermediate_ca_crt | .inject.secrets.x509.intermediate_ca_key = $intermediate_ca_key | .inject.secrets.ca_password = ($intermediate_ca_pw | @base64) | .inject.secrets.provisioner_password = ($provisioner_pw | @base64) | .inject.config.files["defaults.json"].fingerprint = $fingerprint' > test2.json

yq < test-cluster.issuer.yaml | jq --slurpfile ca_helm test2.json '.spec.provisioner.kid = $ca_helm[0].inject.config.files["ca.json"].authority.provisioners[0].key.kid | .spec.provisioner.name = $ca_helm[0].inject.config.files["ca.json"].authority.provisioners[0].name | .spec.caBundle = ($ca_helm[0].inject.certificates.root_ca | @base64) | .spec.url = "https://\( $ca_helm[0].inject.config.files["ca.json"].dnsNames[1])/"' | yq -y | tee test-cluster.issuer-filled.yaml

test1.yaml:

# Helm template
inject:
  enabled: true
  # Config contains the configuration files ca.json and defaults.json
  config:
    files:
      ca.json:
        root: /home/step/certs/root_ca.crt
        federateRoots: []
        crt: /home/step/certs/intermediate_ca.crt
        key: /home/step/secrets/intermediate_ca_key
        address: :9000
        dnsNames:
          - ca.s02-k3s-vault.XXXX
          - step-certificates.step-ca.svc.cluster.local
        logger:
          format: json
        db:
          type: badgerv2
          dataSource: /home/step/db
        authority:
          enableAdmin: false
          provisioners:
            - {"type":"JWK","name":"step-ca-issuer","key":{"use":"sig","kty":"EC","kid":"xx","crv":"P-256","alg":"ES256","x":"xxx","y":"sxx"},"encryptedKey":"xxx","options":{"x509":{},"ssh":{}}}
        tls:
          cipherSuites:
            - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
            - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
          minVersion: 1.2
          maxVersion: 1.3
          renegotiation: false

      defaults.json:
        ca-url: https://ca.s02-k3s-vault.xxx
        ca-config: /home/step/config/ca.json
        fingerprint: xx
        root: /home/step/certs/root_ca.crt

  # Certificates contains the root and intermediate certificate and
  # optionally the SSH host and user public keys
  certificates:
    # intermediate_ca contains the text of the intermediate CA Certificate
    intermediate_ca: |
      -----BEGIN CERTIFICATE-----
      -----END CERTIFICATE-----


    # root_ca contains the text of the root CA Certificate
    root_ca: |
      -----BEGIN CERTIFICATE-----
      -----END CERTIFICATE-----


  # Secrets contains the root and intermediate keys and optionally the SSH
  # private keys
  secrets:
    # ca_password contains the password used to encrypt x509.intermediate_ca_key, ssh.host_ca_key and ssh.user_ca_key
    # This value must be base64 encoded.
    ca_password:
    provisioner_password:

    x509:
      # intermediate_ca_key contains the contents of your encrypted intermediate CA key
      intermediate_ca_key: |
        -----BEGIN EC PRIVATE KEY-----
        -----END EC PRIVATE KEY-----


      # root_ca_key contains the contents of your encrypted root CA key
      # Note that this value can be omitted without impacting the functionality of step-certificates
      # If supplied, this should be encrypted using a unique password that is not used for encrypting
      # the intermediate_ca_key, ssh.host_ca_key or ssh.user_ca_key.
      root_ca_key: |
        -----BEGIN EC PRIVATE KEY-----
        -----END EC PRIVATE KEY-----

test-cluster.issuer.yaml:

apiVersion: certmanager.step.sm/v1beta1
kind: StepClusterIssuer
metadata:
  name: step-cluster-issuer
spec:
  caBundle: XXXXX
  provisioner:
    kid: XXXX
    name: XXXX
    passwordRef:
      key: password
      name: step-certificates-provisioner-password
      namespace: step-ca
  url: https://step-certificates.step-ca.svc.cluster.local/

TODOs: * [!] put finaggle-step-configs.sh and its deps somewhere * document stuff around cert-manager / ingress-shim docs - https://cert-manager.io/docs/usage/ingress/#supported-annotations * multiple provisioners - how to do? * ??? * PROFIT

Caveat: * it might be that x509-types in a modern easy-rsa install is under pki/ instead of in the same dir as the easy-rsa script