A fake kube-apiserver that serves static data from files

  • By Alvaro Aleman
  • Last update: Nov 19, 2022
  • Comments: 7

Static KAS

A fake kube-apiserver that serves static data from an Openshift must-gather. Dynamically discovers resources and supports logs. Requires golang >= 1.17. While there is no explicit documentation for the directory layout, a sample is included for testing in ./pkg/handler/testdata.

Usage:

  1. Start the static-kas in a distinct terminal: go run ./cmd/ --base-dir ../must-gather/quay-io-openshift-release-dev-ocp-v4-0-art-dev-sha256-ec058cf120ee79c97fa385205ae5b4ab7745e4064716cadd1e319652f5999ffd/
  2. Create a Kubeconfig:
cat <<EOF >/tmp/kk
apiVersion: v1
clusters:
- cluster:
    server: http://localhost:8080
  name: static-kas
contexts:
- context:
    cluster: static-kas
    user: ""
    namespace: default
  name: static-kas
current-context: static-kas
kind: Config
EOF
  1. Use kubectl or any other standard client to interact with the static kas: kubectl --kubeconfig=/tmp/kk get pod

Download

static-kas.zip

Comments(7)

  • 1

    Automatically determine default container name when not given

    Background

    Older versions of kubectl/oc, when ran as kubectl logs -n <namespace> <name>, first try to GET the pod resource requested, if they see there's only one container in it, they will then try to GET:

    http://<api>/api/v1/namespaces/<namespace>/pods/<name>/log

    Without any container=... query param (newer versions of kubectl/oc always specify this query parameter regardless). The behavior of the regular kubernetes API server in this scenario is that when the pod has only a single container, the logs for that only container are returned. If the pod has more than one pod, an error response is returned, listing the names of the containers in the pod.

    Issue

    static-kas works fine when the container name is given in the query param (which is the case for newer oc/kubectl clients), but breaks down when the container parameter is omitted.

    Solution

    static-kas will now better imitate the behavior of the regular kubernetes API server

  • 2

    Similar project

    Hey, I've been working on a similar project https://github.com/omertuc/o-must-external-reader

    I've written it in Python rather than Go.

    It's far from perfect and very buggy. My main goal was getting k9s to work somewhat ok, and i've come pretty far. You can view pods, containers, container logs (truncated due to k9s weirdness I've yet to figure out), yaml files.

    I can't seem to get k9s to work at all with your fake API server, it wouldn't even connect. I wonder why. I also couldn't do oc get pods -A .

    Looking forward to seeing how you improve this, hopefully I could use your server with k9s some day. Or if you want, we can join efforts and work on my project (or the other way around) - but I'm not sure how much overlap there is in what we've done so far and what things mine can do and yours can't and vice versa.

    Anyway just thought I should mention this

  • 3

    Too many openfiles issues is not documented

    When running static-kas on a large must-gather with the user limit of 1024 openfiles, you run into the error:

    {"error":"failed to read file namespaces/xxxx/operators.coreos.com/clusterserviceversions/jaeger-operator.v1.30.0.yaml: open namespaces/xxxxx/operators.coreos.com/clusterserviceversions/jaeger-operator.v1.30.0.yaml: too many open files"}],"stacktrace":"main.main\n\t/home/manu/Projects/git/static-kas/cmd/main.go:57\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:255"

    This happens with 15000 files in a must-gather

    it is easy to resolve the error by using the user hardlimit of openfiles, following https://www.baeldung.com/linux/error-too-many-open-files or https://access.redhat.com/solutions/2469

    however this is maybe not trivial for an end user.

    If we agree to open a Wiki for the project, maybe linked from the Readme, we could add a line along the way of:

    When the must-gather you process with static-kas has a large number of files, you may reach into an error "too many open files". In that case you will need to [increase the maximum number of open file descriptors] for the user running static-kas(https://www.baeldung.com/linux/error-too-many-open-files). A limit of 4096 has been found to process correctly large must-gathers.

  • 4

    fix: set default kind and groupVersion

    I have encountered this issue with resources coming from Cilium sysdumps. Is that an OK change for you @alvaroaleman ?

    Signed-off-by: Raphaël Pinson [email protected]

  • 5

    k9s container logs are truncated

    1. type :pods
    2. select an arbitrary pod
    3. press ↵ Return
    4. select an arbitrary container
    5. press L

    Logs are shown, but they're truncated. Pressing 0 does not help

  • 6

    k9s "edit" button doesn't work

    1. Type :pods
    2. Move the cursor to some arbitrary pod
    3. press y - works, shows you the YAML
    4. press esc to go back
    5. press e, should open in editor, but it doesn't

    I don't want editing to literally work, obviously, the gather should be immutable, but it's really nice having the yaml open in your $EDITOR of your choice

    Note that this is an API issue - it works OK with regular API servers

  • 7

    feat: add optional version.json file to return version information

    Note: the Cilium Sysdump tool dumps the Kubernetes server version as Go code, e.g.:

    version.Info{Major:"1", Minor:"21+", GitVersion:"v1.21.5-eks-bc4871b", GitCommit:"5236faf39f1b7a7dabea8df12726f25608131aa9", GitTreeState:"clean", BuildDate:"2021-10-29T23:32:16Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}

    I think it's probably better to provide a JSON file as a standard (I actually don't see an easy way to parse the go version file in go).

    Signed-off-by: Raphaël Pinson [email protected]

Popular Tag

Related