Kubernetes operator for YTsaurus.

  • By YTsaurus
  • Last update: Aug 11, 2023
  • Comments: 8

yt-k8s-operator

YTsaurus is a distributed storage and processing platform for big data with support for MapReduce model, a distributed file system and a NoSQL key-value database.

This operator helps you to deploy YTsaurus using Kubernetes.

Description

Currently available in alpha-version and is capable to deploy a new YTsaurus cluster from scratch, primarily for testing purposes. Also can perform automated cluster upgrades with downtime.

Getting Started

You’ll need a Kubernetes cluster to run against. You can use KIND to get a local cluster for testing, or run against a remote cluster. Note: Your controller will automatically use the current context in your kubeconfig file (i.e. whatever cluster kubectl cluster-info shows).

You can install pre-built versions of operator via helm chart.

Running on the cluster

  1. Install Instances of Custom Resources:
kubectl apply -f config/samples/
  1. Build and push your image to the location specified by IMG:
make docker-build docker-push IMG=<some-registry>/yt-k8s-operator:tag
  1. Deploy the controller to the cluster with the image specified by IMG:
make deploy IMG=<some-registry>/yt-k8s-operator:tag

Uninstall CRDs

To delete the CRDs from the cluster:

make uninstall

Undeploy controller

UnDeploy the controller to the cluster:

make undeploy

Contributing

We are glad to welcome new contributors!

  1. Please read the contributor's guide.
  2. We can accept your work to YTsaurus after you have signed contributor's license agreement (aka CLA).
  3. Please don't forget to add a note to your pull request, that you agree to the terms of the CLA.

How it works

This project aims to follow the Kubernetes Operator pattern

It uses Controllers which provides a reconcile function responsible for synchronizing resources until the desired state is reached on the cluster

Test It Out

  1. Install the CRDs into the cluster:
make install
  1. Run your controller (this will run in the foreground, so switch to a new terminal if you want to leave it running):
make run

NOTE: You can also run this in one step by running: make install run

Modifying the API definitions

If you are editing the API definitions, generate the manifests such as CRs or CRDs using:

make manifests

NOTE: Run make --help for more information on all potential make targets

More information can be found via the Kubebuilder Documentation

License

Copyright 2023.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Download

yt-k8s-operator.zip

Comments(8)

  • 1

    fix: nil map assignment

    2023/06/13 14:26:38 http: panic serving 10.244.0.149:37554: assignment to entry in nil map
goroutine 4279 [running]:
net/http.(*conn).serve.func1()
        /usr/local/go/src/net/http/server.go:1825 +0xbf
panic({0x1674d00, 0x1abfb40})
        /usr/local/go/src/runtime/panic.go:844 +0x258
github.com/ytsaurus/yt-k8s-operator/api/v1.(*Ytsaurus).validateProxies(_, {{0xc0008a1120, 0x20}, {0xc000528d20, 0x12}, {0x0, 0x0, 0x0}, 0xc0005d8550, 0x0, ...})
        /workspace/api/v1/ytsaurus_webhook.go:74 +0x9f5
github.com/ytsaurus/yt-k8s-operator/api/v1.(*Ytsaurus).validateYtsaurus(0xc000601000)
        /workspace/api/v1/ytsaurus_webhook.go:96 +0xa5
github.com/ytsaurus/yt-k8s-operator/api/v1.(*Ytsaurus).ValidateCreate(0xc000601000)
        /workspace/api/v1/ytsaurus_webhook.go:111 +0xbe
sigs.k8s.io/controller-runtime/pkg/webhook/admission.(*validatingHandler).Handle(_, {_, _}, {{{0xc0006cef90, 0x24}, {{0xc000528c18, 0x15}, {0xc0006b38a0, 0x2}, {0xc0006b38a8, ...}}, ...}})
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/webhook/admission/validator.go:71 +0x239
sigs.k8s.io/controller-runtime/pkg/webhook/admission.(*Webhook).Handle(_, {_, _}, {{{0xc0006cef90, 0x24}, {{0xc000528c18, 0x15}, {0xc0006b38a0, 0x2}, {0xc0006b38a8, ...}}, ...}})
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/webhook/admission/webhook.go:146 +0xa2
sigs.k8s.io/controller-runtime/pkg/webhook/admission.(*Webhook).ServeHTTP(0xc00003acc0, {0x7ff14385c360?, 0xc000a1b4f0}, 0xc000530700)
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/webhook/admission/http.go:98 +0xe90
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1({0x7ff14385c360, 0xc000a1b4f0}, 0x1ad7700?)
        /go/pkg/mod/github.com/prometheus/[email protected]/prometheus/promhttp/instrument_server.go:40 +0xd4
net/http.HandlerFunc.ServeHTTP(0x1ad7760?, {0x7ff14385c360?, 0xc000a1b4f0?}, 0xc0000a5a38?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1({0x1ad7760?, 0xc00016c1c0?}, 0xc000530700)
        /go/pkg/mod/github.com/prometheus/[email protected]/prometheus/promhttp/instrument_server.go:117 +0xaa
net/http.HandlerFunc.ServeHTTP(0x26e9da0?, {0x1ad7760?, 0xc00016c1c0?}, 0xc0000a59c0?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerDuration.func2({0x1ad7760, 0xc00016c1c0}, 0xc000530700)
        /go/pkg/mod/github.com/prometheus/[email protected]/prometheus/promhttp/instrument_server.go:84 +0xbf
net/http.HandlerFunc.ServeHTTP(0x207c416c361?, {0x1ad7760?, 0xc00016c1c0?}, 0xc000616d00?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
net/http.(*ServeMux).ServeHTTP(0xc000149161?, {0x1ad7760, 0xc00016c1c0}, 0xc000530700)
        /usr/local/go/src/net/http/server.go:2462 +0x149
net/http.serverHandler.ServeHTTP({0x1acac18?}, {0x1ad7760, 0xc00016c1c0}, 0xc000530700)
        /usr/local/go/src/net/http/server.go:2916 +0x43b
net/http.(*conn).serve(0xc0003aa5a0, {0x1ad85a8, 0xc000a085a0})
        /usr/local/go/src/net/http/server.go:1966 +0x5d7
created by net/http.(*Server).Serve
        /usr/local/go/src/net/http/server.go:3071 +0x4db

    I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

  • 2

    Remove probably wrong build stage

    Without this patch, I'm getting this build error

    [+] Building 1.1s (13/20)                                                                                                                                                                      
 => [internal] load build definition from Dockerfile                                                                                                                                      0.0s
 => => transferring dockerfile: 904B                                                                                                                                                      0.0s
 => [internal] load .dockerignore                                                                                                                                                         0.0s
 => => transferring context: 172B                                                                                                                                                         0.0s
 => [internal] load metadata for gcr.io/distroless/static:nonroot                                                                                                                         0.6s
 => [internal] load metadata for docker.io/library/golang:1.18                                                                                                                            1.1s
 => [auth] library/golang:pull token for registry-1.docker.io                                                                                                                             0.0s
 => [internal] load build context                                                                                                                                                         0.0s
 => => transferring context: 18.52kB                                                                                                                                                      0.0s
 => [builder  1/12] FROM docker.io/library/golang:1.18@sha256:50c889275d26f816b5314fc99f55425fa76b18fcaf16af255f5d57f09e1f48da                                                            0.0s
 => [stage-1 1/3] FROM gcr.io/distroless/static:nonroot@sha256:149531e38c7e4554d4a6725d7d70593ef9f9881358809463800669ac89f3b0ec                                                           0.0s
 => CACHED [builder  2/12] WORKDIR /workspace                                                                                                                                             0.0s
 => CACHED [builder  3/12] COPY go.mod go.mod                                                                                                                                             0.0s
 => CACHED [builder  4/12] COPY go.sum go.sum                                                                                                                                             0.0s
 => CACHED [builder  5/12] RUN mkdir -p /home/psushin/work/                                                                                                                               0.0s
 => ERROR [builder  6/12] COPY ./ytsaurus.tech ./ytsaurus.tech                                                                                                                            0.0s
------
 > [builder  6/12] COPY ./ytsaurus.tech ./ytsaurus.tech:
------
Dockerfile:14
--------------------
  12 |     
  13 |     RUN mkdir -p /home/psushin/work/
  14 | >>> COPY ./ytsaurus.tech ./ytsaurus.tech
  15 |     
  16 |     RUN go mod download
--------------------
ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref moby::dcmfpqxr6eawpdfjucnkygkha: "/ytsaurus.tech": not found
make: *** [Makefile:73: docker-build] Error 1

    Error message makes sense since there is no ytsaurus.tech in the repository

    P.S. I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=ru

  • 3

    Added enableAntiAffinity in instance spec

    Add enableAntiAffinity flag for instanceSpec, so it's possible to enable anti affinity for any node (not only for master nodes). Also, might be useful to add whole corev1.Affinity in InstanceSpec.

  • 4

    Link to the Helm chart is broken

    Hi, Just found that the link to the operator Helm chart: https://github.com/ytsaurus/yt-k8s-operator/blob/ed7a11096472dc7c92c5430bad9114e5175fd394/README.md?plain=1#L15 forwards to the Dockerhub and require authorization, so it is probably not a helm chart nor something which is available at all.

  • 5

    Feature: Extra Env Variables and Cluster Metadata Options

    Implemented two major enhancements:

    1. Additional Environment Variables Option: Enables setting of extra runtime environment variables.
    2. Cluster Metadata (Description, Environment, Group, Theme): Allows detailed cluster identification and management.
  • 6

    Add Support for Different Media Types to ytsaurus Kubernetes Operator

    This pull request introduces the ability to handle different media types in the ytsaurus Kubernetes operator.

    I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

  • 7

    [Feature] Add ability to update Ytsaurus spec

    I would like to propose an enhancement to the Ytsaurus CRD by adding advanced configuration options. These options will enable more granular control over Ytsaurus specifications and cater to varying use cases. The features I am proposing to add are:

    • Update Core Image: Introduce an ability to update the coreImage field under the spec to change the core image of Ytsaurus.
    • Add Instance specs: Allow users to add new instance specifications dynamically.
    • Edit Instance specs: Allow users to edit existing instance specifications. This includes:
      • Instance Count
      • Resources
      • Affinity
      • Volumes, VolumeMounts, Locations
      • Custom labels?
    • Remove Instance specs: Implement an option to remove instance specifications. Special care should be taken while removing data node and masters specifications to prevent data loss or inconsistency
  • 8

    Implement Functionality for Updating Cluster Specifications

    Currently, Ytsaurus clusters lack the option to update specifications such as volumes, locations, resources, etc. Implementing this feature would provide much-needed flexibility.

Popular Tag

Related