Hi

I have a container image which takes care of some test executions. I would like to create it as a executor on testkube.

have gone through the below documentation but it is not having any commands which I can follow : Like command to create a executor with the help of yaml and how to run the test which points to this container ! https://kubeshop.github.io/testkube/test-types/container-executor

Could you please help me with the commands and steps that I need to follow ?

[SOLVED]

• For private github repos:
  • Use CLI-command to create tests according to documentation, using the additional flags --git-username and --git-token
  • (currently, the UI is lacking a field for username)
• For private bitbucket repos:
  • You cannot use username/password authentication, as bitbucket (intentionally) dropped support
  • Just create an "App-Password" and use it as a value for flag --git-token (and NOT the password of your bitbucket user!)

Describe the bug The documentation on how to create a testkube test from a Cypress project does not explain how do do it when it's a private repository.

To Reproduce

1. Place code in a (bitbucket) repository that requires authentication before cloning
2. Create new test from Cypress project via testkube dashboard, with the help of the documentation
3. As type use "git directory"
4. As URl use the HTTPS one, containing the username (like: https://[email protected]/...)
5. As token use an Atlassian API token (see screenshots)
6. Test is created, BUT: When executing it, it fails[1] (I guess logs in testkube are missing, since the test didn't run - this makes sense from a developer perspective, but it's not that end-user friendly)...
7. Note that, when I am using a - publicly accessible! - Cypress demo project my testkube test created from Cypress project works[2]!

[1] Logs from teskube's API-server from DataDog...

[2] Example of succesfull test run, using publicly accessible testkube testrepo

Expected behavior Cypress project is checked out, tests are run.

Version / Cluster

• testkube version: 1.4.21
• AWS EKS
• K8s version: 1.22

Screenshots This is how my test config looks like...

As "Token" I used the value of my personal Atlassian API-Token... sian API-token...

Hi I am unable copy parameter files and script files from bitbucket to Kubernetes where testkube is running.

Could you please let me know how can I achieve this through create new test API

Describe the bug A K8s environment with Istio TLS connection between pods. Without Istio injection "testkube" cannot connect to the actual test endpoint because TLS connection Connection reset by peer error happens.

To Reproduce Steps to reproduce the behavior:

1. Run 'kubectl testkube run test'
2. Inject "testkube" name space with Istio kubectl label namespace testkube istio-injection=disabled --overwrite
3. See error error: error trying to reach service: read tcp 172.17.0.1:59970->172.17.0.38:8088: read: connection reset by peer ⨯ getting test suites executions list (error: api/GET-testkube.TestSuiteExecutionsResult returned error: api server response: '{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"error trying to reach service: read tcp 172.17.0.1:59972-\u003e172.17.0.38:8088: read: connection reset by peer","reason":"ServiceUnavailable","code":503}

Expected behavior A clear and concise description of what you expected to happen.

Version / Cluster

• Which testkube version? : 1.2.48
• What Kubernetes cluster? (e.g. GKE, EKS, Openshift etc, local KinD, local Minikube) Minikube, Istio mTLS between pods.
• What Kubernetes version? v1.21.11

Screenshots

$kube get pods -n testkube
NAME                                                    READY   STATUS    RESTARTS   AGE
testkube-api-server-f86c985b8-297fs                     2/2     Running   1          20h
testkube-dashboard-6f5f84f8d8-5b9t7                     2/2     Running   0          20h
testkube-minio-testkube-64cd475b94-fc5hb                2/2     Running   0          20h
testkube-mongodb-6c9c5db4d5-wq9xh                       2/2     Running   0          20h
testkube-operator-controller-manager-66ff4cdfd4-tblg2   3/3     Running   1          20h

Additional context 172.17.0.37:8088 - API server internal IP address 172.17.0.1:59970 - Not sure which pod's IP address it is.

API server POD log

Available migrations for v1.3.0
No migrations available for v1.3.0
{"level":"warn","ts":1657104355.4716427,"caller":"api-server/main.go:105","msg":"Getting uniqe clusterId","error":null}
{"level":"info","ts":1657104355.5050254,"caller":"v1/server.go:279","msg":"Testkube API configured","namespace":"testkube","clusterId":"clusterbb669eef1b556e914a11107ae51ccfa9","telemetry":false}
segment 2022/07/06 10:45:55 ERROR: sending request - Post "https://api.segment.io/v1/batch": read tcp 172.17.0.37:40666->35.155.223.175:443: read: connection reset by peer
segment 2022/07/06 10:45:55 ERROR: 1 messages dropped because they failed to be sent and the client was closed
{"level":"info","ts":1657104355.5379503,"caller":"api-server/main.go:130","msg":"starting Testkube API server","telemetryEnabled":true,"clusterId":"clusterbb669eef1b556e914a11107ae51ccfa9","namespace":"testkube"}

 ┌───────────────────────────────────────────────────┐
 │                   Fiber v2.31.0                   │
 │               http://127.0.0.1:8088               │
 │       (bound on host 0.0.0.0 and port 8088)       │
 │                                                   │
 │ Handlers ........... 166  Processes ........... 1 │
 │ Prefork ....... Disabled  PID ................. 1 │
 └───────────────────────────────────────────────────┘

segment 2022/07/06 10:45:55 ERROR: sending request - Post "https://api.segment.io/v1/batch": read tcp 172.17.0.37:32808->52.34.77.50:443: read: connection reset by peer
segment 2022/07/06 10:45:55 ERROR: 1 messages dropped because they failed to be sent and the client was closed
segment 2022/07/06 11:45:55 ERROR: sending request - Post "https://api.segment.io/v1/batch": EOF
segment 2022/07/06 11:45:55 ERROR: 1 messages dropped because they failed to be sent and the client was closed
segment 2022/07/06 12:45:55 ERROR: sending request - Post "https://api.segment.io/v1/batch": EOF
segment 2022/07/06 12:45:55 ERROR: 1 messages dropped because they failed to be sent and the client was closed
segment 2022/07/06 13:45:55 ERROR: sending request - Post "https://api.segment.io/v1/batch": EOF
segment 2022/07/06 13:45:55 ERROR: 1 messages dropped because they failed to be sent and the client was closed
segment 2022/07/06 14:45:55 ERROR: sending request - Post "https://api.segment.io/v1/batch": EOF
segment 2022/07/06 14:45:55 ERROR: 1 messages dropped because they failed to be sent and the client was closed
segment 2022/07/06 15:45:55 ERROR: sending request - Post "https://api.segment.io/v1/batch": EOF
segment 2022/07/06 15:45:55 ERROR: 1 messages dropped because they failed to be sent and the client was closed
segment 2022/07/06 16:45:55 ERROR: sending request - Post "https://api.segment.io/v1/batch": EOF
segment 2022/07/06 16:45:55 ERROR: 1 messages dropped because they failed to be sent and the client was closed
segment 2022/07/06 17:45:55 ERROR: sending request - Post "https://api.segment.io/v1/batch": EOF
segment 2022/07/06 17:45:55 ERROR: 1 messages dropped because they failed to be sent and the client was closed
segment 2022/07/06 18:45:55 ERROR: sending request - Post "https://api.segment.io/v1/batch": EOF
segment 2022/07/06 18:45:55 ERROR: 1 messages dropped because they failed to be sent and the client was closed
segment 2022/07/06 19:45:55 ERROR: sending request - Post "https://api.segment.io/v1/batch": read tcp 172.17.0.37:44758->44.241.139.196:443: read: connection reset by peer
segment 2022/07/06 19:45:55 ERROR: 1 messages dropped because they failed to be sent and the client was closed
< Ommiting same error logs>

I created a simple bash executor as follows:

apiVersion: executor.testkube.io/v1
kind: Executor
metadata:
  name: bash-executor
  namespace: ops
spec:
  image: quay.io/openshift/origin-cli:4.7
  command: ["bash"]
  executor_type: container
  types:
  - bash-origin-cli/test

And I have simple test

apiVersion: tests.testkube.io/v3
kind: Test
metadata:
  name: simple-test
  namespace: ops
spec:
  type: bash-origin-cli/test
  executionRequest:
    args:
    - echo "hello world!"

The problem is, when I run the test it fails with the following output

{"type":"event","content":"running test [637fb1f8de2c89f221a10eee]"}
{"type":"line","content":"chmod: .: Operation not permitted\nchmod: .: Operation not permitted\n"}
{"type":"error","content":"process error: exit status 1"}

For what I've seen that's a problem with the ServiceAccount used by the Job creating the Pods to run the tests. I do have a ServiceAccount I can use, but I have changed every possible entry under the values file (I'm using Helm to deploy) but nothing seems to change. The ServiceAccount used by the Job is still default.

What would be the right entry on the values file?

Testkube automatically create pod with best-effort-kube feature to run test . There is no option to specify test size for pod which we want to use to run k6 performance test . This will be a good feature to have to run test and to compete with other performance tool which provide these option .

Solution :

In test or Open API we can pass parameter to create pod with requested resource .

Context

When I open a new PR from my feature branch my CI/CD pipeline creates a new environment for me to run tests on to see if it will break anything.In Testkube I set the branch where the tests are going to be fetched from when I'm creating the test, however, the branch will need to be specified for each execution because it will be a different one.

Question

How can I tell Testkube the branch which I want to fetch my tests from when I'm triggering an execution ?

Describe the bug The Testkube executor initcontainer "testkube-executor-init" tries to execute "chmod" and fails when uid is not "0". This means, that Testkube does not work on OpenShift with scc "restricted", which is the default scc, or Kubernetes clusters where uid "0" is not allowed. The testkube executor initcontainer would run on OpenShift when scc "anyuid" would be allowed, but this is not the case in most clusters as this is a security issue.

{"level":"info","ts":1671465831.2670317,"caller":"minio/minio.go:257","msg":"Getting the contents of buckets [test-curl-test]"} {"level":"info","ts":1671465831.267091,"caller":"minio/minio.go:55","msg":"connecting to minio","endpoint":"testkube-minio-service:9000","accessKeyID":"minio","location":"","token":"","ssl":false} {"level":"info","ts":1671465831.2713344,"caller":"minio/minio.go:268","msg":"Bucket test-curl-test does not exist"} {"type":"line","content":"chmod: .: Operation not permitted\nchmod: .: Operation not permitted\n"} {"type":"error","content":"process error: exit status 1"}

To Reproduce Steps to reproduce the behavior:

1. Run testkube on OpenShift or in a Kubernetes cluster, where uid "0" ist not allowed

Expected behavior The initcontainer works also when run with an uid other than "0". Maybe there is a way to omit "chmod".

Version / Cluster

• Which testkube version? 1.8.2
• What Kubernetes cluster? Openshift
• What Kubernetes version? v1.23

Screenshots See error message.

Additional context Add any other context about the problem here.

Describe the bug running the maven test is failing for me, it cannot connect to github.

To Reproduce testkube create test --git-uri https://github.com/kubeshop/testkube-executor-maven.git --git-path examples/hello-maven-settings --type maven/test --name maven-example-test --git-branch main

testkube run test maven-example-test --copy-files "testkube-executor-maven/examples/hello-maven-settings/se ttings.xml:/tmp/settings.xml" --args "--settings" --args "/tmp/settings.xml" -v "TESTKUBE_MAVEN=true"

expected successful test but it returns this error: ⨯ process error: exit status 128 output: Cloning into 'repo'... fatal: unable to access 'https://github.com/kubeshop/testkube-executor-maven.git/': Could not resolve host: github.com

so could you please tell me what is the issue for connecting to the github, if I try git clone https://github.com/kubeshop/testkube-executor-maven.git, is fine but it fails with testkube command. could you please help. thanks !

Describe the bug

To Reproduce

Having a test with this execution request

  executionRequest:
    envs:
      MY_ENV: bla
      SECOND: two

does not pass them to the executor container

    Environment:
      DEBUG:                   
      RUNNER_ENDPOINT:         testkube-minio-service-testkube:9000
      RUNNER_ACCESSKEYID:      ***
      RUNNER_SECRETACCESSKEY:  ***
      RUNNER_LOCATION:         
      RUNNER_TOKEN:            
      RUNNER_SSL:              false
      RUNNER_SCRAPPERENABLED:  true
      RUNNER_DATADIR:          /data
    Mounts:
      /data from data-volume (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-t945n (ro)

Describe the bug

The bug in https://github.com/kubeshop/testkube/issues/2231 is not fixed in the most recent Helm Chart (testkube-1.6.0)

To Reproduce Steps to reproduce the behavior:

1. Delete the previous install
2. Run either testkube init or helm install
3. Add a test
4. Inspect the logs for the manager container in the testkube-operator-controller-manager-...
5. Observe 4-5 log messages per second ...http: TLS handshake error from 10.42.213.64:52162: remote error: tls: bad certificate

Expected behavior The certificate should be valid. The addition of a test should not cause thousands of errors to be logged even if the cert is not valid.

Version / Cluster

• Which testkube version?

Client Version 1.5.41
Server Version v1.6.0
Commit 192e00e10fa9ae710c2d1f72ad2c328acc64faf2
Built by goreleaser
Build date 2022-09-30T08:15:11Z

• What Kubernetes cluster? (e.g. GKE, EKS, Openshift etc, local KinD, local Minikube)
  • RKE1
  • k3s
• What Kubernetes version?
  • Happens on multiple k8s bersions
    • Server Version: v1.22.11
    • Server Version: v1.23.2+k3s1

Describe the bug

Having these 6 execution times:

5.34
5.75
5.93
6.16
6.56
11.56

Testkube tells me my P95 is 6.56

Whereas the actual P95 (even P85!) is actually 11.56 Multiple percentile calculators throw out the same result

The real issue here is the message/assumption Testkube is sending to users "Everything is fine" – where in reality there is really an issue with the executions because P95 actually is DOUBLE P50

I want Testkube to run a Test or TestSuite when receiving CDEvents defined at https://cdevents.dev/docs/

For each trigger I want to configure

• subject - which CDEvent subject to listen for
• predicate(s) - which corresponding predicate(s) to listen for
• custom fields - which custom fields/values to listen for - some kind of wildcard/regex syntax should be supported
  • fields that are not specified would be ignored
• target test/testSuite - which Test or TestSuite to run
• optional parameters - parameter values to pass to the target Test or TestSuite
  • an added bonus would be if a value could be extracted from one of the incoming fields.

For example

cdevent-trigger:
  subject: service
  predicates:
    - deployed
  fields:
     - name: id
       value: service/petstore
    - name: environment
       value: "production"
  testSuite : my-test-suite

Would run the my-test-suite TestSuite if a deployed event is received for the service subject with the specified id and environment - see https://github.com/cdevents/spec/blob/v0.1.1/continuous-deployment-pipeline-events.md#service-deployed

Testkube would have to listen for these events on an HTTP endpoint that could be targeted by external systems. The events would be received in HTTP binary mode as shown at https://github.com/cdevents/spec/blob/v0.1.1/cloudevents-binding.md#examples

Describe the bug Role "testkube-operator-leader-election-role" is missing since testkube version 1.7. It was there with version 1.6.

So the container "manager" of the pod "testkube-operator-controller-manager" is throwing an error:

E0105 12:50:27.101292 1 leaderelection.go:334] error initially creating leader election record: leases.coordination.k8s.io is forbidden: User "system:serviceaccount:col-testkube-test:testkube-operator-controller-manager" cannot create resource "leases" in API group "coordination.k8s.io" in the namespace "col-testkube-test"

To Reproduce

❯ helm template kubeshop/testkube --namespace col-testkube-test   --version 1.7 | grep testkube-operator-leader-election-role
  name: testkube-operator-leader-election-role

❯ helm template kubeshop/testkube --namespace col-testkube-test   --version 1.6 | grep testkube-operator-leader-election-role
  name: testkube-operator-leader-election-role
  name: testkube-operator-leader-election-rolebinding
  name: testkube-operator-leader-election-role

Expected behavior Include it in the helm chart again.

Version / Cluster

• Which testkube version? all above 1.6

Bumps json5 from 2.2.1 to 2.2.3.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps json5 from 2.2.1 to 2.2.3.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.