Skip to main content

DataOps Kubernetes Runner Troubleshooting

General​

You can use kubectl commands to inspect pod events and logs.

If your pipeline is not finding values provided in your vault.yml, ensure you have either correctly configured the vault: key in your values as a YAML type and check the value of the dataops-vault secret:

kubectl get secret dataops-vault -o json | jq '.data."vault.yml"' --raw-output | base64 -D

EKS with Fargate nodes​

If using EKS with Fargate nodes, it is recommended to specify some resource requests and increase the maximum timeout if you are seeing pods time out.

note

When using EKS with Fargate nodes, you will have to provision any persistent volumes statically.

Resource request and timeout example:

dataops-values.yml
runners:
  config: |
    [[runners]]
      [runners.kubernetes]
        cpu_request = "1"
        memory_request = "1Gi"
  maximumTimeout: 360

Image registry not required​

In Kubernetes environments, when your image registry is the public registry, there's no need to explicitly specify it for runner or job pods when setting up a DataOps runner. Kubernetes automatically pulls container images from the Docker public registry.

However, if your image registry differs, you must specify it when setting up a DataOps runner.

Further reading​

Last updated on