DataOps Kubernetes Runner Troubleshooting

General

You can use kubectl commands to inspect pod events and logs.

If your pipeline is not finding values provided in your vault.yml, ensure you have either correctly configured the vault: key in your values as a YAML type and check the value of the dataops-vault secret:

kubectl get secret dataops-vault -o json | jq '.data."vault.yml"' --raw-output | base64 -D

EKS with Fargate nodes

If using EKS with Fargate nodes, it is recommended to specify some resource requests and increase the maximum timeout if you are seeing pods time out.

note

When using EKS with Fargate nodes, you will have to provision any persistent volumes statically.

Resource request and timeout example:

dataops-values.yml

runners:
  config: |
    [[runners]]
      [runners.kubernetes]
        cpu_request = "1"
        memory_request = "1Gi"
  maximumTimeout: 360

Further reading

• Advanced runner configuration documentation