How to Set up Multiple Pull Policies on the DataOps Runner
Why do you need it?
Losing the network connection to a container registry, used to retrieve the container images required for pipeline execution, can stop production pipelines and result in stale data.
Having the ability to use retries or locally cached container images in the CI jobs can mitigate the impact caused by lost connectivity.
Pull container retry
If the container registry is unreliable, adding retries might increase your chances connecting to it.
You can add a retry by adding multiple times the pull_policy of choice.
Example pull_policy with a retry (DataOps runner configuration):
[runners.docker]
pull_policy = ["always", "always"]
Pull container fallback policy
Another way to mitigate it is to add a different policy. If you use dedicated runners for production workloads, you can use an already present image. Those images would have been pulled at a previous run of the pipeline when the registry was accessible.
Example pull_policy with a fallback (DataOps runner configuration):
[runners.docker]
pull_policy = ["always", "if-not-present"]
In this case, if the runner is not able to contact the container registry, it falls back to the if-not-present policy and use a local image if present.
There is a potential security risk with using the if-not-present policy.
If the container registry is unavailable, and someone can create an image on the runner host, the runner will run the wrong image.