How to Create a Custom Before Script
The before script runs ahead of each job's main
script block. The default lives in the DataOps Reference Project.
It sets various dynamic variables, such as
DATAOPS_DATABASE and variables relating to branch/environment names, which are then available to the apps and scripts running in the job's main part.
It is possible to create an additional before script, either in a single project or in a custom reference project, that adds to or overrides selected DataOps variables to enable custom behavior.
Creating the script
Create a file in your project (or reference project) that contains a shell script (Bash) to set variables and perform other logic. For variables to be available to the subsequent script, your script must
echo these out to stdout.
Don't echo anything else to stdout, as this breaks the variable export mechanism. If you need to print any text to the user, do this via stderr, e.g.:
>&2 echo "This is a note for the logs..."
Here is an example script that overrides the environment suffix for account-level database objects in production:
if [[ $DATAOPS_ENV_NAME == $DATAOPS_ENV_NAME_PROD ]]; then
>&2 echo "Running in master/PROD: setting DATAOPS_DATABASE to $DATAOPS_PREFIX"
Your additional before script always executes after the DataOps before script, so you can redefine the pre-computed values of built-in variables as above.
Enabling script for projects
In most cases, you require only a single variable setting to enable your custom before script.
In a single project
Edit your project's
variables.yml file and add the following variable definition using the path to your script defined above.
In a reference project
If you're using a custom reference project, it makes sense to add your custom before script here, such that it takes effect in all the projects that use your reference project.
Based on the fact that the reference projects clone into a specific location in the pipeline's runtime workspace, the configuration path is a little different from the single project configuration:
my-reference-project and the path following it with the correct name and path for your reference project.