How-To Guides
This section offers several valuable and useful "How-To" guides:
📄️ Build only Changed MATE Models
How-to guide for adding a runner script that builds or tests only the models that have been changed since the previous pipeline commit
📄️ Clone Schemas in Feature Branches
How-to guide for cloning in feature branches only the schemas specified in a configurable list
📄️ Configure a Production-Only Runner
How-to guide for creating a DataOps runner that only runs jobs in the production environment on the main branch
📄️ Create Incremental Models in MATE
How-to guide for incrementally loading data from a staging table to a history table in the curation layer in a DataOps pipeline with the MATE orchestrator
📄️ Create a Custom Before Script
How-to guide for creating a custom before_script that sets dynamic variables to be used by the app and script that run in the job
📄️ Create a Custom Orchestrator Image
How-to guide for creating a custom DataOps.live orchestrator packing your prebuilt business logic and extending existing orchestrators or using generic docker images
📄️ Create a Custom Reference Project
How-to guide for creating a custom reference project that helps to avoid the needless repetition of content
📄️ Detect if a File has Changed
How-to guide for triggering a specific action in a pipeline only if the state of a file has changed within a job
📄️ Git Clone when SSO is Enabled
How-to guide for performing a Git clone when SSO is enabled where you need to create a personal access token for users that do not have a password set
📄️ Host a dbt Package
How-to guide for hosting a dbt package in the DataOps platform to easily manage common macros, models, and other modeling and transformation resources
📄️ Configure the Runner Health Check Script
How-to guide for configuring the health check script to monitor your DataOps runner.
📄️ Manage Log Output
How-to guide for configuring logging in DataOps.live when running pipelines and log size recommendations
📄️ Manage Multiple Databases
Create and work with more than just the default DataOps database in SOLE and MATE.
📄️ Pass Variables from a Pipeline to MATE
How-to guide for setting up a pipeline job to pass a variable from the DataOps pipeline to a MATE model
📄️ YAML CI/CD Config File Wildcards
How-to guide for supporting wildcards to break down your YAML configuration files into multiple smaller files and improve reusability
📄️ Prevent DataOps Pipelines from Running Concurrently
How-to guide for ensuring that a new pipeline does not start before the current pipeline run is completed
📄️ Automatic Retry of Failed Jobs
How-to guide for automatically retrying a failed job and self-healing pipelines. Showing a configuration for setting the number of retry attempts
📄️ Runner Disk Space Maintenance
How-to guide for keeping the disk space usage of your DataOps Runner within a quota
📄️ Git Error - Server Certificate Verification Failed
How-to guide for solving the error server certificate verification failed. CAfile: none CRLfile: none
📄️ Serve MATE Pipeline Documentation Offline
How-to guide for retrieving, storing, and restoring MATE documentation for future offline browsing
📄️ Use a Third-Party Git Repository
How-to guide for using an external Git repository like GitHub, Bitbucket, or Azure DevOps for central code management
📄️ Use SSH with Git
How-to guide for using SSH to communicate with DataOps.live
📄️ Use Visual Studio Code with DataOps
How-to guide for using Visual Studio Code with the DataOps.live SaaS platform. Recommended extensions, development workflow tips, and setup instructions