Skip to main content

Generating Sources for Tables Managed Outside SOLE

We have seen in Auto-Generating Sources from SOLE Ingestion Tables how to avoid manually adding new dbt sources every time you create a table by sharing table definitions between SOLE and MATE.

However, what if the tables are externally managed, outside the scope of SOLE? provides a convenient script — source-builder — that can save you time when creating MATE dbt source definitions for such tables.

Be careful!

The source-builder script is provided as-is and without warranty. Although it is not a part of the data product platform, it has been well tested, and our Support team will be happy to help if they can.


To run the source-builder script, you must:

  • Have a working Python 3.6+ environment — it has only been fully tested on Ubuntu Linux so far (Windows should work, but your mileage may vary)
  • Install dbt locally, or at least create a profiles.yml as the script uses a local dbt profiles.yml file as the source of connection and authentication details

You can download and install the sources from the Model Builder project on — check the README for instructions.

To best use this script, you may need to have your DataOps project cloned locally (using Git over HTTPS).


The source-builder script quickly retrieves some or all of your table structures from Snowflake and converts them into dbt source definition files in YAML format.

usage: source-builder [-h] [--version] [--profiles-dir PROFILES_DIR] [--profile PROFILE] [--database DATABASE]
[--schema SCHEMA] [--warehouse WAREHOUSE] [--project-dir PROJECT_DIR]
[--sources-dir SOURCES_DIR] [--force]

Build dbt source definitions from a Snowflake database.

optional arguments:
-h, --help show this help message and exit
--version, -v show program's version number and exit
--profiles-dir PROFILES_DIR
Which directory to look in for the profiles.yml file (default: /home/sam/.dbt)
--profile PROFILE, -p PROFILE
Which profile to load (default: default)
--database DATABASE, -d DATABASE
Database to interrogate (default None)
--schema SCHEMA, -s SCHEMA
Schema to interrogate (default: None)
Warehouse to use (default: None)
--project-dir PROJECT_DIR, -o PROJECT_DIR
Target directory for source files (default: .)
--sources-dir SOURCES_DIR, -i SOURCES_DIR
Sources directory within the project directory (default: sources)
--force, -f Force overwriting output files/directories