Get the DataOps Runner from Snowflake Marketplace
The DataOps.live app in the Snowflake Marketplace lets you manage DataOps.live runners entirely within your Snowflake environment using Snowpark Container Services.
To use this app, you must already have an active DataOps.live subscription.
Getting started
-
Install the DataOps.live app from the Snowflake Marketplace.
-
Grant the requested account-level permissions and allow connectivity to external endpoints.
-
Activate the app and launch the UI using the buttons provided.
-
Wait for the Streamlit-based user interface to load.
Dependent objects and permissions
This app requires some account-level privileges to function, and it also needs External Access Integration permissions to communicate with DataOps.live services.
Account-level permissions required
- Bind Service Endpoint allows runner services to connect to the DataOps.live platform.
- Create Compute Pool allows this app to create a default compute pool as part of the installation.
- Create Warehouse allows this app to create a default warehouse as part of the installation.
External connectivity
To connect to the External Access Integration (EAI), click the Review button as shown in the image, then click Connect in the pop-up window.
The app creates an External Access Integration and Network Rules during installation to let DataOps runners and pipeline jobs connect to the DataOps.live platform. It pre-configures and executes everything in Snowflake, so you don’t need to take any extra steps.
Additional connectivity
You can attach additional network rules to this EAI to let pipeline jobs access other resources (e.g., other cloud databases, APIs, etc.). Create the necessary network rules, then attach them to this EAI. Please ensure that the default network rule (that allows access to app.dataops.live) is not removed when attaching new rules.
To obtain the name of the EAI that is created by the installation, click the Review button and then expand the Advanced Options section of the pop-up to reveal the External Access Integration name.
Compute resources
DataOps.live runners run as long-running Snowflake SPCS Services using a configured compute pool. During installation, the app creates a small size compute pool and sets it as the default pool for runners.
Additionally, processes running in pipeline jobs often need a Snowflake Warehouse to execute statements. The installation creates a default warehouse for all statements unless you configure otherwise.
Managing runners
User interface showing the list of runners.
Add a runner
Complete the requested information by clicking the + button in the title bar.
Basic information
- Runner name identifies the runner this app and will also form part of the SPCS service identifier.
- Agent tags register as tag names with the DataOps.live platform (use commas to separate multiple values).
- Registration token is retrieves from the DataOps.live platform to add new runners.
Snowflake configuration
This section captures the Snowflake authentication details for DataOps.live jobs to connect to Snowflake. Please refer to the DataOps.live setup documentation for more information.
Advanced configuration
- Compute pool is set to the default pool created during app installation, but you can select any other compute pool if this app has usage privileges.
- Warehouse is set to the default warehouse created during app installation, but you can select any other warehouse if this app has usage privileges.
Remove a runner
Click the Show button next to the runner, then click the Drop button at the bottom of the pop-up.
View runner logs
Click the Logs button next to the runner. Log information can be updated as needed by clicking the Refresh button.