Get Started with DataOps.live

If you are a business analyst responsible for gathering and analyzing business requirements and translating them into data product specifications, DataOps.live is designed to meet your needs.

DataOps.live helps you build data products with simplicity and speed. You can interact with Gen AI on top of dbt Core to boost productivity and create data products instantly. No setup is needed. Only a few clicks are required to create the data product project and feature branch.

Define phase: creating a data product

The steps below let you build a simple data product in minutes using DataOps.live Create.

See Create Data Products with DataOps.live Create for detailed procedures and descriptions.

1. Make sure you have completed the prerequisites.

2. Go to DataOps.live Create and sign in using your email address and password.

   If you connect for the first time, the app asks you for authentication against the data product platform. Use your DataOps user.

   

3. Select Analytical Data product and follow the steps in the stepper to create your data product.

   

4. Data product definition step

   Let's say you want to monitor and analyze customer orders, ensuring they are processed promptly. The solution should offer comprehensive insights into customer order patterns and delivery timelines to enhance customer satisfaction and drive overall performance.

   Ensure your description of the desired outcome is comprehensive since Gen AI uses it to create suggestions for building the chosen dataset.

   Set an owner, a version, and the Service Level Objectives (SLO). For SLO, you instruct Gen AI to generate rules for verifying unique customer identification. Your SLO description must be precise and include clear details to guarantee the generation of meaningful tests.

   Keep the Create new project option toggled on and select the DataOps group where to create the data product project. If you have just a single group, that will be the default.

5. Continue to the Dataset source step

   Select to get the source metadata from an existing Snowflake account. Login to Snowflake using the account details.

6. Continue to the Dataset management step. Select to manage the dataset source within DataOps.live platform.

7. Continue to the Dataset schema step. Select the database and customer tables to use as a source for the data product. You typically need three sources for sales data: CUSTOMERS, Orders, and LINEITEM.

8. (Optional) Review the tests generated on the columns and then the data product summary.

   Default tests are automatically generated based on the constraints defined on the database columns to ensure data integrity and validity.

   The data product summary marks the beginning of the data product contract.

   

9. Click Finish, then Open designer from the confirmation message to launch the development environment where you can refine the design of the desired dataset.

   

   The data product and a feature branch are created. You can now start refining the data product dataset with the help of a powerful AI copilot, Assist Chat.

Design phase: data transformation and operation

The Assist Chat offers instant answers to your questions as you iterate on the data product, helping you achieve optimal outcomes for the final data product.

Creating the initial data product

1. After clicking Open designer in the define phase, allow the DataOps.live extension when prompted, and accept the opening of the website.

   The data product opens in the development environment, showing whether the pipeline and tests have passed.

   

2. Click Continue to generate the SQL code necessary to build the models from CUSTOMERS, Orders, and LINEITEM.

3. On the top right, click the Execute dbt SQL icon to run the script and create the models.

   

4. Click Continue.

5. Click Start Assist Chat.

   The assistant automatically generates a new model/dataset based on the context provided in the previous steps.

6. Click Create under the assistant box to generate the SQL code necessary to build the model.

   

7. On the top right, click the Execute dbt SQL icon to run the script and create the data product, customer_order_analysis_model, in this example.

Creating the final data product

Let's assume you need to refine the created data product to help calculate the average delivery delay for each customer between the order date and the latest ship date.

1. In the prompt chat box, ask to refine the data model to report on order delivery performance, including on-time delivery rate and average delivery delay, and click Submit.

2. Click Create under the assistant box to generate the new SQL.

   

   Running the dataset, you now validate the output data shown in the table. If you get an error running SQL, copy this error to the assistant and ask it to fix your model.

   Back to the chat window, ask the assistant for suggestions on how to test your dataset best. It outlines the types of tests tailored to your data, creates a YAML file with applicable tests, and recommends specific columns to test.

3. Click Create to automatically set up the tests for your dataset.

4. Click Continue to confirm the new dataset and build your data product.

   

5. Review the data product definition and SLO and click Build.

   

6. Click Publish to open your project's New merge request page on the data product platform.

   

7. Fill in the merge request information and click Create merge request.

   The AI-powered copilot automatically summarizes and describes the merge request, helping data product owners quickly understand and approve it.

With every update you make to the data product in the development environment, you publish the data product in the data product registry after each data pipeline run.

For detailed procedures and more information, see the Iterating on the Data Product with Gen AI documentation.