DataOps Architecture

Conceptual architecture

DataOps is logically and physically separated into three distinct planes of operation, as illustrated in the diagram below:

Management plane

The DataOps SaaS platform, hosted on our DataOps.live infrastructure, is the system's Management Plane, providing all testing, observability, release, and deployment through the DataOps pipelines.

Control plane

Pipelines physically run on runners, remote agents hosted on customer infrastructure. Therefore, this execution environment is isolated from the public cloud and provides a secure runtime for the orchestration and preparation of data.

Data plane

Separated again from the DataOps platform's management and control planes, data source systems and target data platforms (particularly Snowflake), are located in the data plane. This can include any external data transformation or governance platforms, such as ELT/ETL, data catalogs, data sharing, etc., all of which can potentially be orchestrated via DataOps.