Skip to main content

DataOps Docker runner Prerequisites

DataOps Account​

To get started, you'll need a DataOps account. If you don't have one, you can set one up by logging in to your Snowflake tenant and clicking on "Partner Connect," then select "DataOps" or contact us at support@dataops.live.

Physical Infrastructure​

The DataOps Runner must be installed on a Linux server, host, or virtual machine in a location with Snowflake access and access for all the other systems/tools you need to connect to from DataOps jobs.

The exact nature of the server/VM is up to you and will differ between bare metal, AWS, or Azure.

Minimum production specifications:

  • Ubuntu 20.04 (18.04 is possible but not recommended)
  • 4 CPU cores
  • 16GB RAM
  • Minimum 50GB Disk/Storage (300GB recommended)
  • As a guide, for most use cases an AWS t3a.xlarge (or equivalent)
  • A sudo user

Minimum PoC/Pilot specifications:

  • Ubuntu 20.04 (18.04 is possible but not recommended)
  • 2 CPU cores
  • 8GB RAM
  • Minimum 50GB Disk/Storage (300GB recommended)
  • As a guide, for most use cases an AWS t3a.large (or equivalent)
  • A sudo user

Network access​

This Server/Host requires outbound internet access, as a minimum to :

See the DataOps Architecture doc for a general overview of system architecture and the DataOps Security and Governance Appendix for a detailed discussion on networking.

Docker​

Docker should be installed following the instructions at the Docker site for your OS of choice.

caution

We recommend not installing Docker via the default Ubuntu repository, as this is often quite old.

Then (if you didn't do it as part of the Docker installation instructions):

  • Run sudo usermod -aG docker $USER (this allows you to run docker without being root)
  • Log out and log in

To test your docker install run:

docker run hello-world
Last updated on