Skip to main content

How to define network rules with SOLE for Data Products

Network rule is a way to access external network locations within Snowflake.

This article provides a detailed guide on using network rules with SOLE for Data Product hooks in Snowflake. It explains the concepts of network rules and offers a step-by-step guide on how to define, assign, and revoke network rules using SOLE for DP hooks.

Snowflake network rules​

Network rules are schema-level objects in Snowflake that store information about types of network traffic.

This traffic falls into two categories:

  • Ingress: Traffic is entering Snowflake from outside.
  • Egress: Traffic is leaving Snowflake to go outside.

Network rules help manage and control network traffic flow to and from Snowflake. A rule does not define whether its identifiers should be allowed or blocked. Instead, the Snowflake feature that uses the network rule specifies whether the identifiers in the rule are permitted or prohibited.

Step-by-step guide​

Ensure a hooks file path is specified in the dataops_config.yml file in the dataops/snowflake directory while working with network rules for SOLE for DP.

dataops_config.yml
config-version: 2
hook-paths: [hooks.yml]

Step 1 - define network rules in SOLE​

  • If the hooks.template.yml file exists:

    Verify the presence of the account_level_hooks object group section in hooks.template.yml. If it does not exist, create this section and include account_level_hooks to be able to add commands.

  • If the hooks.template.yml file does not exist:

    Create a new file named hooks.template.yml within the dataops/snowflake directory.

    Ensure the newly created file includes an account_level_hooks object group section.

    To add commands, place the pre_hooks under the account_level_hooks section. As we need to define network rule before defining external network access integration, we need to add commands in the pre_hooks section.

  • As a network rule is a schema-level object, it requires specifying the database and schema. If you want to use an existing database and schema, include their names when creating the network rule. Otherwise, define them in the pre_hook section of account_level_hooks.

  • Add pre_hooks under account_level_hooks section by creating a new file, hooks.template.yml in the dataops/snowflake directory, which has the following content:

    hooks.template.yml
    account_level_hooks:
      pre_hooks:
        - environment: snowflake
          commands:
            - "CREATE DATABASE IF NOT EXISTS {{ env.DATABASE_NAME_PREFIX }}_<database-name>_{{ env.DATAOPS_ENV_NAME }}"
            - "CREATE SCHEMA IF NOT EXISTS {{ env.DATABASE_NAME_PREFIX }}_<database-name>_{{ env.DATAOPS_ENV_NAME }}.<schema_name>"
            - "CREATE NETWORK RULE IF NOT EXISTS {{ env.DATABASE_NAME_PREFIX }}_<database-name>_{{ env.DATAOPS_ENV_NAME }}.<schema_name>.<network_rule_name>;"

    Here, {{ env.DATABASE_NAME_PREFIX }} serves as the prefix, and {{ env.DATAOPS_ENV_NAME }} acts as the suffix.

  • Create the necessary network rule for allowed and blocked IPs.

    hooks.template.yml
    account_level_hooks:
      pre_hooks:
        - environment: snowflake
          commands:
            - "CREATE DATABASE IF NOT EXISTS {{ env.DATABASE_NAME_PREFIX }}_<database-name>_{{ env.DATAOPS_ENV_NAME }}"
            - "CREATE SCHEMA IF NOT EXISTS {{ env.DATABASE_NAME_PREFIX }}_<database-name>_{{ env.DATAOPS_ENV_NAME }}.<schema_name>"
            - "CREATE NETWORK RULE IF NOT EXISTS {{ env.DATABASE_NAME_PREFIX }}_<database-name>_{{ env.DATAOPS_ENV_NAME }}.<schema_name>.NETWORK_RULE_1 MODE = EGRESS TYPE = HOST_PORT VALUE_LIST = ('translation.googleapis.com');"
Good to know

When using a Snowflake Network Rule with an External Access Integration, only MODE as EGRESS and TYPE as HOST_PORT are supported. Compare with the Snowflake CREATE EXTERNAL ACCESS INTEGRATION documentation.

Step 2 - assign a network rule to an external access integration in SOLE​

  • Add external_access_integration to the existing account.template.yml file. If it does not exist, create a new file named account.template.yml in the dataops/snowflake directory. This file should include the following content:

  • Create an account-level external_access_integration that will be used to group the external access integrations.

    account.template.yml
    - external_access_integration:
        name: "EXTERNAL_ACCESS_INTEGRATION"
        comment: "test integration"
        allowed_network_rules:
          - {{ env.DATABASE_NAME_PREFIX }}_<database-name>_{{ env.DATAOPS_ENV_NAME }}.<schema_name>.NETWORK_RULE_1
        enabled: true

Step 3 - alter network rules in SOLE​

Sometimes, you need to alter some of the properties of a network rule after you have created all other database-level objects. You can do that as follows.

  • Add post_hooks under database_level_hooks section by creating a new file, hooks.template.yml in the dataops/snowflake directory, which has the following content:

    hooks.template.yml
    database_level_hooks:
      post_hooks:
        - environment: snowflake
          commands:
            - "ALTER NETWORK RULE {{ env.DATABASE_NAME_PREFIX }}_<database-name>_{{ env.DATAOPS_ENV_NAME }}.<schema_name>.<network_rule_name> SET <network_rule_property> = <value>;"

    Here, {{ env.DATABASE_NAME_PREFIX }} serves as the prefix, and {{ env.DATAOPS_ENV_NAME }} acts as the suffix.

  • Alter the necessary network rule for allowed and blocked IPs.

    hooks.template.yml
    database_level_hooks:
      post_hooks:
        - environment: snowflake
          commands:
            - "ALTER NETWORK RULE {{ env.DATABASE_NAME_PREFIX }}_<database-name>_{{ env.DATAOPS_ENV_NAME }}.<schema_name>.NETWORK_RULE_1 SET VALUE_LIST = ('example.com', 'company.com:443');"
Please note

You won't notice any impact if you remove a network rule assigned to an external access integration. However, if you try to assign this external access integration to another object, like a procedure, you'll encounter an error stating that the network rule doesn't exist.

Conclusion​

This comprehensive guide provides the necessary steps to manage network rules in Snowflake using SOLE for DP hooks. Following the outlined procedures, you can effectively define, assign, and revoke network rules, ensuring proper control over network traffic in your Snowflake environment.

Last updated on