Skip to main content

MATE Security and Governance

One of the key functions that MATE extends on top of dbt is to apply grants, tags, row access policies, and dynamic masking policies to the data models. This is achieved via simple additions to the YAML file associated with the SQL file that defines the model.

For instance, let's consider the following SQL file that builds the model CUSTOMER_LIFETIME_VALUE:

con_cust_lifetime_value.sql
{{ config(alias='CUSTOMER_LIFETIME_VALUE') }}

SELECT
  CUSTOMERID AS "Customer ID",
  Name AS "Customer Name",
  DATEDIFF(min,MAX(ORDERDATE),CURRENT_DATE()) AS "Mins since last order",
  MIN(TOTALDUE) AS "Minimum Order Size",
  SUM(TOTALDUE) AS "Total lifetime value",
  AVG(TOTALDUE) AS "Average order size",
  MAX(TOTALDUE) AS "Maximum Order Size"
FROM {{ ref('cal_fact_salesorder') }}
GROUP BY 1,2
note

This SQL statement is used as a foundation for all code examples throughout this text.

MATE project configuration​

The MATE security and governance functions rely on a post-hook configured in the dbt_project.yml file, which is located in the project repository in the DataOps modeling root directory (dataops/modelling/dbt_project.yml).

The following code snippet shows that this post-hook is defined in the models: section under the top-level model and is configured as follows:

dataops/modelling/dbt_project.yml
models:
  <TopLevelModelName>:
    +post-hook:
      - "{{ dataops.apply_snowflake_governance() }}"

The macro, dataops.apply_snowflake_governance(), is a MATE macro that, when the pipeline runs, applies the DataOps security and governance policies and rules.

warning

It is imperative to add the post-hook configuration. Not adding it results in the meta tags (see the following code snippet) used to configure security and governance being ignored.

Roles and grants​

Roles and grants play a vital role in multiple functions within DataOps. SOLE (Snowflake Object Lifecycle Engine) administers and manages grants and roles within the DataOps ecosystem.

Roles​

Roles are Snowflake objects and are unique. They are the only objects within DataOps (and Snowflake) given privileges to read, write, and administer the data within Snowflake. Data security best practice mandates that users are not given privileges directly. Therefore, before a user can do anything in DataOps and Snowflake, they must be assigned a role. And then privileges are assigned to this role.

By way of example, here is a YAML code snippet describing the three default DataOps roles:

dataops/snowflake/roles.yml
roles:

  READER:
    namespacing: prefix
     roles:
      - WRITER

  WRITER:
    namespacing: prefix
    users:
      - MAIN
      - INGESTION
      - TRANSFORMATION
    roles:
      - ADMIN

  ADMIN:
    namespacing: prefix
    users:
      - MAIN
    roles:
      - SYSADMIN

Grants​

A grant assigns a user to a particular role. It also assigns a role to another role.

Why is this necessary?

In summary, the Snowflake documentation answers this question as follows:

Granting a role to another role creates a “parent-child” relationship between the roles (also referred to as a role hierarchy)

And:

Granting a role to a user enables the user to perform all operations allowed by the role (through the access privileges granted to the role)

Grants are created in MATE in the following way:

In the CUSTOMER_LIFETIME_VALUE model, a grants key is created, under the meta key, with the different possible grant privileges and then a list of the roles to which these apply.

For example:

con_cust_lifetime_value.yml
version: 2

models:
  - name: con_cust_lifetime_value
    meta:
      grants:
        select:
          - DATAOPS_READER
          - DATAOPS_WRITER
          - DATAOPS_ADMIN
        insert:
          - DATAOPS_WRITER
          - DATAOPS_ADMIN
        update:
          - DATAOPS_WRITER
          - DATAOPS_ADMIN
        delete:
          - DATAOPS_ADMIN

You can also grant all privileges to one or more roles. For instance:

con_cust_lifetime_value.yml
version: 2

models:
  - name: con_cust_lifetime_value
    meta:
      grants:
        all privileges:
          - DATAOPS_READER
          - DATAOPS_WRITER
          - DATAOPS_ADMIN

When the MATE job is run, the output of the above will look similar to:

07:14:59 | 1 of 1 START table model customer_consumption.CUSTOMER_LIFETIME_VALUE [RUN]
       DATAOPS_DEMO5_DEV privileges usage granted to role  DATAOPS_READER [ PENDING ]
       DATAOPS_DEMO5_DEV privileges usage granted to role  DATAOPS_READER [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption privileges usage granted to role  DATAOPS_READER [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption privileges usage granted to role  DATAOPS_READER [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE privileges select granted to role  DATAOPS_READER [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE privileges select granted to role  DATAOPS_READER [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV privileges usage granted to role  DATAOPS_WRITER [ PENDING ]
       DATAOPS_DEMO5_DEV privileges usage granted to role  DATAOPS_WRITER [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption privileges usage granted to role  DATAOPS_WRITER [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption privileges usage granted to role  DATAOPS_WRITER [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE privileges select granted to role  DATAOPS_WRITER [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE privileges select granted to role  DATAOPS_WRITER [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV privileges usage granted to role  DATAOPS_ADMIN [ PENDING ]
       DATAOPS_DEMO5_DEV privileges usage granted to role  DATAOPS_ADMIN [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption privileges usage granted to role  DATAOPS_ADMIN [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption privileges usage granted to role  DATAOPS_ADMIN [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE privileges select granted to role  DATAOPS_ADMIN [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE privileges select granted to role  DATAOPS_ADMIN [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV privileges usage granted to role  DATAOPS_WRITER [ PENDING ]
       DATAOPS_DEMO5_DEV privileges usage granted to role  DATAOPS_WRITER [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption privileges usage granted to role  DATAOPS_WRITER [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption privileges usage granted to role  DATAOPS_WRITER [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE privileges insert granted to role  DATAOPS_WRITER [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE privileges insert granted to role  DATAOPS_WRITER [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV privileges usage granted to role  DATAOPS_ADMIN [ PENDING ]
       DATAOPS_DEMO5_DEV privileges usage granted to role  DATAOPS_ADMIN [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption privileges usage granted to role  DATAOPS_ADMIN [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption privileges usage granted to role  DATAOPS_ADMIN [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE privileges insert granted to role  DATAOPS_ADMIN [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE privileges insert granted to role  DATAOPS_ADMIN [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV privileges usage granted to role  DATAOPS_WRITER [ PENDING ]
       DATAOPS_DEMO5_DEV privileges usage granted to role  DATAOPS_WRITER [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption privileges usage granted to role  DATAOPS_WRITER [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption privileges usage granted to role  DATAOPS_WRITER [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE privileges update granted to role  DATAOPS_WRITER [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE privileges update granted to role  DATAOPS_WRITER [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV privileges usage granted to role  DATAOPS_ADMIN [ PENDING ]
       DATAOPS_DEMO5_DEV privileges usage granted to role  DATAOPS_ADMIN [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption privileges usage granted to role  DATAOPS_ADMIN [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption privileges usage granted to role  DATAOPS_ADMIN [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE privileges update granted to role  DATAOPS_ADMIN [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE privileges update granted to role  DATAOPS_ADMIN [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV privileges usage granted to role  DATAOPS_ADMIN [ PENDING ]
       DATAOPS_DEMO5_DEV privileges usage granted to role  DATAOPS_ADMIN [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption privileges usage granted to role  DATAOPS_ADMIN [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption privileges usage granted to role  DATAOPS_ADMIN [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE privileges delete granted to role  DATAOPS_ADMIN [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE privileges delete granted to role  DATAOPS_ADMIN [ SUCCESS 1 ]
07:15:16 | 1 of 1 OK created table model customer_consumption.CUSTOMER_LIFETIME_VALUE [SUCCESS 1 in 16.44s]

This con_cust_lifetime_value.yml configuration can also be placed inside the dbt_project.yml file to apply grants to models based on the file system hierarchy.

note

The meta tag should be +meta: to disambiguate it from table names.

For instance:

## Models
models:
  MyProject:
    MySchema:
      +meta:
        grants:
          select:
            - DATAOPS_READER
Multiple configurations

If the grant's key is applied at multiple places in the hierarchy, for example, in the project file and a model .yml file, then MATE will use the most specific, and only the grants in this particular definition will be applied. This means that if a SELECT grant is applied in the project file, and an UPDATE grant is applied in a .yml file; only the UPDATE grants will be applied.

For more information on granting privileges to a role, navigate to the Snowflake docs on the topic.

Object tags​

MATE object tags are used as part of MATE's resource selection function. These tags are implemented at the following levels:

  • Model-level tags
  • Column-level tags
tip

Tagging is based on Snowflake's object-tagging.

Model-level tags​

Model-level tags are set at the model level, assuming these are materialized as tables, incremental tables, or views. These tags are essentially Snowflake tags created using the DataOps Snowflake Object Lifecycle Engine.

For example:

con_cust_lifetime_value.yml
version: 2

models:
  - name: con_cust_lifetime_value
    meta:
      tags:
        DATAOPS_META.DATAOPS_COST_CENTRE: "Development"
        DATAOPS_META.DATAOPS_PIPELINE_ID: "{{ env_var('CI_PIPELINE_ID') }}"
        DATAOPS_META.DATAOPS_JOB_ID: "{{ env_var('CI_JOB_NAME') }}"
        DATAOPS_META.ANALYST_PII: True
        DATAOPS_META.NON_FINANCE: True
        DATAOPS_META.OPEN_DEV_ACCESS: True

These tags can then be used with various tools that read Snowflake tags. For instance, in a MATE job, the output from the above script would look similar to the following output sample:

07:14:59 | 1 of 1 START table model customer_consumption.CUSTOMER_LIFETIME_VALUE [RUN]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE tag DATAOPS_DEMO5_DEV.DATAOPS_META.DATAOPS_COST_CENTRE set to Development [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE tag DATAOPS_DEMO5_DEV.DATAOPS_META.DATAOPS_COST_CENTRE set to Development [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE tag DATAOPS_DEMO5_DEV.DATAOPS_META.DATAOPS_PIPELINE_ID set to 12345 [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE tag DATAOPS_DEMO5_DEV.DATAOPS_META.DATAOPS_PIPELINE_ID set to 12345 [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE tag DATAOPS_DEMO5_DEV.DATAOPS_META.DATAOPS_JOB_ID set to manual [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE tag DATAOPS_DEMO5_DEV.DATAOPS_META.DATAOPS_JOB_ID set to manual [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE tag DATAOPS_DEMO5_DEV.DATAOPS_META.ANALYST_PII set to True [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE tag DATAOPS_DEMO5_DEV.DATAOPS_META.ANALYST_PII set to True [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE tag DATAOPS_DEMO5_DEV.DATAOPS_META.NON_FINANCE set to True [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE tag DATAOPS_DEMO5_DEV.DATAOPS_META.NON_FINANCE set to True [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE tag DATAOPS_DEMO5_DEV.DATAOPS_META.OPEN_DEV_ACCESS set to True [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE tag DATAOPS_DEMO5_DEV.DATAOPS_META.OPEN_DEV_ACCESS set to True [ SUCCESS 1 ]
07:15:16 | 1 of 1 OK created table model customer_consumption.CUSTOMER_LIFETIME_VALUE [SUCCESS 1 in 16.44s]

Column-level tags​

In contrast to model-level tags, column-level tags are set at the column level within a model, assuming these are materialized as tables, incremental tables, or views. These tags are essentially Snowflake tags created using the DataOps Snowflake Object Lifecycle Engine as with model-level tags.

For example:

con_cust_lifetime_value.yml
version: 2

models:
  - name: con_cust_lifetime_value
    columns:
      - name: '"Customer ID"'
      - name: '"Customer Name"'
        meta:
          tags:
            DATAOPS_META.pii_name: True
            DATAOPS_META.ANALYST_PII: False
            DATAOPS_META.NON_FINANCE: 7
            DATAOPS_META.OPEN_DEV_ACCESS: "foo"

Again, as with model-level tags, these tags can be used with various tools that read Snowflake tags. For instance, in a MATE job, the output from the above script would look similar to the following output sample:

07:14:59 | 1 of 1 START table model customer_consumption.CUSTOMER_LIFETIME_VALUE [RUN]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE COLUMN "Customer Name" set masking policy DATAOPS_DEMO5_DEV."PUBLIC"."MASK_WITH_SHA2" [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE COLUMN "Customer Name" set masking policy DATAOPS_DEMO5_DEV."PUBLIC"."MASK_WITH_SHA2" [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE COLUMN "Customer Name" tag DATAOPS_DEMO5_DEV.DATAOPS_META.pii_name set to True [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE COLUMN "Customer Name" tag DATAOPS_DEMO5_DEV.DATAOPS_META.pii_name set to True [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE COLUMN "Customer Name" tag DATAOPS_DEMO5_DEV.DATAOPS_META.ANALYST_PII set to False [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE COLUMN "Customer Name" tag DATAOPS_DEMO5_DEV.DATAOPS_META.ANALYST_PII set to False [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE COLUMN "Customer Name" tag DATAOPS_DEMO5_DEV.DATAOPS_META.NON_FINANCE set to 7 [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE COLUMN "Customer Name" tag DATAOPS_DEMO5_DEV.DATAOPS_META.NON_FINANCE set to 7 [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE COLUMN "Customer Name" tag DATAOPS_DEMO5_DEV.DATAOPS_META.OPEN_DEV_ACCESS set to foo [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE COLUMN "Customer Name" tag DATAOPS_DEMO5_DEV.DATAOPS_META.OPEN_DEV_ACCESS set to foo [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE COLUMN "Total lifetime value" tag DATAOPS_DEMO5_DEV.DATAOPS_META.pii_name set to True [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE COLUMN "Total lifetime value" tag DATAOPS_DEMO5_DEV.DATAOPS_META.pii_name set to True [ SUCCESS 1 ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE COLUMN "Mins since last order" tag DATAOPS_DEMO5_DEV.DATAOPS_META.pii_name set to True [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE COLUMN "Mins since last order" tag DATAOPS_DEMO5_DEV.DATAOPS_META.pii_name set to True [ SUCCESS 1 ]
07:15:16 | 1 of 1 OK created table model customer_consumption.CUSTOMER_LIFETIME_VALUE [SUCCESS 1 in 16.44s]

Row access policies​

Row-level access policies enforce row-level security to determine which rows to return as part of a query result. For instance, let's assume that only senior HR management is allowed to view employee and prospective employee social security numbers.

The caveat here is that everyone in the HR department has access to the same data; therefore, these social security numbers are not obfuscated in the current data model. As a result, all the employee data is open to view by anyone in this department.

How do we mask the employee's social security numbers?

Snowflake row access policies are the answer.

The first step is to create the row access policy in Snowflake using the following logic(in pseudocode):

Create a row access policy called view_soc_sec_no. The logic should check if the employee level is higher than grade 4 management or between grades 3 and 1. If true, display the employee's social security number; otherwise, hide it. In this scenario, grade 1 is the organization's CEO, and the rest of the c-suite leadership fall within the grade 2 management level.

tip

DataOps best practices methods recommend that these row access policies be created using the DataOps Snowflake Object Lifecycle Engine.

Once the row access policy has been created, the next step is to apply the policy to the MATE model. In this context, it is vital to note that Snowflake row access policies are set at a model level, including materialized tables, incremental tables, or views.

For example:

con_cust_lifetime_value.yml
version: 2

models:
  - name: con_cust_lifetime_value
    meta:
      row_access_policies:
        '"ROW_ACCESS".random_chance':
          - '"Customer ID"'
be aware

The current Snowflake design allows only a single row access policy per model.

This YAML code snippet (con_cust_lifetime_value.yml) highlights how the row access policy must be specified by both schema and name, but not database since this is determined by the DataOps pipeline based on the branch/environment. Lastly, the column name to which this row access policy applies must also be specified.

In a MATE job, the output of the above YAML code snippet will look similar to:

07:14:59 | 1 of 1 START table model customer_consumption.CUSTOMER_LIFETIME_VALUE [RUN]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE add row access policy DATAOPS_DEMO5_DEV."ROW_ACCESS".random_chance on column "Customer ID" [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE add row access policy DATAOPS_DEMO5_DEV."ROW_ACCESS".random_chance on column "Customer ID" [ SUCCESS 1 ]
07:15:16 | 1 of 1 OK created table model customer_consumption.CUSTOMER_LIFETIME_VALUE [SUCCESS 1 in 1.44s]
READ MORE

https://docs.snowflake.com/en/user-guide/security-row-intro.html

Dynamic masking policies​

The Snowflake dynamic data masking doc provides the following definition of dynamic data masking:

Dynamic data masking is a column-level security feature that uses masking policies to selectively mask plain-text data in table and view columns at query time.

Snowflake dynamic masking policies can be set on the column of a model, including materialized tables, incremental tables or views.

tip

DataOps best practices methods recommend that these dynamic masking policies be created using the DataOps Snowflake Object Lifecycle Engine.

For instance:

con_cust_lifetime_value.yml
version: 2

models:
  - name: con_cust_lifetime_value

    columns:
      - name: '"Customer Name"'
        meta:
          dynamic_masking_policy: '"PUBLIC"."MASK_WITH_SHA2"'

This YAML code snippet (con_cust_lifetime_value.yml) highlights how the dynamic masking policy is specified by both schema and name, but not database since this is determined by the DataOps pipeline based on the branch/environment. Lastly, the column is not explicitly stated since it is defined as a property of the column itself.

In a MATE job, the output of the above YAML code snippet will look similar to:

07:14:59 | 1 of 1 START table model customer_consumption.CUSTOMER_LIFETIME_VALUE [RUN]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE COLUMN "Customer Name" set masking policy DATAOPS_DEMO5_DEV."PUBLIC"."MASK_WITH_SHA2" [ PENDING ]
       DATAOPS_DEMO5_DEV.customer_consumption.CUSTOMER_LIFETIME_VALUE "Customer Name" set masking policy DATAOPS_DEMO5_DEV."PUBLIC"."MASK_WITH_SHA2" [ SUCCESS 1 ]
07:15:16 | 1 of 1 OK created table model customer_consumption.CUSTOMER_LIFETIME_VALUE [SUCCESS 1 in 16.44s]
READ MORE

https://docs.snowflake.com/en/user-guide/security-column-ddm-intro.html

Removal of security and governance objects​

The MATE security and governance functions are "add only." They do not remove existing grants, tags, row access policies, or dynamic masking policies.

Since most models are created as tables or views, this works well because the tables or views are rebuilt each time a pipeline runs, and only the objects defined are added. However, suppose the model is materialized as an incremental table model. In that case, it is not recreated each time, and objects defined and applied and then removed from the definition are not removed. Don't hesitate to contact support@dataops.live for ways to manage this if required.

Last updated on