Skip to main content

Data Product Platform Upgrade Details May 24, 2022

Here are the details for the DataOps.live data product platform upgrade, including the following:

Pipeline configuration and management​

The features in this section include:

Show pipeline graph Job dependencies​

You can now view job dependencies in pipeline graphs when the job order reflects the needs: relationships between jobs. We have also added links between jobs to indicate the order in which jobs must run. In other words, you can see which jobs need to complete before the next job can start.

Support wildcards for YAML CI/CD configuration files​

In this release, we add support to use the * wildcard with the local includes: keyword. As a result, using the * wildcard, your includes: sections will be more dynamic, less verbose, and easier to read.

Lock the latest pipeline artifacts to prevent deletion​

DataOps now automatically locks the latest artifacts produced by a successful pipeline on any active branch, merge request, or tag to prevent them from being deleted based on expiration if they are the most recent artifacts.

Therefore, it is simpler to set a more aggressive expiration policy to achieve the following results:

  • Clean up older artifacts
  • Reduce disk space consumption
  • Ensure you always have a copy of the latest pipeline artifacts

Support variables in CI/CD pipeline workflow:rules​

We have extended support for variables in CI/CD pipeline workflow: rules. So you can set variable values for the whole pipeline if certain conditions match, improving pipeline flexibility.

Developer experience​

The features in this section include:

Mark changes as viewed in Merge Requests (MRs)​

You can now mark files as viewed in MRs to signal that the files have been reviewed.

Default branch name redirect​

As part of the larger effort to rename Git's default initial branch in a DataOps project from master to main, we have added an automatic redirect to streamline this transition.

Previously, when the default branch in a project was renamed, current URLs returned a "404 Not Found", resulting in a frustrating experience when trying to navigate between branches.

However, from this upgrade onward, you can now navigate to a file or directory path whose default branch was renamed, resulting in an automatic redirection to the updated file or directory path.

Team collaboration​

The features in this section include:

Request a follow-up review​

You can now request additional reviews from reviewers who have already reviewed your merge request. This request triggers a To-Do item and email notifications to alert users that you need a follow-up review.

Suggest a custom commit message​

A custom commit message is now suggested when you commit changes to a branch.

Notify participants when a Merge Request is marked ready​

This upgrade includes the feature where all participants in a Merge Request are notified via email when the MR is marked ready.

Predefined CI/CD variable for a commit's author​

Previously, if you needed to identify the author of a commit in a running CI/CD job, you had to add an extra API call to the job to retrieve it. This information is now readily available as the value in the CI_COMMIT_AUTHOR predefined CI/CD variable.

A typical use case for this enhancement is using it in webhooks. For instance, if the branch is not main or qa, you can route the alert to the Commit Author only.

Add pronouns to DataOps user profiles​

Pronouns have been added to the DataOps user profiles. These pronouns appear next to each user name in the Profile tab.

User experience​

The features in this section include:

Improved user experience for a project's members list​

We have improved the project's members lists' UX, including features such as:

  • More descriptive terminology
  • A new layout
  • Headers above all columns
  • Tooltips for key data elements

We have enhanced the left sidebar for improved usability, consistency, and discoverability by:

  • Swapped some links to features around
  • Split up features in the Operations menu into three distinct menus
  • Improved visual contrast
  • Optimized spacing so all the menu items can fit comfortably on a smaller screen.