Introducing: Schema selection in Google Big Query, new looks in Asset Details and more!

decube now supports schema selection in Google Big Query so that you can choose specifically which schemas to view inside your Data Catalog. We've also changed up the look of Asset Details so you can add owners to your assets.

By

Melanie

April 1, 2023

New settings for schema selection in Google Big Query

For Google Big Query connections, you can now specify which schemas are to be scanned and included to have greater control of which assets will be included into decube's Data Quality and Data Catalog.

Now, when you add a new connection to Google Big Query, you will need to test your connection first. If the connection is successful, you now have the choice to either scan and include all schemas within the connection or only scan a specific selection on schemas.

We've also added a new selection here for us to handle new schemas being added to your data source. You can either let us:

Add new schemas automatically as we pick them up in the scan (and thus will appear in your Catalog without any additional effort to add them on your end), or;
Never add them automatically. The new schema would still appear in the list when you head over to Modify Data Source and you can manually add it from there.

Example of list of schemas that can be selected. — Example of list of schemas. You can select those to be included into our scan.

First time setting up your connection? Check our our docs here for more information.

‍

Asset Details has a new look

Example of merchants table. You can now add owners onto the assets!

We've updated the Asset Details to now have improved icons for your data sources, asset types and attributes. Enrich your asset details by adding Data Owners and Business Owners onto your critical data assets. Owners are responsible to maintain the data assets and act as the contact point for the status of the data asset.

We'll be adding the ability for users to tag columns within the data asset (eg. adding PII tags), and also add documentation to each asset for a complete data cataloging experience for your organization. Check out our plan on our roadmap here.

‍

Update your user profile

Didn't like the name you used when you first signed up? Just go ahead and change it in our new tab Update Profile in the My Account page!

‍

You can now edit your name and change your profile image.

‍

... and other improvements.

We've also enhanced the following:

Incident Details graph now show the min/max of the Expected vs the Actual for Volume graphs.

You can now save time by updating your table-level tests (Freshness, Volume, Schema drift) from the Data Quality page directly.

‍

You can also view our Docs Changelog to see the nitty-gritty details of the fixes we've done since our v1.0.

Read other blog articles

Reduce Time to Value for Data Observability

The revamped Config module is a new feature in our data observability solution that can help organizations to reduce time to value and get started with data observability quickly and easily.

By

Melanie Yong

September 29, 2023

Data Governance - Essential for the Modern Data Stack

Data governance is essential for modern businesses, but it can be a complex and time-consuming process. With this new update, you can store and manage all of your governance policies in one place, create custom classifications for data, and automate your data governance processes.

By

Melanie Yong

September 26, 2023

What is Data Observability? Data Engineers Guide

The technology landscape has shifted dramatically in the past few decades. As data becomes the new oil, its smooth flow, availability, and reliability are as essential as the oil flowing through the pipelines of the world. Just as the oil industry requires rigorous checks and measures to prevent leakages and ensure quality, the data industry requires its own set of checks, known as Data Observability. For data engineers, this emerging concept has become vital in ensuring that data systems remain robust, reliable, and trustworthy.