New: Data Recon UI and Preview Sample Data

Data Recon gets a new look, and we've added the ability to preview sample data directly within decube. Check it out here.

By

Melanie Yong

Updated on

January 10, 2024

Since we launched decube last November, we’ve given our Data Quality and Catalog + Governance module each a facelift. So now, it’s time for Data Recon to get a new look!

If you haven’t tried out our Data Recon module before, it’s an easy way to compare and validate two versions of any datasets. This is important if your data is constantly being replicated and moved from one data warehouse to another, or going from staging and production. Discrepancies such as dropped or mutated data happen; and it’s often your team will end up spending precious time manually comparison between datasets with custom SQL queries, custom scripts or spreadsheets. But you don’t have to anymore

With our Data Recon module, it’s easy to just select the tables you want to compare, configure the settings, set the schedule and let us do it for you.

A focused page for configuration

Creating a new data recon has been made into a focused experience where you can now set up your configuration without feeling cramped. We’ve also brought over the new asset icons from the Catalog and made it easier to search and find the tables from your data sources. 

Setting up a new data recon.

We’ve also updated the Data Recon Details page so that you have a focused view on the results of the recon. If you’d still like to see your Config that created the recon, you can check it on the top right button. That’s also where you can modify the schedule of the current recon if needed.

Data Recon details example page.

You can now also download the Details tab from your Recon in .csv format by using our "Export Recon" button in the top right. This will download all the rows that were found to be problematic during the recon job.

Compare between multiple data sources

You can now compare two tables between any supported data sources. That’s right, if you have to compare a table within PosgreSQL with one from BigQuery, you just need to toggle on “Use separate data sources”. You will just need to ensure that both tables share the same primary keys, and when adding the additional fields to be mapped, they are of the same logical type. 

Compare datasets from two sources.

Filter rows by adding conditions

You can also add conditions directly to your recon configuration to help filter the rows to be checked. Use this to exclude rows that are not in the scope of the comparison, or set limits to the time period to be checked. 

How this works is that we use your conditions as a WHERE clause. You just have to make sure that you write the condition in the SQL dialect of the source that you’ve selected. This is an optional step, and you can add them to either of your tables, both or none at all.

Set up filters in your Data Recon to ensure that you're checking the right block of data.

Make sure that your data arrives your destination in the way you expect them to be. Check out how to set up a new recon here.

Preview data directly from Catalog

Sometimes you just want to get a context of the table by seeing data directly from your table, and that’s fine. Now you can do that by navigating to the Preview tab for your table directly within decube’s Asset details itself to see 10 rows of sample data from your table. 

Data that you have marked as PII or Sensitive in the Classifications of our Schema tab will be automatically showed as masked via our platform so that your sensitive data do not get exposed to your organization members.

Example of sample data from a Preview.

Here’s a sneak peek for our readers: When we roll out our Group Access Control feature (very soon!), we’ll also provide the ability to limit access to who can see this tab, or run the Preview, so you’ll have further control over who can see the sample data.

Check out our Public Roadmap for what we're currently working on and what's going to be released real soon!

Table of Contents

Read other blog articles

Grow with our latest insights

Sneak peek from the data world.

Thank you! Your submission has been received!
Talk to a designer

All in one place

Comprehensive and centralized solution for data governance, and observability.

decube all in one image