Data profiling is an essential component in maintaining data quality. Through running a diagnosis of the data, including its sources and metadata, errors and inconsistencies can be picked up on and amended before becoming actionable intelligence. This helps organizations to understand their data better and take action to improve data quality.

What are the different types of data profiling?

Data profiling can be broken down across structure, content and relationship discovery as follows:

• Structure Discovery: Examining the data to ensure consistent formatting.
• Content Discovery: A closer look at the contents of the data, checking for gaps or anomalies in the data itself.
• Relationship Discovery– Investigating how the data relates to other datasets.

What about data profiling by source?

There are different types of profiling by source, which can be categorized by data source, data warehouse, or data management.

• Data source identifies issues with the structure or content of data coming from an external source.
• Data warehouse analyses the data that is stored in a data warehouse.
• Data management assesses the way data is managed within an organization.

How about profiling within the data itself?

This can also be broken down into three types of pattern discovery: column profiling, row profiling and cell profiling.

• Columns assess the data in each column, looking for patterns and relationships.
• Rows assess the data in each row, looking for errors and inconsistencies.
• Cells assess the data in each cell, looking for completeness.

What are the steps of data profiling?

The first step is to understand the business needs, second to identify the target data, third to collect the data, fourth to analyze the data, fifth to document findings, and lastly to resolve issues. 

Through this process, organisations can prevent costly data quality errors, which Gartner describes as duplication, lack of consistency, accuracy and completeness. Moreover, it feeds into the process of data migration, as data needs to be cleansed and profiled before it’s moved into a new location.

What is data migration? 

Data migration is the process of moving data from one location to another. This can be done for a variety of reasons, such as upgrading to a new database or moving to a new server. In order to ensure that data is migrated successfully, it is important to have a plan in place. This plan should take into account the various types of data that need to be moved, as well as the potential risks involved. Additionally, it is important to have a backup plan in place in case data is lost during the migration process. By taking these steps, you can help ensure that data migration is done smoothly and without any issues.

How can Datactics help with data profiling?

This is a core component of Datactics’ Self-Service Data Quality platform. Every project, programme or tactical exercise starts with profiling to help shed light on exactly what data our clients are dealing with. As many data leaders confess to not knowing what’s in their data, or indeed where it is, profiling is an essential step for us to support our customers in delivering end-to-end perfected data across the enterprise.