This post will contain an overview of what purview offers. The diagram created is a representation of the the data flows in purview and in there lays the way to properly architect with an end in mind. in the coming days, I will do an entry per feature in purview, the reason is that purview changes so fast, compartmentalizing it will be the same way.

Feature overview

Data Map

The data map is where you setup your collections (a directory structure used to separate access) Seperating the access in this layer is the end goal of your “directory design”. Once the collection is set, you can add data sources and ingest metadata to add to your catalog. When you register a data source, you scan and ingest metadata information which its a subject of its own.

Data Map: Collection Design

Covering Business Process , Permissions sets and collection considerations:
Collection Design in Purview
The collection design structure is crucial, its one of the components where restarting the design would be difficult. this should be the representation of your data estate, in some cases, even representing Production, Development and Testing. I review design considerations here along with permission sets.

Data Map: Data Scanning

Covering Registering data sources, classifications, scan configurations and custom scans :
(LINK COMING SOON)
When your collection is setup, you will need to consider where to add data sources. when a data source is registered, an asset is consumed in the collections and the assets are automatically accessible in your enterprise catalog. if you have the permission to read the assets of course. I will cover registering data sources, how metadata is ingested and how you should setup scanning for the data stewards or business.

Data Map: Data Scanning Lineage

Covering lineage dependecies, ETL lineage patterns and Enforcing lineage with pyatlas :
(LINK COMING SOON)
A common misconception is that if you were to scan an asset into purview, you would be able to see the lineage for the assets. This is not true, the aim of your purview setup is to register end to end resources , along with the data orchestration. The data orchestrator will provide the lineage of the assets, but these assets need to exist in purview. the end result, is you registering the data sources involved in the data movement and also engineering to maintain the lineage. I also go over how to explicitly enforce the lineage. Although this is part of the scanning I set it aside because of the potential caveats and supported vs not supported cases.

Unified Catalog

The is where you go to explore your assets. It allows you to review what was ingested in your datamap and curate the infromation by adding owners, classifications and lineage. it lets you explore with this same criteria and more, filtering by PII, Department and asset type.

Unified Catalog:Exploring and Curating Assets

Covering using the catalog, adding lineage, classifications and other things:
(LINK COMING SOON)
The is where you go to explore your assets. It allows you to review what was ingested in your datamap and curate the infromation by adding owners, classifications and lineage. it lets you explore with this same criteria and more, filtering by PII, Department and asset type.

Unified Catalog:Governance Domains

Covering designing governance domains, Enterprise Glossary, Data Products and reporting:
(LINK COMING SOON)
Governance domains are a layer on top of the datamap. Although the unified catalog lets you explore, this is intended to let you solve a specific problem. Examples would be grouping PII so data can be consolidated, grouping reports to have a data product for the executive team. I will cover how this differs and how to scale this.

Unified Catalog:Data Quality

Covering Data quality and building rules:
(LINK COMING SOON)
Data Quality is a crucial component, I will cover how to design this so it properly reflects in purview dashboards via data products. I will build come rules to allow you to scale this.