Data Governance Explained: Catalogs, Lineage, and Access Control

Data governance is the set of roles, policies, and tooling that decide who owns each dataset, what it means, who may access it, and how long it is kept. In practice it rests on a few concrete building blocks: clear ownership, a data catalog so people can find and understand data, lineage so they can trace where it came from, classification of sensitive fields, enforced access control, and retention rules. Done well, governance makes data easier to use safely, not harder.

Why does data governance matter for engineering teams?

Without governance, a data platform tends to drift toward the same set of problems. Nobody knows which of three similarly named tables is the correct one. A column holding email addresses sits in a table everyone can query. A dashboard breaks and no one knows who to ask. Old data piles up indefinitely because deleting anything feels risky.

None of these are exotic. They are what happens by default as more teams produce and consume data. Governance is the deliberate answer, and the engineering side of it is mostly about making the right behavior the easy behavior through automation.

What are the core components of data governance?

Who should own data?

Ownership is the foundation. Every dataset needs one accountable team, not an individual who might leave and not "the data team" as a catch-all. The owner is responsible for the dataset's definition, quality, access decisions, and change communication.

A workable model separates two roles:

• Data owner: the team accountable for the dataset, usually the one that produces it or understands its business meaning best.
• Data steward: the person or group who maintains definitions, reviews access requests, and keeps catalog entries accurate.

Make ownership machine-readable. If every model or table definition must declare an owner before it can be deployed, you never end up with orphaned data. Data contracts, discussed in data quality testing and monitoring, formalize this for datasets shared across teams.

What is a data catalog?

A data catalog is a searchable inventory of your data assets and their metadata. Think of it as documentation that is generated as much as possible from the systems themselves, then enriched by people.

A useful catalog entry includes:

• Name, description, and owner.
• Schema with column descriptions and types.
• Classification tags, such as PII or financial.
• Freshness and quality status.
• Upstream and downstream lineage.
• Usage signals, such as how often it is queried and by whom.

Tools in this space include open-source projects such as DataHub, OpenMetadata, and Amundsen, as well as catalogs built into cloud platforms and lakehouse products. The tool matters less than two habits: ingest metadata automatically from warehouses, orchestrators, and transformation tools, and keep descriptions in version control next to the code that builds the table. A catalog that relies on people typing into a web form goes stale quickly.

If you use dbt, model and column descriptions written in YAML flow into most catalogs automatically; see dbt explained for how that works.

What is data lineage?

Lineage is the graph of how data flows from sources through transformations into tables, dashboards, and models. It answers two practical questions:

• Upstream: where did this number come from? Useful when debugging a wrong value.
• Downstream: what breaks if I change this? Useful before altering or dropping a column.

Lineage can be table-level or column-level. Table-level lineage is easier to collect and covers most impact analysis. Column-level lineage is more valuable for sensitive data, because it shows everywhere an email column propagates, including derived tables that renamed it.

Lineage is usually collected by parsing SQL, reading metadata from orchestrators and transformation tools, or analyzing query logs. The OpenLineage specification defines a common event format so different tools can emit lineage in a consistent way.

How do you classify PII and sensitive data?

You cannot protect what you have not labeled. Classification assigns each column a sensitivity level so policies can act on it.

A simple scheme is often enough:

1. Public: safe to share broadly.
2. Internal: fine for employees, not for external sharing.
3. Confidential: business-sensitive, restricted to specific teams.
4. Restricted: personal or regulated data such as names, emails, government IDs, or payment details.

Automated scanners can flag likely PII by column name patterns and sampled values, but treat their output as suggestions for owners to confirm. Store the final tags in the catalog and, ideally, in the warehouse itself so access policies can reference them directly.

How should access control work?

Access control turns classification into enforcement. A few principles keep it manageable:

• Grant to roles, not individuals. People join and leave; roles like analyst_finance stay stable.
• Least privilege by default. New users start with access to public and internal data only.
• Tag-based policies for sensitive columns. Instead of configuring each table, define a policy once, such as "columns tagged pii are masked unless the user has the pii_reader role", and let tags drive it.
• Row-level security where needed. For example, regional managers see only rows for their region.
• Audit everything. Keep query and grant logs so you can answer who accessed what.

Here is an example of role-based grants and a masking view in standard SQL; exact syntax for native masking policies varies by platform:

CREATE ROLE analyst;
CREATE ROLE pii_reader;

GRANT SELECT ON analytics.fct_orders TO analyst;

-- Analysts query the view; only pii_reader can see raw emails
CREATE VIEW analytics.dim_customer_safe AS
SELECT
  customer_id,
  CASE
    WHEN pg_has_role(current_user, 'pii_reader', 'member') THEN email
    ELSE '***masked***'
  END AS email,
  country,
  signup_date
FROM analytics.dim_customer;

GRANT SELECT ON analytics.dim_customer_safe TO analyst;

The example uses a PostgreSQL function for the role check. Many warehouses provide dedicated masking and row access policies that attach to columns or tags, which scale better than hand-written views.

How should data retention be handled?

Keeping data forever is a cost and a liability. Retention policies define how long each class of data is kept and what happens after, whether deletion, anonymization, or archival to cheaper storage.

Practical steps:

• Set a default retention per classification level, and let owners justify exceptions.
• Partition large tables by date so expiring old data is a cheap partition drop rather than a large delete.
• Plan for deletion requests on personal data, which means knowing, through lineage, every table a person's data reached.
• Remember that backups, snapshots, and table-format time travel also retain data; include them in the policy.

A practical data governance rollout plan

Trying to govern everything at once usually stalls. A phased approach works better:

1. Pick critical datasets. Start with the tables behind key reports and anything holding personal data.
2. Assign owners. Add a required owner field and block deployment without one.
3. Stand up a catalog. Ingest metadata automatically and add descriptions for the critical tables first.
4. Classify sensitive columns. Run a scanner, have owners confirm, and store tags centrally.
5. Enforce access. Move to role-based grants and tag-driven masking for restricted data.
6. Add lineage and quality signals. Surface them in the catalog so trust is visible.
7. Define retention. Automate expiry for each classification level.
8. Expand and review. Extend to more datasets and review access periodically.

This fits naturally with platform choices discussed in data warehouse vs data lake vs lakehouse, since governance features differ across those architectures.

Key takeaways

• Data governance is ownership, discoverability, traceability, protection, and lifecycle management for data.
• Every dataset needs one accountable owning team, declared in code and enforced automatically.
• A data catalog is only useful if metadata is ingested automatically and descriptions live with the code.
• Lineage powers both debugging and impact analysis, and column-level lineage matters most for PII.
• Classify sensitive columns, then drive access control and masking from those tags.
• Roll out in phases, starting with critical and sensitive datasets.

Frequently asked questions

What is the difference between data governance and data management?

Data management is the broad practice of storing, moving, and processing data. Data governance is the subset that sets rules and accountability: who owns data, who can access it, and how it must be handled. Governance defines the policies; management carries them out day to day.

Do small teams need data governance?

Yes, in a lighter form. Even a small team benefits from named owners, basic descriptions, and restricted access to personal data. Starting early is far easier than retrofitting governance onto a sprawling platform.

Is a data catalog the same as data governance?

No. A catalog is a tool that supports governance by making metadata, ownership, and lineage visible. Governance also requires policies, enforced access control, retention, and people accountable for decisions.

How is data lineage collected?

Lineage is typically built by parsing SQL transformations, reading metadata from orchestrators and transformation tools, or analyzing warehouse query logs. Standards such as OpenLineage let multiple tools report lineage in a common format.