Data products over data projects

The recognisable end state of project-based data work is a monthly meeting in which four teams present four different figures for the same measure, and the discussion becomes about whose extract is correct rather than what to do. Each figure is defensible. Each pipeline was built by a project that closed.

Nobody is at fault. A project is funded to deliver a report, and it does. It is not funded to own the definition afterwards, monitor upstream schema changes, or tell downstream consumers when the logic changes. So none of that happens.

A data product has a named owner accountable for it while it exists - not for the duration of a delivery phase. It publishes a schema contract that consumers can build against, and breaking changes go through versioning and deprecation like any other API.

It has documented semantics: what 'active customer' means here, what is excluded, what the known caveats are. It has freshness and quality expectations that are monitored, with alerts going to the owning team rather than to whoever notices the number looks odd.

Crucially, it has consumers who are known. When you can list who depends on a dataset, you can change it safely. Most of the fear around touching data platforms comes from not knowing who is reading.

This is primarily a funding and staffing change rather than a technology one. It means product-style ongoing allocation for a small number of high-value data products instead of a series of project budgets, and it means accepting a smaller catalogue that is trustworthy over a large one that is not.

Start with one metric that causes visible arguments. Give it an owner, a contract, monitoring and documentation. The improvement is obvious enough within a quarter to justify the second one, and that is a far easier internal case than a platform-wide reorganisation.