Welcome to Part Three of our three-part series on dbt project health where we look at the three areas that most commonly hold data platforms back: pipeline reliability, compute costs and deployment velocity. In this blog we argue that data team velocity is structural, driven by validation processes that scale with platform size, shared development environments and unclear data contracts, and makes the case that solving velocity is a prerequisite for AI and advanced analytics rather than a separate workstream. It closes with three organisational decisions that compound to accelerate delivery.
Blog Series Introduction
Our three-part series on dbt project health is written specifically for CDOs and Heads of Data and looks at the three areas that most commonly hold data platforms back: pipeline reliability, compute costs and deployment velocity (this blog).
Most data platforms look healthy on the surface. Jobs run, dashboards load and dashboards show green. But underneath that surface, three separate problems tend to accumulate quietly: data that’s wrong but undetected, compute spend that’s climbing for no clear reason and delivery timelines that keep stretching further from what the business expects. None of these show up as a single dramatic failure. They build up gradually, and by the time they’re visible enough to act on, the cost (in credibility, budget or lost momentum) has already been paid.
Slow Data Delivery Isn’t A Resourcing Problem
Delivery velocity is the metric most data leaders feel acutely but measure least systematically. The symptoms are familiar: a simple business request takes weeks to reach production, your backlog grows faster than it shrinks, engineers are busy, but output feels slow and the business has quietly stopped expecting data to move at the speed of its decisions.
The instinct is to treat this as a resourcing problem, which it rarely is. The organisations where data teams move fastest aren’t the ones with the most engineers – they’re the ones where the platform, the ownership model and the engineering culture have been deliberately structured for speed. That structure doesn’t emerge organically. It requires intentional decisions made, that most platforms never make, because each individual addition to the platform seemed reasonable at the time.
The Structural Debt That Slows Everything Down
Data platforms that grow without intentional governance accumulate a specific kind of technical debt that is different from, and more insidious than, conventional software debt. It doesn’t manifest as broken code, but as friction. Every change takes longer than it should, every modification requires more coordination than it really warrants and the cumulative effect is a team that is working hard but delivering slowly.
This friction is typically concentrated in three places:
- Validation processes that scale with platform size, not change size. When a quality check process runs the entire platform to validate every change, regardless of how small, the time cost grows with the platform. A team with a fifty-minute validation job on a three-hundred-model platform pays the same cost to fix a typo as to restructure a core data asset. That asymmetry destroys your delivery velocity.
- No safe space to develop. Engineers who test changes in shared environments – or against production data – move slowly because the consequences of mistakes are immediately visible to others. Development speed requires isolation. Isolation requires investment in environment management that most platforms defer indefinitely.
- Absent or implicit data contracts. When the distinction between stable interfaces and internal implementation details isn’t explicit, every change is treated as potentially breaking – because it might be. The review overhead this creates is enormous. Engineers spend time validating things that don’t need validating and the review process becomes a bottleneck rather than a quality control.
The Velocity-Capability Connection
The strategic case for investing in delivery velocity extends well beyond the day-to-day frustration of slow request fulfilment. The organisations making the most progress on AI and advanced analytics are, almost without exception, the ones that have solved the velocity problem first.
Moving from descriptive reporting to predictive analytics and AI-driven decision-making requires rapid iteration, testing hypotheses, building features, measuring outcomes, adjusting. A data team that takes six weeks to add a metric to a report cannot support that kind of iteration cycle. The sophisticated capabilities are blocked by the slow foundations.
Velocity is also a talent question, and it compounds. Good data engineers are drawn to platforms where they can move fast, see impact quickly and work without unnecessary friction. Platforms with accumulated structural debt are harder environments to retain talent in, and the loss of that institutional knowledge that follows, makes the debt harder still to address.
The Path Forward
The single most impactful technical change available to most data platforms is also one of the fastest to implement: configuring validation processes to run only against what changed, rather than the entire platform. A fifty-minute check can be reduced to under five minutes with no reduction in quality. The implementation takes hours, not weeks. For most teams, the return is immediate.
The harder work is organisational. Three key decisions can accelerate delivery velocity more than any tooling change could.
- Establish domain ownership. Data assets should have named owners, accountable for their quality, cost and reliability. Where ownership is ambiguous, every change requires cross-team coordination that adds days or weeks. Resolving that ambiguity can be unglamorous work, but has a compounding return.
- Define your stable interfaces. The assets that the rest of the organisation builds on should be explicitly identified and managed as contracts. Everything else should be changeable without issue. Drawing that line removes the review burden that accumulates when every change is treated as potentially breaking.
- Measure delivery time, not just backlog size. Time from request to production should be tracked by request type and trending over time, is a leading indicator of structural health. A deteriorating trend is an early warning that debt is accumulating faster than it’s being addressed.
What This Means for Data Leaders
The conversation about velocity is ultimately a conversation about what a data platform is for. A platform the business trusts to move at the speed of its decisions is a fundamentally different asset from one where requests queue for weeks and stakeholders have learned to work around it, and that work around probably involves an Excel sheet that has embedded code no one understands.
The data teams best positioned to deliver on AI and advanced analytics are the ones building the structural foundation now, not retrofitting it after the demand has already arrived.
Analytics8 has spent over two decades helping data organisations move faster without cutting corners. We bring the cross-industry pattern recognition to identify where structural debt is concentrated, the senior engineering capability to fix it and the enablement model that ensures your team owns and can sustain the improvements. No handoffs. No diagnosis without delivery. The same team that assesses your platform builds the solution.
Our dbt Project Health Check assesses deployment velocity alongside pipeline reliability and cost efficiency, giving you a complete picture and a clear, prioritised path forward.
Ready to find out what’s slowing your platform down? Book a free, no-pressure strategy session with one of our senior consultants. We’ll talk through where you are, what’s blocking progress, and what a practical path forward could look like.
