What I want from a BI tool

Most layers of the modern data stack are well served. Snowflake, BigQuery, dbt, Fivetran — all are great. But BI sucks. It’s just terrible. Even Looker, the best in class tool, sometimes brings me to the verge of tears. It’s just one paper cut after another.

Paper cuts aside, what’s most frustrating is that most BI tools seem to totally miss the point. They exist in some weird universe that doesn’t match how teams use data to understand the world.

There’s hope. A bunch of new product are rethinking how to present data. The new ideas cropping up in these different tools really question what BI should be. I figured I’d put together my own wish list. This is rough thinking, not a product spec!

Relationships

BI tools put very little effort into clarifying how different dashboards relate to each other. They have basic concepts like folders, but no cross-linking or nesting. There’s also a lack of relationship between metrics or charts. Business users know that some metrics are inputs for others, but somehow this is not codified in any BI tool I’ve seen. This section of a recent interview with Abhi Sivasailam felt immediately familiar to me: Being explicit about dashboard hierarchy, and using that to represent an opinionated view of what matters most in the business and how the support metrics relate, is a critical part of BI that is not well served.

At a minimum, I’d like some ability to link between dashboards to make discovery easier. I’m thinking, for instance, of how Notion does both hierarchy and cross-referencing:

To go further still, you could begin to codify relationships between metrics in a way that implies causality or measures correlation. DoubleLoop is doing interesting things here; it’s not a replacement for BI but it’s a hint at what things could look like if you started from a different point.

Collaboration and preservation

Charts and dashboard produce so much conversation. And yet, none of that conversation happens in the BI tool. Instead, teams copy and paste charts into Slack; they copy and paste charts into meeting agendas; they copy and paste charts into emails. Those conversations provide critical context around the data, and are still valuable months later — except they’re totally undiscoverable the next time someone looks at the dashboard.

I like what Hex has started to do with comments. This is just the basics, for now. But it could go much, much further. What would collaboration for BI look like if it felt like Figma? Why isn’t there a BI tool that integrates Loom? What if you could track the lineage of a chart to see who has copied and modified it, to track a sort of evolutionary history? Count seems to be asking some of these questions, though it seems more focused on ad hoc investigation than BI per se.

Most effective meetings have an agenda shared beforehand as a place to aggregate topics and questions for discussion. Teams that want to use data in this way — as the raw material for discussion, shared in advance, and decorated with questions — can't do it. They either screenshot and paste charts into a Google doc, or they review the dashboard live. This seems broken. This is a missed opportunity to capture a team's understanding of an issue, allowing future teammates to review it. In cases where teams review the data live, the discussion and questions evaporate into the ether. Not only that, the data itself will change. So many times I think to myself, "I wish I could remember what we were looking at, and what we thought, on this issue 3 months ago," only to resign myself to the fact that the discussion happened in front of a live dashboard and then evaporated.

Chart lockups

BI tools are surprisingly unopinionated about the right way to present data. As a result, I find myself doing so much chart boilerplate. The core of the problem is a narrow focus on individual charts and tiles, instead of thinking about the chart as part of a broader lockup.

Because I have a standard set of supporting metrics I want to surround a chart with, I find myself recreating calculations for things like w/w change. In BI tools today, each of the tiles below is its own object, with its own specs.

Things would get a lot easier if the BI tool thought of the objects above as part of one related, configurable unit. Calculations for rates of change, etc. could be done automatically and toggled on and off.

Opinionated data models

BI tools provide no out-of-the-box reporting. And yet, certain reports should be nearly standard across businesses. The limiting factor is at the data model layer. Every team models its data differently, even when they shouldn’t.

I wonder if the BI layer could address this, using standardized reports as the carrot to encourage standardized datasets. If the BI layer could offer dataset templates that, when used, would produce a bunch of standard measures and reporting, data teams could get to value much more quickly. It might have the secondary effect of enforcing simplicity further down in the data warehouse. (BI tools today often have “dashboard templates” using dummy datasets. But these templates are nice to look at and nothing more, because there’s no opinion on how to structure your own dataset to match the needs of the template.)

SaaS tools do out-of-the-box reporting well because they own the underlying data models. I find myself wishing a BI tool would approximate the same by simply saying: “If you can give me a table with the following columns at a minimum, we will give you a dashboard that calculates all the common metrics you’d want. Give us every revenue event in a specific format, for instance, and we’ll give you a set of revenue retention charts automatically.”

Phew, that’s all for now!