The first step to solving any problem is diagnosis; this article aims to not only help you spot the most common organisational data problems, but also discuss different strategies you can employ to fix them. Let’s dive into some of them and how you can go about solving them.
The vendor avalanche
You probably have applications that run your business: Google Workspace, Salesforce, Google Analytics, Microsoft Teams, and many more. Each of these tools generates its own silo of data. Your Marketing team sees the truth in the all-seeing book of Google Analytics. Yet, Sales genuinely believe that their Salesforce Reports are what drive the business. Departments become tribally dependent on these platforms and their analytical tools, which are sub-par and mostly an afterthought. The more tools that are onboarded, the more energy the departmentalised avalanches gain, making them harder to stop.
The lack of data standardisation across departments and platforms leads to many different versions of the truth. Imagine being asked to tell the time with ten clocks, all calibrated to different time zones. You’d be better off just guessing. As a result, trust in data wanes over time and you’d be better off using your ‘gut feeling’ when making decisions. Try explaining that to your director. We also call this ‘vendor avalanche’ an application blowout.
The metric swamp
Most organisations have yet to clean up their mess of metrics. A metric swamp is a symptom of distributed data chaos created by overlapping priorities, with people consuming data in different ways across an organisation - a poor organisational design.
A problem like this isn’t so much technological as it is cultural. In a well-organised data culture, there should be a clear set of metrics that people live and breathe. The sales reps should know about their average contract value, targets, and how they’re doing regarding those targets. Sales managers should see the average deal size, annual contract value, and total contract values segmented by reps. Marketing should be able to see leads generated by source, ROI on campaigns and average website session length. There should be no ambiguity regarding where the data is coming from, what the data means, and who it's for. This can be achieved by deploying a unified data platform, such as Looker.
If you were to ask your department leads on 3-5 metrics that they work with, would they be able to answer with absolute confidence?
More importantly, if you were to ask any employee within your organisation about the metrics they use to measure their performance, would they mumble a slew of KPIs they heard their managers talk about once or twice, or would they be able to answer in confidence? Would they be aligned if you were to ask the same question to different employees with the same responsibilities? And would they be able to explain why they use that metric?
What’s worse is when employees start to bicker over what KPIs mean. “I thought our targets were total yearly revenue?”,
“Noo! It’s total annual contract value - which means it’s like a 12 month ‘sliding window’ over our revenue count”.
Then someone pipes up: “What the heck is a sliding window?”.
It’s a mess.
Navigating a business where employees generally don’t know their KPIs is like navigating through a thick swamp (hence the name metric swamp). You can do it, but it’s slow, tedious, and challenging to find a clear route anywhere - meanwhile, your competitors are racing ahead, striking chords with your potential customers and staff members.
Never mind not knowing the numbers you’re measured against; imagine knowing but not being able to see these numbers.
The reporting chokehold
Some people may call this a bottleneck; we call it a chokehold. An organisation's lack of data throughput effectively starves the company of helpful information. A resource is effectively being ‘choked out’.
This happens in organisations that have a ‘data person’ or perhaps a ‘data team’ that was hired to solve some organisational problems but eventually became relegated to becoming a ‘data helpdesk’.
“Hello, sir, ACME corp data desk; how can I help you?”
“Yes, hello. We need the most recent financial figures tomorrow for our quarterly meeting. Can you do that for us?”
“Yes, you’re ticket number #41023. Progress updates will be given in Slack, have a nice day!”
Does that sound like a fulfilling job to those initially brought in to explore and generate insights?
Unfortunately, organisations succumb to this helpdesk-ification of data teams over time due to end-users using the path of least resistance to getting reports and insights. This can be a symptom of a lack of skills; or, sub-par data technology.
The reporting chokehold leads good-hearted and well-intentioned employees down the dark path into the shadow realm.
The Shadow Realm
People get fed up with having to wait for metrics.
After the third time chasing the (already over-capacity) data team, Jordan from finance says, “Sod it, I’ll do it myself”. And Jordan heads into the shadow realm.
The shadow realm is where self-starter, well-intentioned employees go to solve their problems because the ‘proper’ way of resolving them is too:
- Slow
- Expensive
- Difficult
Jordan, our valiant warrior with the best intentions, unintentionally created an ‘off-grid’ workflow. Jordan’s finance department has started to rely on this workflow to generate its monthly reports. Jordan maybe buys a data tool to help with this workflow using their monthly budget. Maybe Jordan gets bold and decides to use a separate bit of software to visualise this data. Jordan takes ownership of this process, and everyone lives happily ever after.
That is, until:
- Jordan leaves, and now nobody knows what to do.
- The organisation’s financial information is freely floating through the dark web, since Jordan didn’t think of the security implications.
- Jordan tries to coordinate with other departments and realises they have entirely different definitions of what “revenue” means for the company.
We call this the shadow realm because it’s a prime example of Shadow IT.
“Shadow IT is the use of information technology systems, devices, software, applications, and services without explicit IT department approval.” (source)
Usually, people associate Shadow IT with software, which is true, but what about shadow data? This is data that someone has gone off to create and nobody knows about.
Because data is generated so rapidly, and departments jump at the chance to make data decisions quickly, IT can’t keep up with employees going rogue and doing their own thing.
So, how do we steer our employees away from being enticed by the prospects of instant insights by establishing their own data workflows? Remember, this usually happens due to the difficulty of procuring data in the ‘official’ ways. We have to make data easy to access!
Having one central-data residency that all departments, teams and employees are familiar with is a big start in controlling shadow data. The second step should be enabling the people that are likely to deviate to ‘see the light’ and do it the correct way. People aren’t malicious by intent - tell them why it’s essential to have a single source of truth.
How do we solve these problems?
As with all things, the strategy is straightforward, it’s the execution that’s hard, so let’s break it down into basics.
As with any effective digital transformation initiative, it’s important to consider both the technology that will be implemented and the people who’ll be using it, so we’ve divided the solutions likewise.
Vendor avalanche
The problem: too many data sources creating irrelevant/ redundant/ contradictory data.
The people fix: Gather your users, and ask what is being used, what is important, and jot down a high-level analytical workflow. Prioritise the important use cases.
The technological fix: Establish a data pipeline that centralises these applications into a singular datalake. We recommend BigQuery.
Metric Swamp
The Problem: Users don’t know what to measure, what’s being measured, and whether to trust the measurements that are happening.
The people fix: Align your departments and key metrics from the top down. What are your CxO’s priorities? What are your departmental priorities? What are your employees’ priorities? Map these to metrics, and identify where these metrics are being tracked. Come up with a shared definition for each of your metrics.
The technological fix: Once you identify these core metrics, use a BI platform to easily distribute the relevant data. We like Looker: because it’s a self-service platform, it’s easier to empower your employees to do this.
Reporting Chokehold
The problem: Your data person(s) are super busy, and reports take ages to go out. There’s not much time for ‘data ideation’.
The people fix: Teach your end users how to analyse your centralised data platform once it’s up and running, and alleviate your data team to do higher value activities.
The technological fix: Reduce the time required to extract value from your data sources. This means having a tool that is usable by your end users, that grants a high level of governance and accessibility. We recommend Looker.
The ‘shadow realm’
The problem: You’ve tried your hardest to herd your colleagues to use the tools you’ve deployed, yet there’s still usage of ‘hidden’, ‘unapproved’ software and data.
The people fix: Keep beating the drum. People need to be reminded more than they need to be instructed.
The technological fix: Get technologies that people want to use, and ensure that they have the appropriate training to get up and running on them. We like BigQuery and Looker, and so do the likes of Deliveroo, and goCardless.
“Instead of just giving someone a dashboard, Looker gives people the ability to build their own queries. And that was game-changing for us because, essentially, all the ad-hoc requests that would come into the business team, went down a lot after we adopted Looker.”
- Deliveroo
If your organisation is suffering from these problems and you need help fixing them, why not get in touch with us? Drop your email below and we’ll get back to you to discuss how we can alleviate your organisation’s data problems!
