What is data mesh?

Data mesh: you may have heard about the concept lately. In the world of digital marketing, it’s almost as big of a buzzword as “efficiency” right now. 

Say no more. We can help you understand what data mesh is, how it affects marketing, and what its true business value is. 

Data mesh defined

At its core, data mesh is simply an approach or strategy to data management. The approach focuses on the decentralization of data, and considers data as a strategic asset — as opposed to just a bunch of metrics that are produced as an outcome of your actions. 

The term "data mesh" refers to the interconnected nature of data in this architecture. Here's the breakdown:

Mesh: Like a physical mesh network, data in a data mesh architecture is distributed across various domains (departments/teams). These domains act as nodes, and data products they create act as the connections between them.

Data: This architecture focuses heavily on the data itself. Each domain takes ownership of its data and creates consumable data products for other domains.

So, the name "data mesh" reflects the distributed ownership and interconnectedness of data that lies at the heart of this approach

Data mesh and data fabric: what’s the difference?

Data mesh is still an emerging architecture – only introduced by technologist Zhamak Dehghani in 2018. Dehghani created the concept to tackle limitations of working in a centralized data warehouse or unstructured central data lake, as a way to ensure better data quality, remove bottlenecks and give different departments a sense of ownership over their data. 

Before that, in the late 2010s, data fabric emerged with the similar aim to break down data silos and improve data accessibility, with a focus on data democratization and modernization. But they work in different ways, with different results.

While data mesh empowers individual domains to manage their data, data fabric takes a centralized approach, acting as a unified layer for accessing and governing all your organization's data. Here are the three key differences to consider:

• Focus: Data mesh focuses on decentralized ownership, with domains taking responsibility for their data as products. Data fabric prioritizes centralized governance, ensuring consistent management across the organization.
• Data Access: Data mesh involves domains publishing consumable data products for others to integrate. Data fabric offers a single point of access to all data sources through a unified interface.
• Complexity: While data mesh can be more complex to implement due to its decentralized structure, data fabric might face scalability challenges as data volume and variety increase.

A brief history on the evolution of data analytics

Don’t worry, no textbooks are required for this history lesson. Just quickly think back over the past couple of decades. It wasn’t that long ago when data (and data analysis) became the thing successful businesses needed. 

Fast forward to today, the ever-growing amount of data we're collecting has demanded greater and greater storage solutions. Thankfully, the cost of those storage solutions has decreased over time. This increase in data storage capacity and decrease in storage cost also spurred the rise in machine learning. 

However, this all tends to favor a centralized approach to data management. Find a data lake, stuff all of your data in it, and let your powerful machine learning capabilities find juicy insights for your business to act on.

This centralized approach can have its advantages. For instance, a business intelligence team can combine data from areas like marketing or finance to gain a much greater perspective on the health of the business and where it should go next. 

The central problem of a centralized data approach

At the end of the day, more data means more complexity. If you’re working with vast volumes of data from every part of the business, you might need a central data team just to manage the storage and organization of it all. 

Now, while the centralized approach for a local streetcar company may not be such a big issue, just think about the data needs of a multinational apparel brand. The storage and maintenance needs of the sales team alone could be massive. 

Also read: What is The Modern Data Stack

Or perhaps, since we’re all marketers here, think of the marketing and advertising data. There are so many platforms, it’s hard to constantly collect everything in one place yourself. And the data seems to keep multiplying somehow. 

In fact, the global volume of data tends to double every three years. Doubling data! And as more teams create more data, they will want to work with that data more. However, a centralized data approach runs everything through one location and one team. That single team is then expected to serve the needs of the other parts of the business. 

Serving up bottlenecks

Let’s step back and think of a restaurant for a moment. In this example, it’s going to be a fancy restaurant. White table cloths. Little plates of tiny, Instagram-worthy food that seem more like art than cuisine. A multi-sensory experience. Maybe a star or two? You know, that kind of restaurant. 

If we peek into the kitchen, we’ll see a head chef leading his team to craft each of the exquisite plates. But wait! The chef won’t share her recipes with the brigade and must approve every dish before it is sent out to customers. It will quickly become impossible to cook and serve food for every guest in a timely manner. 

This is essentially what happens in a centralized data approach. A bottleneck is created – and that's where data mesh comes in.

How data mesh can help

In a decentralized approach, that same chef would share her recipes with the rest of the team and allow them to make the dishes themselves. The chef manages the team and makes sure each station is maintaining the highest of standards, tasting and examining food throughout the service. 

A good data mesh is just like a well-managed restaurant. Each station in the kitchen (or domain) manages its own dishes (the data) and resources. Well-trained waiters (APIs) carry the dishes out to the right places. This fosters collaboration and sharing, while maintaining the autonomy of each district.

Data mesh allows for this kind of decentralized approach – emphasizing ownership and self-service by individual domains (which could be departments or teams) within an organization. 

The key features of data mesh architecture 

• Decentralized ownership
  In a data mesh, each domain manages its own data – so marketing controls marketing data, finance controls finance data, and so on.
• Self-service
  Each domain in the data mesh paradigm is empowered to prepare and publish their domain data for other teams. This self-service data platform approach creates a collaborative environment.
• Focus on data products
  Instead of raw data dumps, domains create well-defined data products with clear documentation and access controls. The result is consistent data quality and easy use for other teams.
• Interconnectedness
  With a data mesh, domains can access and integrate other data products and domain data from other teams. When data can be included from various sources, it’s possible to create more holistic analysis.

The benefits of data mesh architecture

• Faster insights
  Combining domain expertise with a self-service data platform leads to faster analysis, creation of data products, and decision-making.
• Improved data quality
  When teams own their domains within a data mesh, they are incentivized to maintain high data quality that’s relevant to their needs.
• Increased agility
  The decentralized data architecture of a data mesh allows for faster adaptation to business needs.
• Scalability
  A data mesh can easily grow together with an organization to accommodate new data sources and domains.

Empowering the experts

With a data mesh approach, the teams with domain-specific expertise maintain ownership of their data while also sharing it with relevant teams. That means the marketing team owns the marketing data, while also perhaps sharing it with a BI team for broader analysis with the rest of the organization’s data. 

This has a couple benefits. A marketing team will know what data is meaningful for their needs, so it makes sense for them to be able to work with it directly. That means more meaningful and faster analysis. 

Making data integral to your process

In a data mesh approach, data isn’t just a bunch of numbers to throw into a report about last month’s numbers. Instead, data analysis becomes a core part of the business’ operations. 

Data analysis becomes particularly important for marketing teams. It can inform on what tactics or creatives worked and guide you toward the best next steps. It becomes the past and the future — a sort of marketing circle of life. 

With new machine learning capacity, data can help inform on emerging consumer behaviors, which can shape ad creatives, product designs, price changes, and much more. 

Managing it all

There is a fine, gray line when it comes to managing data permissions and access across your organization in a data mesh approach. On one hand, allowing self-service among teams can free up bottlenecks. But you may need to make sure that those teams aren’t affecting the underlying data during their analysis — before it is shared with the rest of the organization. 

Depending on your own data stack, the volume of your data, and the complexity of your organization, you’ll need to find the best solution for your own specific needs. 

The trick is to balance the needs of all the items in your organization who can benefit from working with data. Even in the fancy restaurant, the chef is still overseeing her team to make sure that everything is prepared to specification. She is tasting the food before it goes out to make sure it’s perfect. 

Some organizations may need this “head data chef” to maintain some sense of stability and consistency. 

More agility and speed for a modern world

The great thing about a decentralized or “data mesh” approach is the speed and agility it can grant teams like marketing. By avoiding bottlenecks, you can generate custom reports more quickly, leading to more insights, causing your team to make better-informed and more accurate decisions. 

Plus, as more teams start working with data more frequently across the organization, you will all become naturally more data fluent. Over time, you will become more comfortable and skilled in your data analysis, making you a more data-driven and data-savvy organization on the whole. 

If you’re interested in learning more about data mesh, including some more real-world examples of how a decentralized approach can benefit marketing, be sure to check out our latest Funnel Tip. And be sure to subscribe to our YouTube channel where you’ll get all of the tips and tricks to make you a better data-driven marketer.