Hey there, marketer! You have probably heard of Funnel, or even better, you are using it for all your marketing data. If so, you already know that getting an overview of all your data across platforms has never been easier.
But what about your Business Intelligence (BI) & Analytics Team? Are they able to consolidate all their data into a single source of truth in an automated way, or are they spending a lot of time on manual processes which are not only costly but also error-prone? This is where a marketing data warehouse comes in.
How to take that next step towards data maturity
Data maturity is a way to measure the data handling capabilities of a company. There are various phases of data maturity, which lead eventually to high levels of understanding and data literacy, with data as a critical part of the daily functioning of the business. So, how does a marketing data warehouse help?
A marketing data warehouse empowers companies to consolidate all their data into a single source of truth. It helps organizations become more data-driven in their decision making.
Read more on how Dustin reduced manual work and focused on value-adding activities such as data enrichment. Or how Social Lab Group, an agency based in Brussels, taps into its benchmark data using a data warehouse.
This article will discuss how you can take that next step towards increasing your data maturity.
Related reading: why data maturity is the key to future success.
What is a data warehouse?
In general, a data warehouse (DW or DWH) is a system that enables reporting and data analysis. A data warehouse is home to your high-value data, generated by different business applications used across your organization, such as marketing, product, finance and sales. It is generally quite cost-effective to store data using data warehousing, and offers high performance when reading from it.
Historically, companies chose to have on-premise data warehouses. More recently, there has been a trend towards cloud-based (Amazon Redshift, Google BigQuery, Snowflake, etc.) that BI or IT teams can deploy within minutes.
What is a marketing data warehouse?
A marketing data warehouse is a data warehouse that is primarily used for marketing data. It connects to multiple data sources, including marketing platforms, your website, Google Analytics and your CRM. A marketing data warehouse can contain large amounts of marketing data and is meant to help organizations making the right business decisions.
A marketing data warehouse is essentially a giant storage unit specifically designed for marketing data. Imagine it like a central library for all your marketing information, but instead of books, it holds data from various sources.
In practice, organizations often use one and the same data warehouse for both their marketing data and other data.
Why do I need a data warehouse?
As businesses become more customer-centric, an average marketer uses multiple different marketing technology solutions such as account-based marketing, social media marketing, content marketing, and many more. That means a lot of marketing data every single day.
But getting an overview across departments such as finance, marketing, CRM, etc., can be incredibly painful as data from these systems are in their silos, and there is no single source of truth.
Many companies still use Excel or Google Sheets to stitch together this data, which requires manual work and can be error-prone. This limits the possibility to scale their reporting and the type of analysis they can perform on their marketing data. Excel can work for small use cases but has limitations in the size of the sheet and can be incredibly fragile.
A marketing data warehouse provides a strong foundation for data-driven marketing. By giving you a clear view of your marketing efforts and customer behavior, it helps you make better decisions and achieve your marketing goals.
Getting insights with data from various sources
It’s one thing to gather all this marketing data and learn about data warehousing – but it’s another to understand it. But without actionable insights, the power of your marketing data is untapped.
For example, if your main objective is to answer the question “Tell me what happened on this platform?” You can easily do this by looking at one of the pre-built reports from these platforms to understand that. And a tool like HubSpot can quickly answer “How many leads were created in March.”
But if we were to probe the questions further, “Tell me what happened and why?, getting an answer is no longer possible by just looking at one platform. To answer “What caused a 40% increase in leads for March compared to the previous month?”, we need to understand what other activities happened on different platforms during the same period.
Answering such questions requires you to look at the data holistically, as many different driving factors might have affected the outcome.
That is where a marketing data warehouse comes into the picture. It offers limitless scalability and is a much more robust solution than an Excel sheet. It can harmonize your data coming from multiple sources, making it the main data source to create actionable insights.
How does a marketing data warehouse help your business?
Data from everywhere: Marketing data gets pulled in from all your different marketing channels, website analytics tools and Google Analytics, and even your CRM. Data sources can include things like ad campaign performance, website traffic stats, and customer purchase history
Cleaning and organization: Raw data isn't just dumped in a pile. It goes through a cleaning and transformation process to ensure it's consistent and usable for analysis. Think of it like librarians sorting and organizing books into categories.
Centralized location: All this cleaned-up structured data gets stored in one central location, typically in the cloud. Stored data is easy for marketers and analysts to access and analyze the information.
Analysis power: With all your marketing data in one place, you can use powerful tools to identify trends, understand customer behavior, and measure the effectiveness of your campaigns. It's like having a powerful research center for your marketing efforts.
Which marketing data warehouse best suits my needs?
There are various options available when choosing a marketing data warehouse, and some of them can be very technical and require a lot of effort to maintain. But, modern solutions are designed to be user-friendly and can be as simple as using a Google sheet.
Not all data warehouses are created equal. They can range from being a personal or departmental level scope with one or two subject areas to an Enterprise scope, driving strategy, and integrating various aspects of the business. When starting your data warehouse journey, you can start small with up to 10 different data sources and grow slowly from there.
The most popular data warehousing solutions
The most popular cloud-based data warehouse products such as Snowflake, Google BigQuery, Microsoft Azure, and Amazon Redshift offer columnar storage and data pipelining. This results in fast query performance, cost-effectiveness and high scalability, and streamlined analytics data pipelines. So, as all of these cloud data warehouses are well equipped to cover most business cases, any one of them can be a good starting point if this is your first time building one.
Most of these platforms are pretty similar to each other, so to help you make a better decision on the data warehouse you want, try to narrow down your choices to one or two. Start by understanding the type of data you wish to store and how much storage capacity you will need. You will also need to plan what kind of queries you want to execute and how that need will grow in the coming one to two years. That should give you enough information to choose the right one for your business.
Check within your organization and talk to your IT team; the best cloud-based solution might already be available and is being used by other departments. Reach out to your trusted advisor or technology partner to get expert advice to decide which data warehousing solution is best for your business.
Below are a few things to consider before making your final decision:
-
-
Define the use case: Decide on what kind of outcomes you want to achieve and if there would be other teams involved. This information can help design the data architecture to drive these outcomes and support other teams. If there are special requirements such as data storage location, security certifications, or row/column level security, make sure to check if the data warehouse vendor offers that before making the decision.
-
Data cost: Storing big data can be cheap using file storage but the cost of processing can pile up in the long run if you do not plan correctly.
-
Query performance: Decide on how much data you will process, how long you can wait for that process, and how much you are willing to spend. Some platforms might charge a premium for varying levels of performance.
-
Maintenance: Plan for who will maintain and track the adding/removing of metrics, adding/removing of access, and if that would be done in-house or in collaboration with a technology partner.
-
Vendor: Keep in mind which business productivity suite is being used within the organization. If you are a Microsoft enterprise customer, there is an advantage of choosing Azure, as your organization might already have some technical knowledge and agreement in place.
-
Once you have gathered all the business needs and budget information, use this information to compare the different data warehouse platforms.
How can Funnel help export your marketing data to a data warehouse?
Funnel is platform agnostic when it comes to the data destination you choose for your exports. It offers three options:
-
-
Sending the data directly to your on-premise data servers using SFTP (Secure File Transfer Protocol)
-
Direct connection to some of the most popular cloud data warehouses like BigQuery and Snowflake
-
High-speed data transfer to cloud-based file storage platforms such as Google Cloud Storage, Amazon S3, Azure Blob Storage, from where you can easily move your data to a database of your choice.
-
The main benefit of using this approach is that you, as a customer, can decide if you want to export the data directly to a data warehouse or build a data lake that performs some data enrichment before that data lands at your final destination.
(i) Companies who prefer to store and manage their on-premise data can choose an SFTP option to connect Funnel to their file storage system using a Secure Shell (SSH). The files exported to this location can be directly consumed using a BI tool, or be used to trigger an automated data pipeline, which then transfers it to the database running securely within your organization. Building and maintaining a data pipeline requires technical implementation and is typically done by an IT/Data team.
(ii) Companies using BigQuery and Snowflake can use Funnel’s direct connection to these cloud data warehouses without any technical implementation as both these destinations are Plug and Play. This enables marketers to add and remove fields directly within Funnel, without having to do any technical configuration and knowledge of data pipelining.
This is a popular choice amongst marketers who are savvy in both digital marketing and data analysis. They can quickly experiment with their reporting by adding/removing KPIs without depending on their data team.
(iii) For companies using other data warehouses, such as Amazon Redshift, Microsoft Azure, Micro Focus Vertica, Teradata, Amazon DynamoDB, PostgreSQL, or others, they can first export their data to a file storage system and then use a data integration service to move it to the final destination.
Most of these data warehouses are available on cloud platforms such as Google, AWS, and Azure. Each cloud offers its file storage system and a data integration service:
-
-
Google Cloud: Export data to Google Cloud Storage. Then you can either write your automated script/function to move data to the database or use Cloud Data Fusion (a no-code fully managed data integration service).
-
Amazon Web Services: Export data to AWS S3. Then either write your code to transfer data to the database or use AWS Glue (a no-code managed service).
-
Microsoft Azure: Export data to Azure Blob Storage. Then you can either write your code to move the data to the database or use Data Factory (a no-code fully managed service).
-
People are often a little intimidated by the term (marketing) data warehouse, as they are primarily used by engineers and data analysts. I hope this post was able to help you as a marketer understand what they are and to speak the same language as your data team.
Using Funnel and a marketing data warehouse
Below is a flowchart that can help with your decision-making process when choosing a data destination in Funnel. If you want to know more about sending your analysis-ready data to a marketing data warehouse, don’t be a stranger - get in touch!
FAQ
Data warehouse vs data lake: what’s the difference?
Data warehouses and data lakes are both data storage solutions, but they have some key differences in terms of structure, purpose, and use cases.
A data warehouse is very structured and designed for analytical reporting and business intelligence. Data warehousing allows users to analyze historical data to gain insights into trends, performance, and customer behavior. It’s usually used for sales analysis, marketing campaign effectiveness and financial reporting.
A data lake, meanwhile, is more flexible and can store raw data as well as structured data. It’s used as a central repository for all data, and is great for data discovery and more advanced data analysis like data science and machine learning.
What’s the difference between a marketing data warehouse and a data warehouse?
A marketing data warehouse is just a data warehouse that’s used for marketing data. Some tools have more marketing-specific analysis competences, and are better suited to marketing insights than general business or financial insights.
If a data warehouse is a general library, you can think of a marketing data warehouse like a specialized library that only has books about marketing!