Marketing data warehouse: everything you need to know

Hey there, marketer! You have probably heard of Funnel, or even better, you are using it. If so, you already know that getting an overview of all your data across platforms has never been easier.

But what about your BI & Analytics Team? Are they able to consolidate all their data into a single source of truth in an automated way, or are they spending a lot of time on manual processes which are not only costly but also error-prone? This is where marketing data warehouses come in. 

How to take that next step towards data maturity

A marketing data warehouse empowers companies to consolidate all their data into a single source of truth. It helps organizations become more data-driven in their decision making.

Read more on how Dustin reduced manual work and focused on value-adding activities such as data enrichment. Or how Social Lab Group, an agency based in Brussels, taps into its benchmark data using a data warehouse.

This article will discuss how you can take that next step towards increasing your data maturity. 

Related reading: why data maturity is the key to future success.

What is a data warehouse?

In general, a data warehouse (DW or DWH) is a system that enables reporting and data analysis. It is home to your high-value data, generated by different business applications used across your organization, such as marketing, product, finance and sales. It is cheap to store data and offers high performance when reading from it.

Historically, companies chose to have on-premise data warehouses. More recently, there has been a trend towards cloud-based (Amazon Redshift, Google BigQuery, Snowflake, etc.) that BI or IT teams can deploy within minutes.

What is a marketing data warehouse?

A marketing data warehouse is a DW that is primarily used for marketing data. It contains data from multiple sources, including marketing platforms, your website, Google Analytics and your CRM. A marketing data warehouse can contain large amounts of data and is meant to help organizations making the right business decisions.

In practice, organizations often use one and the same data warehouse for both their marketing data and other data. 

Why do I need a data warehouse?

As businesses become more customer-centric, an average marketer uses multiple different marketing technology solutions such as account-based marketing, social media marketing, content marketing, and many more. That is a lot for daily operations. 

But getting an overview across departments such as finance, marketing, CRM, etc., can be incredibly painful as data from these systems are in their silos, and there is no single source of truth. 

Many companies still use Excel or Google Sheets to stitch together this data, which requires manual work and can be error-prone. This limits the possibility to scale their reporting and the type of analysis they can perform. Excel can work for small use cases but has limitations in the size of the sheet and can be incredibly fragile.

Getting insights with data from various sources

For example, if your main objective is to answer the question “Tell me what happened on this platform?” You can easily do this by looking at one of the pre-built reports from these platforms to understand that. And a tool like HubSpot can quickly answer “How many leads were created in March.”

But if we were to probe the questions further, “Tell me what happened and why?, getting an answer is no longer possible by just looking at one platform. To answer “What caused a 40% increase in leads for March compared to the previous month?”, we need to understand what other activities happened on different platforms during the same period.

Answering such questions require you to look at the data holistically, as many different driving factors might have affected the outcome. 

That is where a marketing data warehouse comes into the picture. It offers limitless scalability and is a much more robust solution than an Excel sheet. It can harmonize your data coming from multiple sources, making it the main data source to create actionable insights.

Which data warehouse best suits my needs?

There are various options available, and some of them can be very technical and require a lot of effort to maintain. But, modern solutions are designed to be user-friendly and can be as simple as using a Google sheet. 

Not all data warehouses are created equal. They can range from being a personal or departmental level scope with one or two subject areas to an Enterprise scope, driving strategy, and integrating various aspects of the business. When starting your data warehouse journey, you can start small with up to 10 different data sources and grow slowly from there. 

The most popular data warehousing solutions

The most popular cloud-based data warehouse products such as Snowflake, Google BigQuery, Microsoft Azure, and Amazon Redshift offer columnar storage and data pipelining. This results in fast query performance, cost-effectiveness and high scalability, and streamlined analytics data pipelines. So, as all of these cloud data warehouses are well equipped to cover most business cases, any one of them can be a good starting point if this is your first time building one.

Most of these platforms are pretty similar to each other, so to help you make a better decision on the data warehouse you want, try to narrow down your choices to one or two. Start by understanding the type of data you wish to store and how much storage capacity you will need. You will also need to plan what kind of queries you want to execute and how that need will grow in the coming one to two years. That should give you enough information to choose the right one for your business.

Check within your organization and talk to your IT team; the best cloud-based solution might already be available and is being used by other departments. Reach out to your trusted advisor or technology partner to get expert advice to decide which data warehouse solution is best for your business.

Below are a few things to consider before making your final decision:

• • Define the use case: Decide on what kind of outcomes you want to achieve and if there would be other teams involved. This information can help design the data architecture to drive these outcomes and support other teams. If there are special requirements such as data storage location, security certifications, or row/column level security, make sure to check if the data warehouse vendor offers that before making the decision.

  • Data cost: Storing big data can be cheap using file storage but the cost of processing can pile up in the long run if you do not plan correctly.

  • Query performance: Decide on how much data you will process, how long you can wait for that process, and how much you are willing to spend. Some platforms might charge a premium for varying levels of performance.

  • Maintenance: Plan for who will maintain and track the adding/removing of metrics, adding/removing of access, and if that would be done in-house or in collaboration with a technology partner.

  • Vendor: Keep in mind which business productivity suite is being used within the organization. If you are a Microsoft enterprise customer, there is an advantage of choosing Azure, as your organization might already have some technical knowledge and agreement in place.

Once you have gathered all the business needs and budget information, use this information to compare the different data warehouse platforms.

How can Funnel help export your marketing data to a data warehouse?

Funnel is platform agnostic when it comes to the data destination you choose for your exports. It offers three options: 

• • Sending the data directly to your on-premise data servers using SFTP (Secure File Transfer Protocol)

  • Direct connection to some of the most popular cloud data warehouses like BigQuery and Snowflake

  • High-speed data transfer to cloud-based file storage platforms such as Google Cloud Storage, Amazon S3, Azure Blob Storage, from where you can easily move your data to a database of your choice.

The main benefit of using this approach is that you, as a customer, can decide if you want to export the data directly to a data warehouse or build a data lake that performs some data enrichment before that data lands at your final destination.

(i) Companies who prefer to store and manage their on-premise data can choose an SFTP option to connect Funnel to their file storage system using a Secure Shell (SSH). The files exported to this location can be directly consumed using a BI tool, or be used to trigger an automated data pipeline, which then transfers it to the database running securely within your organization. Building and maintaining a data pipeline requires technical implementation and is typically done by an IT/Data team.

(ii) Companies using BigQuery and Snowflake can use Funnel’s direct connection to these cloud data warehouses without any technical implementation as both these destinations are Plug and Play. This enables marketers to add and remove fields directly within Funnel, without having to do any technical configuration and knowledge of data pipelining. 

This is a popular choice amongst marketers who are savvy in both digital marketing and data analysis. They can quickly experiment with their reporting by adding/removing KPIs without depending on their data team.

(iii) For companies using other data warehouses, such as Amazon Redshift, Microsoft Azure, Micro Focus Vertica, Teradata, Amazon DynamoDB, PostgreSQL, or others, they can first export their data to a file storage system and then use a data integration service to move it to the final destination.

Most of these data warehouses are available on cloud platforms such as Google, AWS, and Azure. Each cloud offers its file storage system and a data integration service:

• • Google Cloud: Export data to Google Cloud Storage. Then you can either write your automated script/function to move data to the database or use Cloud Data Fusion (a no-code fully managed data integration service).

  • Amazon Web Services: Export data to AWS S3. Then either write your code to transfer data to the database or use AWS Glue (a no-code managed service).

  • Microsoft Azure: Export data to Azure Blob Storage. Then you can either write your code to move the data to the database or use Data Factory (a no-code fully managed service).

People are often a little intimidated by the term (marketing) data warehouse, as they are primarily used by engineers and data analysts. I hope this post was able to help you as a marketer understand what they are and to speak the same language as your data team.

 Using Funnel & a marketing data warehouse 

Below is a flowchart that can help with your decision-making process when choosing a data destination in Funnel. If you want to know more about sending your analysis-ready data to a marketing data warehouse, don’t be a stranger - get in touch!