The team at WiseAnalytics worked on setting up a Customer Data Platform PoC for a global retailer. The Client wanted to prove the value of a Customer Data Platform and wanted to have quick results demonstrated through marketing uplift use cases.
As it was improving its' backend systems the client wanted to strive for a system that could process data in (near) real-time, while at the same time being able to offer centralized customer data through an API. The solution had to be hosted and setup within the standard of the group, leveraging Azure Infrastructure and NodeJS Microservices hosted on Kubernetes.
The long term plan for the project involved the unification of customer data across the group, to be able to leverage the data consolidated in a Datalake and to power different internal services with the acquired customer data in a cost effective manner, avoiding any point to point integration.
The team at WiseAnalytics was involved throughout the project, designing the solution and driving it throughout its’ implementation.
Data Ingestion and Integration
Most of the data source supported from the operating companies of the group didn’t support real-time integration, but we needed to design a solution that could support real-time ingestion and processing needs.
We separated the responsibility of ingestion and integration along the way, ingestion jobs being responsible for integrating the data onto the Azure Service bus topics in a canonical format for each entity (customers, orders, appointments, …). The choice of AzureServiceBus as a message broker was driven by its highly transactional features (locking, DLQ, …) and due to ASB being a first class citizen on Microsoft’s Cloud.
Customer Data Platform Core
The short time to market requirement, required us to look at leveraging out of the box solution for a Customer Data Platform. After having scoped the different Customer Data Platform on the market, and done an RFP, we came to the conclusion that no data platform could, at the time, meet all of our requirements, but some could play a role in our landscape.
We opted to leverage mParticle for 2 main reasons: one was its support for real-time data ingestion and processing, the second was the plethora and depth of downstream integration available, which could save us integration time and allow us to deliver business in reduced time. We furthermore relied on mParticle to perform identity resolution.
mParticle however did not offer the possibility for us to store customer’s data for longer period of time or expose it effectively. For this purpose, we developed what we called the “Customer Data Repository”. This allowed us to expose internal data to the company’s customer service tool, while at the same time avoiding data duplication in yet another tool.