A CDP Checklist - NO!
Abstract
This article is an all-encompassing look at Enterprise Data Management for Marketing Technologists that need a clear context for the components required to support robust marketing operations in a multi-dept, multi-faceted, and large enterprise. UPDATE: The EDM supports and feeds data to the CDP. The components below should, in many cases, be separated from the CDP. While some features below in the checklist are located in the CDP, many are more predominant as part of an EDM platform. Many vendors are abstracting their UI/UX and mixing metaphors of data management. While that is not wrong, it does cause more complexity for the marketer to understand specific data roles across the workflow. My argument premise is that the CDP provides a more narrow role. It does not house Identity resolution, dataset management, or even other more granular capabilities like AI/ML, Query, or Profile Management tools. My argument is it should supply the enforcement options for the profiles, what destinations they are subscribed to or assigned, and macro-audience management to coordinate omnichannel subscription and movement into the delivery tools that provide the assigned profile with the right content from broad-based to 1:1 narrow-based campaign initiatives. It's a 4-layer, perhaps even a 5-layer architecture from source to destination, horizontally, then it's 5-6 or more layers vertically at each bookend for the sources and the delivery destinations.
In an era driven by data and customer personalization, Marketing Technologists must comprehensively understand the tools and techniques that drive success. Below is what a marketing dept and marketing technologists need to look at to support their enterprise data management strategy, data strategy, and overall Martech stack. The analysis includes components like Customer Data Platforms (CDP) and downstream data apps like Campaign Management, Personalization, and Journey Management. Note that these apps are not an Enterprise Data Platform - they are mostly subscriber components of the data platform.
Many vendors will say their CDP is an Enterprise Data Management Platform, but this is far from the truth. There are only a few vendors in this genre - you will hear Reverse ETL, ELT, Zero-copy, Lifetime CDP, Universal Data Hub, et al...and many other confusing terms that cloud(pun haha) the reality of a real Enterprise Data Platform and strategy and what you must have and need to support your marketers.
Additionally, some companies(primarily pundits, CEO/CTOs of vendors, and consulting firms) must clarify what a CDP is and NOT blend it into an Enterprise Data Platform. A CDP is a small data app at the edge - it should always be distinct from your company's enterprise and Marketing Data Platform. A CDP is a component that complements an EDM, not replaces it, as many of these pundits allude to on their sites and collateral.
Adobe is one vendor that provides a multi-faceted, multi-source data platform like a Data Lake and Lake House without buying a CDP component, but they do have a CDP. Adobe makes the data platform an abstraction of many other technologies, so your company or a marketing technologist team doesn't have to go through all the processes of multi-vendor selection. Salesforce comes close, but they still need some primary capability today and are working on catching up. They have hired several people with these skill sets lately. There are a few others in this genre and many that are not. Buyer beware.
The technology abstractions the more prominent vendors are creating are freeing up massive implementation time. You still have to implement it's just a shorter time frame, and you have to commit deeply to the process but in bite-size steps to succeed. You could build a modern data platform yourself, as Adobe has done. For example, imagine having to hire staff to develop and provision all of the tools to create a self-hosted or your private cloud with a mix of solutions like Databricks or Snowflake, along with the periphery tools like Kafka, Airflow, dBT, Fivetran, and dozens more. I have the list if you are interested. You could still do this, and enterprises do, but there are benefits for the marketing department and the dedicated technologist assigned to them not to be in that business. It is faster for them to use these vendor-level abstractions, and then tie that system or platform of tools into a more significant transformation initiative within the enterprise. It's only an opinion, but it seems easier based on skillsets out there right now, time-to-market, and overall cost.
Do your due diligence on all vendors and map your requirements to what they can provide. The list below should give an excellent guide to what you need to ask and look for in your selection process. Many vendors offer all or some of these capabilities. Adobe and Salesforce have deep abstractions for them, Algonomy(a niche player for retailers and QSR-quick-serve restaurants), Amperity(deep identity resolution and analytics tools but lacks in other areas), Amplitude( approaching the Adobe/Salesforce level), and Segment and Tealium(mainly orchestration tools). There are dozens if not hundreds more(Acquia, RedPoint, TreasureData, Blueconic, SimonData et al...) - be careful and thoughtful based on need.
What does this ultimately look like? It could look like this, and much of the infrastructure to the LEFT of the Data apps could be abstracted to no UI screens or APIs, not seen, or touched at all by the user, or it could be completely exposed for more control by the client owner(you) with deep UI/UX and robust APIs. Both approaches are neither right nor wrong – depending on how much control the client(you) wants. The more control they want, the more staff and processes they must incorporate. Regardless of which approach per vendor you choose, you have opted to have them help you with managing devops or the core infrastructure vs building it out all on your own. Build vs. buy is always a common debate with larger enterprises, which is a huge task and would require even more staff and resources and huge amounts of time. I would opt for depth and breadth, lots of transparency to the infrastructure, and deep API-first approaches, but not so much that I need to manage it 100%. Adobe, Salesforce, and a few others are along this line, whereas other smaller CDP-only vendors only give you enough on the far right for a few capabilities within the CDP that would traditionally be EDM features in most cases. The biggest key is to ensure you have 100% of your historical data stored to take full advantage of your data.
Summary
Here's a summary list of what an Enterprise Data Platform may include, including the periphery or edge data apps that subscribe, interact, and consume the data from the platform. you could argue that these are independent micro-services within a larger platform, now called composable components or services, or they could be additional independent vendor tools integrated through APIs or a comprehensive monolithic single platform. The latter I do not recommend buying.
Data Integration Frameworks: The ability to seamlessly ingest structured and unstructured data, including large volumes of data, batch inputs, real-time data, and more, across various source channels.
Data Management: Encompassing everything from managing cookies and personal information to industry-specific data models, this part emphasizes adaptability, scalability, and long-term data retention.
Identity and Golden Customer Record: Ensuring customer data's cleanliness, verification, and probabilistic match, strongly focusing on maintaining a 'golden record' for individual customers and households.
Analytics: Providing tools for data exploration, predictive modeling, pre-built dashboards for various KPIs, advanced segmentation, and recommendation engines. An essential aspect for extracting actionable insights.
Reporting: Tools for customized data analysis, automated daily reports, and alert-based reports. Essential for a tailored understanding of business conditions or anomalies.
Advanced Segmentation and Predictive Modeling: Includes segmentation of customer personas, look-alike models, market basket analysis, churn prediction, and tools for micro-segmentation to complement macro-segmentation. These are key for targeted marketing strategies.
Personalization and Recommendations: Leveraging user behavior and contextual data to provide personalized content, offers, and recommendations across different channels and platforms. A vital aspect of modern customer engagement.
Customer Journey Management: Encompassing tools for journey mapping, real-time orchestration, cross-channel integration, and event-triggered actions. Essential for unified and optimized customer experiences.
Data Serving: Concentrating on timely and versatile data sharing with external systems, including real-time sharing, API-based connections, and analytical data set conversions.
Marketing Channels: A robust system for managing various marketing channels, including email, SMS, mobile app, web, and API support. The tools provided cater to modern marketing demands.
Data Security: Recognizing the critical importance of security, this section encompasses everything from data center selection to encryption, secure deletion of data, GDPR compliance, and operational security. Deep security controls, labels, published frameworks, and dedicated staffing ensure trust and compliance with legal and regulatory requirements.
By integrating these components, marketing technologists can build a sophisticated and responsive CDP that resonates with today's dynamic marketing environment. It allows a multi-faceted approach to customer engagement, personalization, segmentation, and more, all under a secure and efficient umbrella.
Data Integration Frameworks
Capability | Description |
---|---|
Structured Data Integration | Capable of absorbing data that is organized and structured, such as information stored in tables within relational databases or traditional file systems. |
Unstructured and Semi-Structured Data | Equipped to handle and process both semi-structured and unstructured data types. |
Big Data Handling | Designed to manage, store, process, and distribute substantial volumes of data, often associated with industry giants, such as those in the retail sector. |
Batch Data Processing | Offers the capability to integrate data provided through batch file inputs. |
Real-Time Data Management | Enables the ingestion, storage, processing, and sharing of data that is supplied in real-time. |
API Data Ingestion | Facilitates the absorption of data transmitted via API connections. |
Website Tagging | Employs unique site tags to capture and integrate data directly from websites. |
Mobile App Tagging via SDK | Utilizes specific tagging within mobile applications, capturing pertinent data via Software Development Kits (SDKs). |
Pre-configured Connectors | Features ready-made connectors to enable smooth data ingestion from various source systems. |
Integration of Diverse Data Sources | Highly versatile in handling a wide array of commonly encountered data, including, but not limited to, Point of Sale (POS) systems, Product Catalogs, loyalty programs, transactions, campaign details, mobile and web clickstreams, feedback, and more. |
Data Management
Capability | Description |
---|---|
Web Visitor Tracking | Utilizes cookie or ID management to monitor and identify website visitors. |
Personal Information Handling | Incorporates pre-configured connectors to assimilate personal identifiers like name, address, and phone number, associating them with customer profiles and securely storing the information. |
Compliance with PII Regulations | Ensures the handling of personal identifiers in alignment with legal mandates and organizational guidelines concerning security, privacy, and consent. |
Identification Mapping | Capable of attributing a persistent ID to an unidentified individual, household, or business, and later linking that ID with specific personal, household, or business information when available. |
In-Memory Data Storage | Employs in-memory storage for significant portions of its data, facilitating rapid retrieval and modification. |
Dynamic Resource Scaling | Automatically modifies resource utilization, such as processing power and storage, to sustain predefined service levels. |
Industry-Specific Data Models | Offers standardized data models catering to distinct industry segments in Retail, Financial, Telecom, Healthcare. |
Long-Term Data Preservation | Ensures the storage of ingested data for durations as dictated by user requirements. |
Retention of Detailed Records | Allows for the ingestion and preservation of all details conveyed from source systems. |
Complex Customer Association | Supports a multi-table data model, connecting customers with various entities like messages sent, products purchased, website sessions, customer service interactions, etc., and aligning specific details with each entity, such as time, date, location, price, duration, and more. |
Consent Management | Manages and enforces customer consent for data usage in compliance with privacy regulations. |
Department-Based Data Segregation | Facilitates data partitioning across different departments within a single warehouse, implementing access restrictions as per departmental roles. |
Identity Resolution and Golden Customer Record
Capability | Description |
---|---|
Data Cleaning & Standardization | Includes functionalities to cleanse and normalize input data, identifying and rectifying missing or incorrect information using specified rules, reference tables, and transformations. |
Profile Enrichment (Reverse Append) | Features ready-to-use options to enhance customer profiles by associating each profile with third-party data sources. |
Contact Information Verification | Possesses the ability to authenticate contact details, such as email IDs and phone numbers. |
Probabilistic Matching | Capable of analyzing strings (like names and addresses) to derive a similarity score, assisting in data linkage and identity matching. |
Golden Record Compilation | Employs rule-based algorithms to curate and distribute a 'golden record', representing the most accurate attributes for an individual's identity. |
Customer Household Grouping | Enables the storing of relationships among identifiers and the grouping of customers into households, utilizing both deterministic and probabilistic rules. |
Creation of Persistent IDs | Assigns enduring IDs to individuals, households, and other entities, ensuring that these IDs are linked to various identifiers across systems and remain consistent despite changes in other identifiers. |
Analytics
Capability | Description |
---|---|
Data Exploration Tools | Equipped with tools for comprehensive data analysis, encompassing visualization features, cross tabs, and other expert level data analysis tools. |
Manual Predictive Modeling | Offers tools for expert users to craft predictive models such as Churn, Customer Lifetime Value (CLTV), propensity, and lookalike models |
Automated Model Generation | Capable of autonomously creating predictive models for users without specialized skills. |
Pre-Built Dashboards and KPIs | Provides ready-to-use KPIs and dashboards for various analytics areas, including:
|
Specialized Dashboards | Offers out-of-the-box dashboards for specific analysis needs, such as:
|
Reporting
Capability | Description |
---|---|
Customization | Enables users to craft personalized analyses within the tool. |
Automated Reporting | Facilitates sending automated reports on a daily basis. |
Alert-Based Reporting | Supports sending reports based on specific business conditions or anomalies. |
Advanced Segmentation and Predictive Modeling
Capability | Description |
---|---|
Segmentation | Utilizes algorithms to identify customer personas based on various variables. |
Look-Alike Models | Identifies optimal target groups for campaigns, products, or events algorithmically. |
Market Basket Analysis (MBA) | Employs algorithms to recognize complementary product sets. |
Customizable and Visual RFME Framework | Categorizes customers based on Recency, Frequency, Monetary, and Engagement variables. |
Churn Prediction | Utilizes algorithms to gauge the likelihood of customer attrition. |
Propensity Analysis | Employs algorithms to predict customer responses based on specific criteria. |
Micro-Segmentation Tools | Offers various features for managing and creating microsegments, including drag-and-drop interfaces, multi-conditional list creation, transformation and filtering, and templated creation. |
Personalization and Recommendations
Capability | Description |
---|---|
Personalized Recommendations | Algorithms that tailor product or content recommendations to individual preferences and behavior. |
Personalization Engine | A tool or system that enables dynamic content, images, and offers per user type or segment. |
Multi-Channel Personalization | Ability to deliver personalized experiences across different customer touchpoints such as web, email, mobile apps, etc. |
Real-Time Personalization | Ability to adapt the content, images, and offers in real-time based on the individual’s behavior and interactions. |
Behavioral Targeting | Utilizing user behavior data to enhance personalization strategies and content delivery. |
Contextual Personalization | Personalizing content or offers based on the current context, such as location, weather, or time of day. |
Customer Journey Personalization | Adapting and tailoring the customer experience based on the individual’s journey stages and previous interactions with the brand. |
Customer Journey Management
Capability | Description |
---|---|
Journey Mapping | Ability to visualize and map customer journeys across touchpoints. |
Journey Analytics | Analyze customer journeys for insights and optimization. |
Real-Time Orchestration | Deliver real-time experiences that adapt as the journey unfolds. |
Cross-Channel Integration | Ensure that customer journeys are integrated across all channels for a unified experience. |
Journey Optimization | Tools for continually optimizing and improving the customer journey. |
Event-Triggered Actions | Executing actions based on specific customer behaviors or events along the journey. |
Data Serving
Capability | Description |
---|---|
Time-to-Value | Implementing a modern data platform or CDP within a large enterprise can vary in time, from weeks for simple tasks like website tags to months for complex data transformations. Success is attainable by breaking the process into small, manageable projects, guided by solid project management. While delivering insights into other systems might be a slow process, a methodical and persistent approach will lead to effective results. |
Data Extraction Methods |
|
Marketing Channels
Capability | Description |
---|---|
Email Marketing |
|
SMS Marketing |
|
Mobile App Marketing |
|
Web Marketing |
|
API Support for Marketing |
|
Data Security
Capability | Description |
---|---|
Data Center Selection | Compliance: Data centers hosting the Customer Data Platform (CDP) must meet stringent standards including: CSA Cloud Security Alliance Controls, ISO 9001, ISO 27001, ISO 27018, SOC1, SOC2, and PCI DSS Level 1 at minimum. |
Data Residency | Compliance: The location of data centers must adhere to jurisdictional data residency regulations and guidelines. |
Network Security | Protocols: Must implement perimeter firewalls and robust encryption measures, with stringent authentication and transmission security settings. |
Security Event Monitoring | Integration: Should interface with Security Information and Event Management (SIEM) tools for comprehensive oversight. |
Encryption & Key Management | Standards: Must employ encryption for data protection in storage and transmission according to applicable laws and regulations. Keys should be maintained securely. |
Data Ingestion | Transfer Protocols: Must support secure and encrypted data transfer methods such as IPSEC VPN tunnels and SSL over REST protocol. |
Data at Rest | Encryption: Must use hardware and key-based encryption consistently in accordance with data classification. |
Secure Deletion of Data | Protocols: Must allow secure disposal of data following standards such as NIST SINGLE PASS, DOD 3, or DOD 7 Pass. |
Data Inventory and Lineage | Tracking: Should support comprehensive reporting and tracking of data origin, flows, and residency within the CDP. |
Identity & Access Management (IAM) | Compliance: Must support strong multifactor authentication, non-sharable secrets, and stringent audit tools in line with industry and regulatory standards. |
Data Security and Integrations | Standards: Must ensure secure network protocols and adhere to industry standards for applications and interfaces, such as OWASP for web applications. |
Security Audit | Reporting: Should be subject to independent auditing and periodic reporting of processes. |
Applications and Service Security | • Assessments: Must conduct vulnerability assessments on tools and services, with available reports. • Malware Prevention: Should implement measures to prevent malware and virus attacks within the CDP. |
Operational Security | Confidentiality: Must execute agreements to protect data and operational details. |
GDPR Compliance | Privacy by Design: Should incorporate privacy as a core principle across all operations and provide controls for data stewards to manage opt-ins, data erasure, and the right to be forgotten. |
Member discussion