15 min read

A CDP Checklist - NO!

An all-encompassing look at Enterprise Data Management for Marketing Technologists that need a clear context for the components required to support robust marketing operations in a multi-dept, multi-faceted, and large enterprise.

Abstract

This article is an all-encompassing look at Enterprise Data Management for Marketing Technologists that need a clear context for the components required to support robust marketing operations in a multi-dept, multi-faceted, and large enterprise. UPDATE: The EDM supports and feeds data to the CDP. The components below should, in many cases, be separated from the CDP. While some features below in the checklist are located in the CDP, many are more predominant as part of an EDM platform. Many vendors are abstracting their UI/UX and mixing metaphors of data management. While that is not wrong, it does cause more complexity for the marketer to understand specific data roles across the workflow. My argument premise is that the CDP provides a more narrow role. It does not house Identity resolution, dataset management, or even other more granular capabilities like AI/ML, Query, or Profile Management tools. My argument is it should supply the enforcement options for the profiles, what destinations they are subscribed to or assigned, and macro-audience management to coordinate omnichannel subscription and movement into the delivery tools that provide the assigned profile with the right content from broad-based to 1:1 narrow-based campaign initiatives. It's a 4-layer, perhaps even a 5-layer architecture from source to destination, horizontally, then it's 5-6 or more layers vertically at each bookend for the sources and the delivery destinations.

Each layer is a discreet capability and is standalone. The farther right represents a data app with a specific function/role.

In an era driven by data and customer personalization, Marketing Technologists must comprehensively understand the tools and techniques that drive success. Below is what a marketing dept and marketing technologists need to look at to support their enterprise data management strategy, data strategy, and overall Martech stack. The analysis includes components like Customer Data Platforms (CDP) and downstream data apps like Campaign Management, Personalization, and Journey Management. Note that these apps are not an Enterprise Data Platform - they are mostly subscriber components of the data platform.

Many vendors will say their CDP is an Enterprise Data Management Platform, but this is far from the truth. There are only a few vendors in this genre - you will hear Reverse ETL, ELT, Zero-copy, Lifetime CDP, Universal Data Hub, et al...and many other confusing terms that cloud(pun haha) the reality of a real Enterprise Data Platform and strategy and what you must have and need to support your marketers.

Additionally, some companies(primarily pundits, CEO/CTOs of vendors, and consulting firms) must clarify what a CDP is and NOT blend it into an Enterprise Data Platform. A CDP is a small data app at the edge - it should always be distinct from your company's enterprise and Marketing Data Platform. A CDP is a component that complements an EDM, not replaces it, as many of these pundits allude to on their sites and collateral.

Adobe is one vendor that provides a multi-faceted, multi-source data platform like a Data Lake and Lake House without buying a CDP component, but they do have a CDP. Adobe makes the data platform an abstraction of many other technologies, so your company or a marketing technologist team doesn't have to go through all the processes of multi-vendor selection. Salesforce comes close, but they still need some primary capability today and are working on catching up. They have hired several people with these skill sets lately. There are a few others in this genre and many that are not. Buyer beware.

The technology abstractions the more prominent vendors are creating are freeing up massive implementation time. You still have to implement it's just a shorter time frame, and you have to commit deeply to the process but in bite-size steps to succeed. You could build a modern data platform yourself, as Adobe has done. For example, imagine having to hire staff to develop and provision all of the tools to create a self-hosted or your private cloud with a mix of solutions like Databricks or Snowflake, along with the periphery tools like Kafka, Airflow, dBT, Fivetran, and dozens more. I have the list if you are interested. You could still do this, and enterprises do, but there are benefits for the marketing department and the dedicated technologist assigned to them not to be in that business. It is faster for them to use these vendor-level abstractions, and then tie that system or platform of tools into a more significant transformation initiative within the enterprise. It's only an opinion, but it seems easier based on skillsets out there right now, time-to-market, and overall cost.

Do your due diligence on all vendors and map your requirements to what they can provide. The list below should give an excellent guide to what you need to ask and look for in your selection process. Many vendors offer all or some of these capabilities. Adobe and Salesforce have deep abstractions for them, Algonomy(a niche player for retailers and QSR-quick-serve restaurants), Amperity(deep identity resolution and analytics tools but lacks in other areas), Amplitude( approaching the Adobe/Salesforce level), and Segment and Tealium(mainly orchestration tools). There are dozens if not hundreds more(Acquia, RedPoint, TreasureData, Blueconic, SimonData et al...) - be careful and thoughtful based on need.

What does this ultimately look like? It could look like this, and much of the infrastructure to the LEFT of the Data apps could be abstracted to no UI screens or APIs, not seen, or touched at all by the user, or it could be completely exposed for more control by the client owner(you) with deep UI/UX and robust APIs. Both approaches are neither right nor wrong – depending on how much control the client(you) wants. The more control they want, the more staff and processes they must incorporate. Regardless of which approach per vendor you choose, you have opted to have them help you with managing devops or the core infrastructure vs building it out all on your own. Build vs. buy is always a common debate with larger enterprises, which is a huge task and would require even more staff and resources and huge amounts of time. I would opt for depth and breadth, lots of transparency to the infrastructure, and deep API-first approaches, but not so much that I need to manage it 100%. Adobe, Salesforce, and a few others are along this line, whereas other smaller CDP-only vendors only give you enough on the far right for a few capabilities within the CDP that would traditionally be EDM features in most cases. The biggest key is to ensure you have 100% of your historical data stored to take full advantage of your data.

The CORE blue box components can be exposed, controlled, hidden by UI, or not seen at all. It's vendor-specific and runs the gamut across vendors on what level of abstraction they give the client in the UI and in APIs. I opt for more control and transparency vs. less.

Summary

Here's a summary list of what an Enterprise Data Platform may include, including the periphery or edge data apps that subscribe, interact, and consume the data from the platform. you could argue that these are independent micro-services within a larger platform, now called composable components or services, or they could be additional independent vendor tools integrated through APIs or a comprehensive monolithic single platform. The latter I do not recommend buying.

Data Integration Frameworks: The ability to seamlessly ingest structured and unstructured data, including large volumes of data, batch inputs, real-time data, and more, across various source channels.

Data Management: Encompassing everything from managing cookies and personal information to industry-specific data models, this part emphasizes adaptability, scalability, and long-term data retention.

Identity and Golden Customer Record: Ensuring customer data's cleanliness, verification, and probabilistic match, strongly focusing on maintaining a 'golden record' for individual customers and households.

Analytics: Providing tools for data exploration, predictive modeling, pre-built dashboards for various KPIs, advanced segmentation, and recommendation engines. An essential aspect for extracting actionable insights.

Reporting: Tools for customized data analysis, automated daily reports, and alert-based reports. Essential for a tailored understanding of business conditions or anomalies.

Advanced Segmentation and Predictive Modeling: Includes segmentation of customer personas, look-alike models, market basket analysis, churn prediction, and tools for micro-segmentation to complement macro-segmentation. These are key for targeted marketing strategies.

Personalization and Recommendations: Leveraging user behavior and contextual data to provide personalized content, offers, and recommendations across different channels and platforms. A vital aspect of modern customer engagement.

Customer Journey Management: Encompassing tools for journey mapping, real-time orchestration, cross-channel integration, and event-triggered actions. Essential for unified and optimized customer experiences.

Data Serving: Concentrating on timely and versatile data sharing with external systems, including real-time sharing, API-based connections, and analytical data set conversions.

Marketing Channels: A robust system for managing various marketing channels, including email, SMS, mobile app, web, and API support. The tools provided cater to modern marketing demands.

Data Security: Recognizing the critical importance of security, this section encompasses everything from data center selection to encryption, secure deletion of data, GDPR compliance, and operational security. Deep security controls, labels, published frameworks, and dedicated staffing ensure trust and compliance with legal and regulatory requirements.

By integrating these components, marketing technologists can build a sophisticated and responsive CDP that resonates with today's dynamic marketing environment. It allows a multi-faceted approach to customer engagement, personalization, segmentation, and more, all under a secure and efficient umbrella.

Data Integration Frameworks

Capability Description
Structured Data Integration Capable of absorbing data that is organized and structured, such as information stored in tables within relational databases or traditional file systems.
Unstructured and Semi-Structured Data Equipped to handle and process both semi-structured and unstructured data types.
Big Data Handling Designed to manage, store, process, and distribute substantial volumes of data, often associated with industry giants, such as those in the retail sector.
Batch Data Processing Offers the capability to integrate data provided through batch file inputs.
Real-Time Data Management Enables the ingestion, storage, processing, and sharing of data that is supplied in real-time.
API Data Ingestion Facilitates the absorption of data transmitted via API connections.
Website Tagging Employs unique site tags to capture and integrate data directly from websites.
Mobile App Tagging via SDK Utilizes specific tagging within mobile applications, capturing pertinent data via Software Development Kits (SDKs).
Pre-configured Connectors Features ready-made connectors to enable smooth data ingestion from various source systems.
Integration of Diverse Data Sources Highly versatile in handling a wide array of commonly encountered data, including, but not limited to, Point of Sale (POS) systems, Product Catalogs, loyalty programs, transactions, campaign details, mobile and web clickstreams, feedback, and more.

Data Management

Capability Description
Web Visitor Tracking Utilizes cookie or ID management to monitor and identify website visitors.
Personal Information Handling Incorporates pre-configured connectors to assimilate personal identifiers like name, address, and phone number, associating them with customer profiles and securely storing the information.
Compliance with PII Regulations Ensures the handling of personal identifiers in alignment with legal mandates and organizational guidelines concerning security, privacy, and consent.
Identification Mapping Capable of attributing a persistent ID to an unidentified individual, household, or business, and later linking that ID with specific personal, household, or business information when available.
In-Memory Data Storage Employs in-memory storage for significant portions of its data, facilitating rapid retrieval and modification.
Dynamic Resource Scaling Automatically modifies resource utilization, such as processing power and storage, to sustain predefined service levels.
Industry-Specific Data Models Offers standardized data models catering to distinct industry segments in Retail, Financial, Telecom, Healthcare.
Long-Term Data Preservation Ensures the storage of ingested data for durations as dictated by user requirements.
Retention of Detailed Records Allows for the ingestion and preservation of all details conveyed from source systems.
Complex Customer Association Supports a multi-table data model, connecting customers with various entities like messages sent, products purchased, website sessions, customer service interactions, etc., and aligning specific details with each entity, such as time, date, location, price, duration, and more.
Consent Management Manages and enforces customer consent for data usage in compliance with privacy regulations.
Department-Based Data Segregation Facilitates data partitioning across different departments within a single warehouse, implementing access restrictions as per departmental roles.

Identity Resolution and Golden Customer Record

Capability Description
Data Cleaning & Standardization Includes functionalities to cleanse and normalize input data, identifying and rectifying missing or incorrect information using specified rules, reference tables, and transformations.
Profile Enrichment (Reverse Append) Features ready-to-use options to enhance customer profiles by associating each profile with third-party data sources.
Contact Information Verification Possesses the ability to authenticate contact details, such as email IDs and phone numbers.
Probabilistic Matching Capable of analyzing strings (like names and addresses) to derive a similarity score, assisting in data linkage and identity matching.
Golden Record Compilation Employs rule-based algorithms to curate and distribute a 'golden record', representing the most accurate attributes for an individual's identity.
Customer Household Grouping Enables the storing of relationships among identifiers and the grouping of customers into households, utilizing both deterministic and probabilistic rules.
Creation of Persistent IDs Assigns enduring IDs to individuals, households, and other entities, ensuring that these IDs are linked to various identifiers across systems and remain consistent despite changes in other identifiers.

Analytics

Capability Description
Data Exploration Tools Equipped with tools for comprehensive data analysis, encompassing visualization features, cross tabs, and other expert level data analysis tools.
Manual Predictive Modeling Offers tools for expert users to craft predictive models such as Churn, Customer Lifetime Value (CLTV), propensity, and lookalike models
Automated Model Generation Capable of autonomously creating predictive models for users without specialized skills.
Pre-Built Dashboards and KPIs Provides ready-to-use KPIs and dashboards for various analytics areas, including:
  • Sales and Business Overview
  • Loyalty Program Analysis
  • Customer Feedback Analysis
  • Customer Spend Analysis
  • Digital Engagement Trends
  • Promotion and Offer Analytics
  • Demographic Trends
  • Product Purchasing Analysis
Specialized Dashboards Offers out-of-the-box dashboards for specific analysis needs, such as:
  • Root Cause Analysis (RCA) Dashboards
  • Campaign and Promotion Dashboards
  • Sales Growth Analysis
  • Member vs Non-member Analysis

Reporting

Capability Description
Customization Enables users to craft personalized analyses within the tool.
Automated Reporting Facilitates sending automated reports on a daily basis.
Alert-Based Reporting Supports sending reports based on specific business conditions or anomalies.

Advanced Segmentation and Predictive Modeling

Capability Description
Segmentation Utilizes algorithms to identify customer personas based on various variables.
Look-Alike Models Identifies optimal target groups for campaigns, products, or events algorithmically.
Market Basket Analysis (MBA) Employs algorithms to recognize complementary product sets.
Customizable and Visual RFME Framework Categorizes customers based on Recency, Frequency, Monetary, and Engagement variables.
Churn Prediction Utilizes algorithms to gauge the likelihood of customer attrition.
Propensity Analysis Employs algorithms to predict customer responses based on specific criteria.
Micro-Segmentation Tools Offers various features for managing and creating microsegments, including drag-and-drop interfaces, multi-conditional list creation, transformation and filtering, and templated creation.
The table above could have much more included, like Attribution or Gen-AI. This area is always evolving.

Personalization and Recommendations

Capability Description
Personalized Recommendations Algorithms that tailor product or content recommendations to individual preferences and behavior.
Personalization Engine A tool or system that enables dynamic content, images, and offers per user type or segment.
Multi-Channel Personalization Ability to deliver personalized experiences across different customer touchpoints such as web, email, mobile apps, etc.
Real-Time Personalization Ability to adapt the content, images, and offers in real-time based on the individual’s behavior and interactions.
Behavioral Targeting Utilizing user behavior data to enhance personalization strategies and content delivery.
Contextual Personalization Personalizing content or offers based on the current context, such as location, weather, or time of day.
Customer Journey Personalization Adapting and tailoring the customer experience based on the individual’s journey stages and previous interactions with the brand.

Customer Journey Management

Capability Description
Journey Mapping Ability to visualize and map customer journeys across touchpoints.
Journey Analytics Analyze customer journeys for insights and optimization.
Real-Time Orchestration Deliver real-time experiences that adapt as the journey unfolds.
Cross-Channel Integration Ensure that customer journeys are integrated across all channels for a unified experience.
Journey Optimization Tools for continually optimizing and improving the customer journey.
Event-Triggered Actions Executing actions based on specific customer behaviors or events along the journey.

Data Serving

Capability Description
Time-to-Value Implementing a modern data platform or CDP within a large enterprise can vary in time, from weeks for simple tasks like website tags to months for complex data transformations. Success is attainable by breaking the process into small, manageable projects, guided by solid project management. While delivering insights into other systems might be a slow process, a methodical and persistent approach will lead to effective results.
Data Extraction Methods
  • Batch Extraction: Permits data sharing with external systems by conducting batch extraction processes to generate files.
  • API-Based Sharing: Facilitates data sharing with external systems through API or API-based connector access, enhancing integration capabilities.
  • Real-Time Sharing: Enables data sharing with external systems in real-time, allowing for queries regarding specific customer information.
  • Analytical Data Set Conversion: Provides the ability to share data with external systems by transforming it into formats needed for analytical platforms, such as AVRO or PARQUETdata sets or 'flattened' comma-separated files.

Marketing Channels

Capability Description
Email Marketing
  • Spam Testing: Features tools for ensuring email deliverability by conducting spam tests.
  • Template Support: Offers access to a library of email templates and supports HTML rendering across various devices for consistent viewing experiences.
SMS Marketing
  • Local Telco Integration: Comes with pre-integration support for local telecommunications companies.
  • Third-Party Gateway Support: Possesses the capability to work with third-party SMS gateways to extend reach and functionality.
Mobile App Marketing
  • Rich Push Content: Allows to send rich push content, including images, GIFs, videos, and emojis.
  • Real-Time In-App Banners: Supports real-time, engaging in-app banners for interactive user experiences.
Web Marketing
  • Customer Interaction-Based Pop-Ups: Facilitates the display of pop-ups on the website tailored to individual customer interactions.
  • Embedded Banners: Incorporates embedded banners on webpages that can showcase algorithmically-generated or static offers.
API Support for Marketing
  • Third-Party Integration: Provides API support to transmit customer information to external data sources such as kiosks, call centers, etc., enabling a seamless connection across marketing channels.

Data Security

Capability Description
Data Center Selection Compliance: Data centers hosting the Customer Data Platform (CDP) must meet stringent standards including: CSA Cloud Security Alliance Controls, ISO 9001, ISO 27001, ISO 27018, SOC1, SOC2, and PCI DSS Level 1 at minimum.
Data Residency Compliance: The location of data centers must adhere to jurisdictional data residency regulations and guidelines.
Network Security Protocols: Must implement perimeter firewalls and robust encryption measures, with stringent authentication and transmission security settings.
Security Event Monitoring Integration: Should interface with Security Information and Event Management (SIEM) tools for comprehensive oversight.
Encryption & Key Management Standards: Must employ encryption for data protection in storage and transmission according to applicable laws and regulations. Keys should be maintained securely.
Data Ingestion Transfer Protocols: Must support secure and encrypted data transfer methods such as IPSEC VPN tunnels and SSL over REST protocol.
Data at Rest Encryption: Must use hardware and key-based encryption consistently in accordance with data classification.
Secure Deletion of Data Protocols: Must allow secure disposal of data following standards such as NIST SINGLE PASS, DOD 3, or DOD 7 Pass.
Data Inventory and Lineage Tracking: Should support comprehensive reporting and tracking of data origin, flows, and residency within the CDP.
Identity & Access Management (IAM) Compliance: Must support strong multifactor authentication, non-sharable secrets, and stringent audit tools in line with industry and regulatory standards.
Data Security and Integrations Standards: Must ensure secure network protocols and adhere to industry standards for applications and interfaces, such as OWASP for web applications.
Security Audit Reporting: Should be subject to independent auditing and periodic reporting of processes.
Applications and Service Security • Assessments: Must conduct vulnerability assessments on tools and services, with available reports.
• Malware Prevention: Should implement measures to prevent malware and virus attacks within the CDP.
Operational Security Confidentiality: Must execute agreements to protect data and operational details.
GDPR Compliance Privacy by Design: Should incorporate privacy as a core principle across all operations and provide controls for data stewards to manage opt-ins, data erasure, and the right to be forgotten.