Customer data in most organizations is fragmented across systems that were never designed to talk to each other. Marketing has email engagement data in Mailchimp, website behavior in Google Analytics, and ad performance in Meta and Google Ads. Sales has pipeline data in Salesforce and conversation history in Gong. Support has ticket data in Zendesk and satisfaction scores in Delighted. Product has usage data in Amplitude or Mixpanel. Finance has billing data in Stripe and revenue records in the ERP.
Each system holds a partial view of the customer. None holds the complete picture. A customer data platform unifies these fragments into a single, authoritative profile for each customer, enabling the kind of personalized, consistent experience that customers increasingly expect and that drives measurable business outcomes.
What a CDP Actually Does
A customer data platform serves four core functions: data collection, identity resolution, profile unification, and activation. Understanding each function prevents the common mistake of building (or buying) more or less than you need.
Data collection ingests customer interaction data from every touchpoint. Website visits, email opens, purchase transactions, support tickets, app usage events, form submissions, and ad clicks all flow into the CDP. The collection layer must handle both real-time event streams (a user clicking a button right now) and batch imports (last month’s transaction records from the ERP).
For real-time collection, the CDP exposes an event tracking API that client-side SDKs, server-side applications, and third-party webhooks can send events to. The standard event format includes: a user identifier (or anonymous ID), an event name, a timestamp, and a properties object containing event-specific data. The Segment tracking API specification has become a de facto industry standard for this format, and designing your custom CDP’s collection API to follow this convention makes integration with existing tooling easier.
For batch collection, the CDP ingests data from source systems on a scheduled cadence. CSV exports, database replications, API polling, and warehouse-to-CDP syncs (reverse ETL) all feed data into the platform. The batch layer must handle deduplication, schema evolution, and late-arriving data gracefully.
Identity resolution is the most technically challenging function. A single person might interact with your business through multiple devices, email addresses, phone numbers, and account IDs. The anonymous visitor who browsed your website last week, the lead who signed up for a webinar yesterday, and the customer who purchased today might all be the same person. Identity resolution stitches these fragmented identities into a single unified profile.
Deterministic matching links identities through known shared attributes: same email address, same phone number, same account ID. When a user logs in on their phone with the same email they used on desktop, the two anonymous profiles merge into a single authenticated profile. Probabilistic matching uses behavioral and contextual signals (same IP address, similar device fingerprint, overlapping browsing patterns) to suggest likely matches when deterministic identifiers are not available. Probabilistic matching is useful but must be tuned carefully to avoid false positives that merge distinct people into a single profile.
Profile unification takes the resolved identity and constructs a comprehensive profile by merging data from all sources. The unified profile includes: demographic attributes (name, email, location), behavioral history (every tracked event), computed traits (total lifetime value, product usage tier, engagement score), and segment memberships (high-value customers, at-risk accounts, product-qualified leads).
Activation makes unified profiles available to downstream systems. Marketing automation platforms receive enriched profiles to power personalized campaigns. Sales tools receive buying signals to prioritize outreach. Support platforms receive customer context to reduce resolution time. Analytics tools receive unified data to power accurate reporting. Activation happens through real-time API access, webhook-based event forwarding, and scheduled syncs to data warehouses and marketing platforms.
Data Architecture for a Custom CDP
The technical architecture of a CDP reflects the dual nature of the data: high-frequency events that arrive in real time and profile data that evolves through aggregation and computation.
The event ingestion layer uses an API gateway (AWS API Gateway, Kong, or a lightweight custom endpoint) that validates incoming events, enriches them with server-side context (IP geolocation, user agent parsing), and publishes them to a message stream. Apache Kafka is the standard choice for the message backbone due to its durability, replay capability, and ecosystem of stream processing tools. For smaller-scale deployments, Amazon Kinesis or Redis Streams provide similar functionality with less operational overhead.
Stream processing consumes events from the message backbone and performs real-time operations: identity resolution updates, real-time trait computation, segment membership evaluation, and event forwarding to activated destinations. Apache Flink provides stateful stream processing with exactly-once semantics, which is important for accurate counting and deduplication. For less complex processing needs, Kafka Streams or AWS Lambda consumers work well.
The profile store is the system of record for unified customer profiles. This store must support fast point lookups by customer ID (to serve API requests and power real-time personalization), efficient batch scans (to compute segments and generate exports), and flexible schema evolution (because the set of tracked events and computed traits changes as the business evolves).
PostgreSQL with JSONB columns provides a pragmatic starting point. The core profile attributes (ID, email, name) live in typed columns for efficient querying, while the full event properties and computed traits live in JSONB for flexibility. As the dataset grows beyond what PostgreSQL handles comfortably, Apache Cassandra or ScyllaDB provide horizontal scalability for the profile store, while the event archive moves to a columnar store like ClickHouse or a data lakehouse on S3 with Apache Iceberg.
A data warehouse (Snowflake, BigQuery, or a self-hosted ClickHouse) serves as the analytical layer. Unified profiles and events are synced to the warehouse for complex analysis, segmentation, and reporting that the operational profile store is not designed to handle efficiently. Marketing analysts, data scientists, and business intelligence tools query the warehouse, not the operational store.
Identity Resolution Implementation
Identity resolution deserves detailed treatment because it is the CDP function most likely to produce incorrect results if implemented naively.
Maintain an identity graph where nodes represent known identifiers (email addresses, phone numbers, device IDs, account IDs, anonymous cookie IDs) and edges represent observed associations between identifiers. When a user logs in with email [email protected] on a device with anonymous ID abc-123, an edge is created between those two identifiers.
Transitivity is where identity resolution gets complex. If anonymous ID abc-123 is linked to [email protected], and [email protected] is linked to phone number 555-0100, then abc-123, [email protected], and 555-0100 all belong to the same identity cluster. This is correct. But if 555-0100 was previously associated with [email protected] (perhaps Alice and Bob share a phone number in a family account), the transitive closure would merge Alice and Bob into a single profile, which is incorrect.
Guard against over-merging with rules. Certain identifier types (like shared IP addresses, company phone numbers, or generic email addresses like info@) should not create identity edges. Set maximum cluster sizes; if a merge would create a profile with more than a threshold number of distinct email addresses (say, 5), flag it for review rather than automatically merging. Implement unmerge capabilities so that incorrectly merged profiles can be separated.
Store the identity graph in a graph database (Neo4j, Amazon Neptune) or as an adjacency list in PostgreSQL. Run connected-component analysis to identify identity clusters. When new events arrive with identifier pairs, update the graph and recompute affected clusters incrementally rather than re-running the full analysis.
Privacy Compliance and Consent Management
A CDP concentrates customer data from across the organization, which makes it a focal point for privacy compliance. GDPR, CCPA, and other privacy regulations impose specific obligations on how customer data is collected, stored, processed, and deleted.
Consent management must be integrated into the CDP’s data collection layer. Before collecting data from a user, verify that appropriate consent has been obtained. This typically involves checking the user’s consent record (stored in a consent management platform like OneTrust, Usercentrics, or a custom consent store) and filtering incoming events based on the consent categories the user has accepted.
Data subject access requests require the CDP to export all data associated with a specific individual. Because the CDP holds unified profiles, it is the natural system to fulfill these requests. Build an administrative interface that lets privacy officers look up a customer by any known identifier, see the complete unified profile, and export it in a machine-readable format.
Right to deletion requests require the CDP to delete all data associated with a specific individual and propagate that deletion to downstream systems that received the data via activation. This is architecturally non-trivial. The CDP must maintain a record of which downstream systems received which customer’s data, send deletion requests to those systems, and confirm completion. Build deletion as a first-class workflow, not an afterthought, because retrofitting it into a system that was not designed for it is painful and error-prone.
Data retention policies should be configurable by data type and purpose. Behavioral event data might be retained for 24 months for analytics purposes, while profile attributes are retained for the duration of the customer relationship. Automated retention enforcement ensures that data is purged on schedule without manual intervention.
Measuring CDP Value
A CDP is a significant engineering investment, and it must demonstrate measurable business value to justify that investment.
Track marketing efficiency metrics: email open rates and click rates for personalized versus generic campaigns, conversion rates for segmented versus untargeted audiences, and customer acquisition cost reduction attributable to better targeting. A CDP that enables marketing to shift from batch-and-blast emails to behaviorally triggered campaigns typically shows 2-3x improvement in engagement metrics.
Track customer experience metrics: support ticket resolution time (with versus without customer context from the CDP), cross-sell and upsell conversion rates (with versus without propensity signals), and churn reduction from proactive retention campaigns powered by at-risk scoring.
Track operational efficiency: time saved by analysts who no longer need to manually join data from five systems, reduction in duplicate customer records, and improvement in data quality metrics (completeness, consistency, accuracy).
Building a customer data platform is a significant undertaking that, done right, becomes the intelligence backbone of your customer operations. If you are considering unifying your customer data and want to evaluate whether a custom CDP or a commercial platform is the right fit, contact us. We can help you assess your data landscape and design an architecture that scales with your business.