Why does adding a second in-app tool usually create a duplicate data pipeline?

Because the fastest way to get a new tool functioning is to point its SDK at the same raw event sources the existing CEP already reads from, since those events are already instrumented and require no new engineering work. This produces two independent systems computing segmentation from separately ingested copies of the same underlying events, on different schedules with different logic, and the two computations frequently disagree because nothing forces them to converge. The fix is architectural: route through a shared source of truth rather than letting each new tool build its own independent copy of the same signal.

What is the difference between the three integration patterns for adding an in-app layer?

Direct SDK instrumentation wires the new tool to the same raw events the CEP receives, producing the lowest latency but the highest data volume and weakest governance profile, since the vendor independently ingests a full copy of the event stream. CDP-forwarded events route through a customer data platform that standardises and forwards a governed copy to each destination, offering moderate latency and stronger governance since data minimisation can be enforced once at the CDP layer. CEP-forwarded segments send only computed segment membership, a pseudonymous identifier plus a label, which has the lowest data volume and strongest governance of the three, at the cost of the in-app layer only being as fresh as the CEP's own segment computation cycle.

What does zero PII movement actually mean in practice?

It means the data that moves between systems is a pseudonymous identifier and a segment label, not a user's name, email, or other identifying attribute. Pseudonymised data is still classified as personal data under GDPR if the key that reverses the pseudonymisation exists anywhere, but it is a categorically smaller and safer payload than raw attributes, because a vendor holding only a pseudonymous token cannot independently re-identify the user even if their own systems were compromised, since the reversal key never leaves the system that issued it.

What is a security review actually objecting to when it raises a PII concern about a new vendor?

Usually one of three distinct things: whether the vendor processes personal data at all, even transiently, whether the vendor stores personal data persistently after a transaction completes, and whether the vendor can independently re-identify a user without needing anything from the original system. A well-designed segment-forwarding architecture keeps the vendor firmly in the first category and largely out of the third, and naming this distinction explicitly in exactly these terms is frequently what moves a stalled security review forward, since it replaces a vague objection with a specific, answerable question.

In-App Layer Integration Without Duplicate Data Pipelines

TL;DR

The technical objection that kills these deals is never about features. It is a security review asking why a second vendor now holds user identifiers.
The honest answer is that it doesn't have to, but only if the integration is designed for that from the start, and most are not designed that way.
The default path of wiring a second tool to the same event sources produces two systems computing overlapping state from divergent copies.
This article covers why the duplicate pipeline happens by default, and the three integration patterns available with their data volume, latency, and governance profiles.
It covers how to design for zero PII movement using pseudonymous identifiers and segment membership.
It covers where the PII boundary actually sits and which part of it a security review is really objecting to.
It covers treating a CEP's segment output as a stable contract rather than raw data, and what happens when two systems disagree about a user's segment mid-session.
It covers the security review checklist to have ready before the review starts, and the instrumentation hygiene that prevents a second tool from becoming a second source of truth.

The conversation that kills an in-app layer evaluation rarely happens in the product demo. It happens three weeks later, when security asks why a new vendor needs a copy of the user table, and the growth team, who was only trying to make campaigns look native instead of like a templated overlay, does not have a good answer ready. The frustrating part is that the honest answer, in a well-designed integration, is that the vendor does not need a copy of the user table at all. It needs to know which bucket a pseudonymous identifier currently belongs to. That is a categorically smaller and safer request than what most integrations end up asking for, and the gap between the two exists because of how the integration was built, not because of any inherent requirement of the category.

Why the Duplicate Pipeline Happens by Default

The default path is almost always the same, and it is nobody's deliberate decision. A team adopts a second in-app tool. The tool needs event data to trigger and target correctly. The fastest way to get it that data is to point the new SDK at the same event sources the existing CEP already reads from, transaction completed, screen viewed, feature used, because those events already exist, are already instrumented, and require no new engineering work to wire up.

This produces two systems computing overlapping state from divergent copies of the same underlying signal. The CEP has its own segmentation engine, evaluating the same event stream on its own schedule, with its own logic for how a user moves between lifecycle stages. The new in-app tool, receiving the same raw events independently, now runs a second, parallel segmentation computation, using whatever targeting logic the growth team configured in the new tool's own interface. Both systems believe they know which segment a specific user currently belongs to. They frequently do not agree, because they are computing that answer from independently ingested copies of the same events, on different schedules, using different logic, with no mechanism forcing the two computations to converge.

The core discipline that prevents this is architectural, not procedural: move to a modular, source-of-truth-first integration architecture that minimises data movement and enforces event schema and governance early, rather than allowing each new tool to build its own independent copy of the same underlying signal. The fix is not asking teams to be more careful about keeping two systems in sync manually. It is removing the second copy entirely, which is what the three integration patterns below are actually choosing between.

The Three Integration Patterns

Direct SDK instrumentation. The in-app tool's SDK is wired directly to the app's own event sources, receiving the same raw events the CEP also receives, independently. This pattern has the lowest latency, since the in-app tool computes its own targeting decisions from live events with no intermediary hop, and the highest data volume, since every event is sent to two independent systems in full. Its governance profile is the weakest of the three, because the in-app tool now independently ingests and stores the complete raw event stream, including whatever identifiers and properties travel with those events, which means the vendor's data footprint is a full copy of the app's own instrumentation, not a filtered or reduced view of it.

CDP-forwarded events. A customer data platform sits between the app's raw event sources and every downstream tool, ingesting events once and forwarding a governed, schema-validated copy to each destination that needs it. A CDP simplifies capturing events from apps and services, standardising them, and reliably routing them to every destination that needs them, which dramatically simplifies instrumentation and makes it easy to add or remove downstream tools without rewriting tracking code each time. This pattern has moderate latency, since events pass through the CDP's own processing layer before reaching the in-app tool, and the CDP becomes the single point where event schema and data minimisation rules are enforced once, rather than independently by every downstream vendor. The governance profile is stronger than direct instrumentation specifically because the CDP layer is the natural place to strip or pseudonymise identifiers before forwarding, rather than relying on each destination vendor to apply its own filtering correctly.

CEP-forwarded segments. Rather than forwarding raw events at all, the existing CEP computes segmentation once, using its own event data and its own targeting logic, and forwards only the resulting segment membership, a pseudonymous identifier plus a label, to the in-app layer. This pattern has the lowest data volume of the three by a significant margin, since a segment membership update is a tiny fraction of the size of the raw event stream that produced it, and the strongest governance profile, because the in-app tool never receives raw events, user attributes, or anything beyond a segment label attached to an identifier it cannot independently resolve to a person. The tradeoff is that the in-app tool's targeting is only as fresh as the CEP's own segment computation cycle, and it cannot build entirely novel segmentation logic the CEP was never asked to compute, which is the specific capability tradeoff covered in the next section.

Designing for Zero PII Movement

Zero PII movement does not mean zero data movement. It means the data that moves is a pseudonymous identifier and a segment label, not a user's name, email, phone number, or any attribute that identifies them without a separate lookup.

Pseudonymised data is data where direct identifiers have been replaced with reversible tokens, and it is still classified as personal data under GDPR if the key that reverses the pseudonymisation exists anywhere. Anonymous data is data altered such that identification is no longer reasonably possible, and this is generally outside GDPR's scope, though the bar for genuine anonymisation is high and contextual. This distinction is the entire architecture in miniature. An integration that passes a pseudonymous identifier, a token that means nothing outside the system that issued it, plus a segment label, is passing personal data in the strict legal sense, since the token is reversible by the party holding the key. It is passing a categorically smaller and safer version of personal data than an integration that passes email addresses, names, or behavioural attributes directly, because the in-app vendor holding only the pseudonymous token cannot independently re-identify the user even if their systems were compromised, since the reversal key never left the CEP or the app's own identity system.

What capability you give up by doing this. The in-app layer cannot personalise content based on attributes it was never given, a user's name for a greeting, their specific purchase history for a recommendation, their exact account balance for a contextual message, unless those specific attributes are deliberately included in what gets forwarded, which reopens the PII question for each attribute individually rather than closing it globally. The practical resolution most integrations land on is a tiered approach: segment membership and lifecycle stage travel freely as the default, low-risk payload, and any additional attribute a specific campaign genuinely needs is evaluated and approved individually, rather than defaulting to sending the full user record because some future campaign might need some subset of it.

Where the PII Boundary Actually Sits

A security review's objection is rarely one undifferentiated concern. It is usually a specific worry about one of three distinct things, and identifying which one matters most to a given review determines what the architecture actually needs to solve for.

Processing identifiers. Does the vendor's system handle personal data at all, even transiently, as part of computing a targeting decision. This is the lightest form of concern, and even a well-designed segment-forwarding integration involves some processing, since the vendor's system receives and acts on a pseudonymous token.

Storing identifiers. Does the vendor retain a copy of personal data, or data that becomes personal data when combined with a reversal key, in persistent storage after the immediate transaction completes. This is a meaningfully higher-risk concern than transient processing, because stored data has a retention window, a backup footprint, and a breach exposure surface that transient processing does not.

Being able to re-identify a user. Does the vendor, on its own, without needing anything from the CEP or the app's own systems, have the ability to connect the identifier it holds back to a real person, a name, an email, a phone number, an account. This is the concern that actually drives most security review objections, because it is the scenario where a breach of the vendor's systems alone produces a meaningful privacy incident, independent of whether any other system is also compromised.

A segment-forwarding architecture, done correctly, keeps the in-app vendor firmly in the first category and largely out of the third: it processes a pseudonymous token to render the correct experience, it may store that token briefly for delivery and frequency-capping purposes, but it cannot independently re-identify the user, because the reversal key that connects the token to a real identity never left the systems that issued it. Naming this distinction explicitly, in exactly these terms, is frequently the fastest way to move a security review forward, because it replaces a vague objection about "PII" with a specific, answerable question about which of the three categories the actual architecture falls into.

Segment-as-Contract

The most durable way to prevent the in-app layer from ever needing raw attributes to target correctly is to treat the CEP's segment output as a stable, versioned API surface, the same discipline a well-run engineering team applies to any interface between two systems, rather than as an informal data export that happens to work today.

This means defining a fixed set of segment names and the conditions under which a user enters or exits each one, documenting that contract the same way an API contract is documented, and requiring any change to the underlying segmentation logic to go through the same review process a breaking API change would require, rather than being silently updated inside the CEP with no notice to the systems consuming its output. Defining clear data contracts for each consumer, specifying what data they can access, in what format, with what freshness guarantees, and subject to what access controls, prevents the ambiguity that leads to misuse of data or over-reliance on poorly governed datasets. Applied to segment forwarding specifically, this means the in-app layer's targeting logic is built against a documented, versioned segment definition, "user has completed KYC and made zero investments," not against an implicit assumption about what a segment currently means that could silently change the next time someone edits a rule inside the CEP.

The payoff of this discipline is that the in-app layer never needs to request raw attributes to compensate for an unclear or unstable segment definition. If the segment contract is precise and stable, "belongs to segment X" is sufficient information to target correctly, and the temptation to ask for additional raw data "just in case the segment definition changes" disappears, because the segment definition is a governed contract, not an assumption that might drift.

Reconciliation and Drift

Even a well-designed segment-forwarding architecture has a real question to answer: what happens when the CEP's computed segment and the in-app layer's understanding of that segment disagree, because the CEP's segment update has not yet propagated by the time a user takes an action inside a live session.

This is a standard eventual consistency problem, and the resolution strategies used across distributed systems generally apply directly. Eventual consistency allows for temporary inconsistencies across a distributed system, trading immediate synchronisation for availability and low latency, on the principle that data can be temporarily inconsistent across different nodes but will converge to a consistent state over time. The practical question for an in-app layer is not whether temporary disagreement can happen, it can, and will, given any architecture with more than one system computing or holding state, but which resolution strategy the team deliberately chooses for when it does.

Last-write-wins. The most recently updated segment value is treated as authoritative, discarding the older value. This is the simplest resolution strategy, assigning each version a timestamp and picking the version with the highest one when a conflict occurs, but it carries a specific risk: if clocks between systems are not perfectly synchronised, or if the "most recent" update was itself based on stale input, last-write-wins can silently discard a genuinely correct value in favour of an incorrect one that merely arrived later.

Source-of-truth precedence. Rather than resolving by timestamp, the architecture designates one system, typically the CEP, as authoritative for segment membership at all times, and the in-app layer's own cached copy is always treated as provisional, refreshed as frequently as latency requirements allow but never trusted over a fresher read from the source of truth when the two are available to compare. This is a stronger guarantee than last-write-wins for this specific use case, because it does not depend on clock synchronisation, only on which system is authoritative being unambiguous.

Session guarantees. Read-your-writes guarantees ensure that once a client performs an action, all subsequent reads within the same session reflect that action, which is the specific guarantee most relevant to the in-app use case: a user who just completed the action that should move them into a new segment should not see in-app content built for their old segment for the remainder of that same session, even if the CEP's batch segment recomputation has not yet run. This typically requires the in-app layer to accept a same-session, locally-computed override for the specific event that just occurred, layered on top of whatever the CEP's last-known segment value was, rather than waiting for the next full segment sync.

The practical recommendation: use source-of-truth precedence as the default reconciliation rule, with a session-level override for the specific, narrow case of an event that just happened in the current session and should visibly change behaviour before the next full sync cycle completes. Document which rule applies to which category of disagreement, the same way the segment contract itself should be documented, so a discrepancy discovered during debugging has a clear, pre-agreed answer rather than becoming a fresh architectural debate every time it surfaces.

The Security Review Checklist

Having these artefacts ready before the review starts, rather than scrambling to produce them once asked, is frequently the difference between a review that closes in a week and one that stalls for a month.

Data residency. Where is data processed and stored, and does that location satisfy any data localisation requirements the app's own regulatory obligations impose. Enterprise customers increasingly demand data localisation, EU customers want EU-region storage, Indian companies require DPDP Act localisation for sensitive personal data, and the architecture needs to support routing rules that direct data to specific geographic regions accordingly. For a segment-forwarding architecture specifically, this question is meaningfully easier to answer, because the data volume and sensitivity of what actually crosses a border is a fraction of what a full raw-event pipeline would require.

Sub-processor lists. Any third party with access to personal data, including infrastructure providers, analytics services, and support tools, counts as a sub-processor, and a current, published sub-processor list is a standard expectation in any serious vendor security review. Have this list ready and current before the review starts, not produced reactively when asked.

Retention windows. How long is any data held after it is received, and what is the deletion mechanism once that window closes. For a segment-forwarding architecture, this answer should be short, since the in-app layer's legitimate need to retain a pseudonymous segment membership is measured in the delivery and frequency-capping window, not an indefinite archival period.

Encryption in transit and at rest. AES-256 at rest and TLS 1.3 in transit are now the standardised baseline expectation in vendor security reviews, and a vendor that cannot confirm both without hesitation is a vendor that has not yet reached a security review baseline most competitors already clear.

DPA terms. The DPA must correctly classify the party roles, confirming the app is the controller, determining what data is processed and why, with the vendor as processor acting only on documented instructions, not the reverse. GDPR Article 28(3) requires the processor contract to set out the subject matter, duration, nature, and purpose of processing, the type of personal data, and the categories of data subjects explicitly, and a DPA missing any of these elements is not ready for a serious review regardless of how strong the underlying technical architecture is.

Deletion propagation. When a user requests deletion under GDPR, DPDP, or a comparable regime, does that deletion request propagate to the in-app vendor's systems, and within what timeframe. For a segment-forwarding architecture, this is again structurally easier to satisfy, since deleting the mapping between a pseudonymous token and a real identity at the source effectively renders any data the in-app vendor holds meaningless, without requiring a separate deletion request to reach every downstream system independently.

Instrumentation Hygiene

None of the architectural discipline above survives an event schema that drifts unmanaged over time, which is why instrumentation hygiene is the operational precondition for everything covered so far, not a separate concern.

Naming conventions. A consistent, documented naming pattern for events and properties, applied from the start and enforced going forward, is what prevents the same underlying user action from being tracked under two different event names in two different systems, which is one of the most common causes of segment disagreement that has nothing to do with reconciliation timing and everything to do with the two systems simply not agreeing on what counts as the same event in the first place.

A single source of truth for event definitions. A documented tracking plan, the authoritative reference for what every event and property means, who owns it, and what triggers it, needs to exist as a shared artefact both the CEP team and the in-app layer team read from, rather than each team maintaining its own informal understanding of what a given event represents. This is the same discipline covered in detail in the data foundation prerequisite for any personalisation work: a clean, governed event taxonomy is not a nice-to-have refinement layered on top of a working integration. It is the thing that makes segment-as-contract actually hold, because a contract built on an ambiguous or drifting event definition is not a stable contract at all.

Schema governance. A change management process for any new event or property, requiring review against the existing tracking plan before it ships, is what prevents the second tool's introduction from becoming the moment schema drift accelerates, since adding a second system that reads from the same event sources doubles the blast radius of any naming inconsistency that was already present but previously only affected one downstream consumer.

Topics Not in the Brief That Teams Should Know

The shopping cart anomaly is the canonical cautionary example for why naive conflict resolution fails. A well-documented incident at Amazon involved a shopping cart anomaly that arose specifically from poor conflict resolution logic in a distributed system, illustrating why manual or naively automated conflict resolution, rather than a deliberately chosen strategy like source-of-truth precedence, is an unacceptable burden to leave unresolved in any system handling user-facing state. Teams designing the reconciliation layer for segment drift should treat this as the specific failure mode to design against, not a theoretical edge case.

Vendor DPAs are frequently drafted to protect the vendor, not the customer, and need active redlining, not passive acceptance. Common issues in vendor-provided DPAs include weak purpose limitations that allow "service improvement" language broad enough to permit uses beyond the original scope, blanket sub-processor approval with no notification requirement, and breach notification windows of 30 or more days rather than the 24 to 72 hour standard a serious security posture requires. A security review that only checks whether a DPA exists, without checking whether its specific terms are actually protective, has not completed the review.

India's DPDP Act adds a specific wrinkle to sub-processor architecture that a GDPR-only compliance posture misses. Indian companies require DPDP Act localisation specifically for sensitive personal data, which is a distinct requirement from GDPR's more general residency preferences, and under DPDP, the data fiduciary, the app itself, remains accountable for any misuse or violation regardless of what the vendor's own contract or infrastructure looks like, which means a segment-forwarding architecture that minimises what any vendor holds is not just a nice-to-have risk reduction under DPDP specifically. It is a direct reduction in the fiduciary's own residual liability.

A CDP is not a prerequisite for this architecture, only one path to it. Teams without an existing CDP in their stack can implement CEP-forwarded segments directly, using the CEP's own export or webhook capability to push segment membership to the in-app layer, without needing to introduce a full customer data platform as an intermediate layer. The CDP-forwarded pattern is the right choice when a team already has, or genuinely needs, a CDP for reasons beyond this specific integration. It is not a requirement for achieving zero PII movement on its own.

Key Takeaways

The duplicate pipeline problem happens by default because the fastest way to get a new tool working is to wire it to the same raw event sources the existing CEP already reads from, which produces two independent segmentation computations that frequently disagree, not because anyone chose that outcome deliberately.

The three integration patterns, direct SDK instrumentation, CDP-forwarded events, and CEP-forwarded segments, trade latency, data volume, and governance against each other in a consistent direction: CEP-forwarded segments has the lowest data volume and strongest governance profile, at the cost of freshness and the inability to build targeting logic the CEP was never asked to compute.

Zero PII movement means passing a pseudonymous identifier plus a segment label, not zero data movement. Pseudonymised data is still personal data under GDPR if the reversal key exists anywhere, but it is a categorically safer payload than raw attributes, because the vendor holding it cannot independently re-identify the user.

A security review's objection usually concentrates on one of three distinct concerns: processing identifiers, storing identifiers, or being able to re-identify a user independently. A well-designed segment-forwarding architecture keeps the in-app vendor firmly out of the third category, and naming this distinction explicitly is frequently what moves a stalled review forward.

Treating the CEP's segment output as a versioned, documented contract, rather than an informal export, is what prevents the in-app layer from ever needing raw attributes to compensate for an unclear or drifting segment definition.

Segment disagreement between systems is a standard eventual consistency problem. Source-of-truth precedence, with a session-level override for events that just occurred in the current session, is a stronger default resolution strategy than naive last-write-wins, because it does not depend on clock synchronisation between systems.

The security review checklist, data residency, sub-processor lists, retention windows, encryption standards, DPA terms, and deletion propagation, should be prepared before a review starts, not produced reactively, and a segment-forwarding architecture makes several of these categorically easier to satisfy because of how little the vendor actually holds.

None of this holds without instrumentation hygiene: consistent naming conventions, a single documented source of truth for event definitions, and schema governance that prevents a second tool's introduction from doubling the blast radius of any drift that was already present in the event pipeline.

Integrating an In-App Layer Without Duplicate Data Pipelines or PII Movement