Integrate customer data and recover from failures
Plan source-to-destination data movement around stable identifiers, schema changes, checkpoints and an explicit reconciliation process.
What you'll be able to do
- Define how a connector handles repeated events and interrupted runs
- Plan for schema changes and deleted records
- Explain a reconciliation strategy during an integration interview
Before this: enterprise-ai-architecture
A customer integration has to keep working when inputs repeat, APIs time out and upstream teams change fields. Begin by defining the business entity you are moving. An order ID identifies an order. An event ID identifies one notification about it. Preserve both when the source supplies them, along with the source version and tenant.
Write the data contract
For a fictional order-status feed, record the required fields, allowed status transitions, timestamp format and ownership of corrections. Ask whether pages are stable while new orders arrive and how the source represents deletions. Agree on what freshness the customer needs and how you will measure lag.
A checkpoint advances only after the corresponding records are durably handled. Reconciliation checks the result against the source.
Use a stable cursor or documented pagination method. For each valid record, make the destination update repeatable. Handle duplicate notifications using the source event ID or a defined entity-and-version rule. Advance the checkpoint after durable writes. If the process stops between a write and its checkpoint, replay should reach the same destination state.
Keep bad records inspectable
Suppose a required currency field disappears after a schema update. Capture the rejected record's identifier, reason and source version in a controlled error queue. Alert the owning team and describe which downstream work is delayed. Establish a replay path after the contract is corrected. Protect sensitive fields in diagnostic output.
For a schema migration, agree on an overlap period or explicit version negotiation. Test the old and new shapes against the destination. Define what the system does when both versions appear during the transition.
Recover when an outcome is uncertain
An API timeout can happen after the receiving service has committed a write. Query the operation's durable identifier or reconcile with the source before repeating a non-idempotent action. Use bounded retries for transient failures and respect the provider's published retry guidance. Record progress so recovery has a starting point.
Practice. A connector imported 97 of 100 expected orders. Describe how you would distinguish missing source records, validation failures, duplicate IDs and failed writes. Your answer should identify the evidence and responsible system for each possibility.
Practise this stage with ChannelPulse
Use the coding and system-design workbenches to explain your recovery strategy and test boundary cases. Add the target role to Your interviews so your preparation stays attached to that opportunity. Open interview practice. Add your interview.
Reading and attribution
Tech Mahato's enterprise-AI guide and bicced's production guide informed these integration topics. The handbook is licensed under CC BY 4.0. The order feed and exercises are original ChannelPulse examples.