AI interviews · Forward-deployed engineering from discovery to interviews

Work a customer deployment case from brief to handoff

Walk through a fictional warranty-support engagement, calculate its pilot outcome, diagnose failures and practise defending the delivery decision in an interview.

15 min read Free to read Patterns: customer-case-walkthrough, decision-with-evidence

What you'll be able to do

Before this: fde-interview-preparation

This is a fictional ChannelPulse case. The organization, workload, sample sizes and outcomes are invented for practice. Use it to rehearse your reasoning before applying the same questions to a project you actually delivered.

1Clarify the brief▾

Northline Tools wants warranty tickets handled faster. Its director requests a chatbot and automatic refunds. Discovery identifies 240 eligible daily tickets with an observed 12-minute average handling time. Employees need to locate the correct policy for a purchase date and region, then ask a policy specialist about exceptions.

Agree on employee-reviewed answer drafts for one product family. The pilot tracks handling time, correct policy selection, rework and escalation. The finance owner will review refund authority as a later decision. The first working slice returns a draft, source links and a clear escalation path.

2Deliver a visible first slice▾

Connect one approved policy source and a historical ticket export. Preserve document versions and permissions. Show the support lead a complete request path and review several examples together. Log which evidence reached the model and which policy version each draft cites.

When a regional exception is missing, the assistant should expose that uncertainty and route the case for review. The prototype makes the missing information inspectable so the next discussion has a concrete example.

3Diagnose the evaluation failures▾

Replay 40 labelled development cases. In this illustrative run, 28 meet all required checks and 12 fail. Review produces the four primary causes used in the evaluation lesson. Five cases lack evidence, four use stale policy versions, two contain unsupported claims and one mishandles an exception.

Locate where the forty-case replay loses passing cases
stagevolumestep
Development cases 40 cases –
Applicable evidence 35 cases 88%
Current policy 31 cases 89%
Supported claims 29 cases 94%
Correct handoff 28 cases 97%
volume reaching this stage

The losses are five, four, two and one case respectively. The final 28 of 40 pass all checks, or 70%. Stages represent the first failing check in this illustrative review.

Improve the evidence lookup and version-selection path, then rerun the development cases. Before declaring improvement, review new held-out examples with the policy specialist. Keep security boundary tests as release gates. A development score by itself is insufficient to establish readiness for every customer workflow.

4Read the pilot outcome▾

Suppose a supervised pilot handles 60 tickets. Employees accept 42 drafts after review and escalate 18. The accepted cases average 8 minutes of handling time. Assuming those same cases would have taken the 12-minute baseline, the gross saving is 42 × 4 = 168 minutes, or 2.8 hours. Escalated cases require their own measured time and any additional review work before calculating the overall result.

The 70% draft acceptance rate measures employee behavior on this pilot. The earlier 70% development pass rate measures a different property on a different set. Report the denominator and definition beside each number. For a stronger time comparison, match comparable tickets or assign eligible tickets between the current workflow and the assistant, then account for complexity and rework.

5Respond to a production failure▾

The pilot exposes a document that remains searchable after its source access is revoked. Stop access to the affected retrieval path, determine which requests could have received it and follow the customer's incident process. Correct the permission propagation and cache invalidation behavior, then add a regression case for revocation. Recheck the full request path before resuming the affected feature.

Tell the support lead what is affected, what employees should use now, who owns the fix and when the next update is due. Explain the result in terms of their workflow.

6Demonstrate the handoff▾

Ask the customer's operator to locate an affected request, inspect its source version and disable the assistant using the runbook. Transfer the evaluation set, deployment procedure and escalation contacts. Agree who reviews failed drafts and how often the pilot outcome is revisited.

Defend the case out loud

Give yourself two minutes. State the original problem, agreed scope, first implementation, one failure, measured pilot result and handoff. Then answer a follow-up asking what you would change for ten times the traffic. Discuss connector limits, retrieval latency, evaluation coverage, cost and the operator's ability to investigate incidents.

Practise this stage with ChannelPulse

Rehearse the case as a system-design or behavioral answer and use the feedback to improve your explanation. Add your real project details to Profile when you switch from this fictional case to your own experience. Practise the case. Prepare your project context.

Reading and attribution

The case uses topics identified in Tech Mahato's enterprise-RAG project and bicced's evaluation and FDE guides. The handbook is licensed under CC BY 4.0. All narrative, calculations, examples and practice prompts are original ChannelPulse material.