AI Engineer interview questions & answers

20 AI Engineer interview questions with complete model answers, spanning Technical, Product & growth, System design, Behavioral, Coding. The bank holds 447 AI Engineer questions in total, tagged by round and difficulty.

BehavioralMediumAI Engineer

1. Tell me about a time you delivered good work outside your own domain.

The full question

Tell me about a time you delivered good work outside your own domain. How did you keep your judgement sound when you could not check every detail yourself?

Model answer

Anthropic's October 2026 scorecard analysis found the biggest increases for technical candidates were in comments on autonomy and judgement, while domain expertise shrank as a share of assessments. One hiring manager quoted in that report expects models to stay "jagged" and wants people who are "fungible across domains". This question tests whether you can be trusted with work you are not an expert in, which is increasingly normal when a model does much of the specialist drafting.

Choose a real stretch. A backend engineer shipping a mobile feature, an infrastructure engineer writing the eval for a model launch, an applied engineer drafting a data-processing agreement with legal. The domain gap should be obvious to the interviewer.

Explain how you built enough understanding. Not "I read the docs": which two or three concepts you had to get right, who you asked, what you deliberately did not learn because it did not affect the outcome.

Show your verification strategy. This is the core. When you cannot check every line, you check the things that would hurt most: invariants you can test mechanically, a small sample reviewed by a domain expert, comparison against a known-good reference, a staged rollout that limits the cost of being wrong. If a model drafted the work, say how you checked its claims rather than its fluency.

Name where you deferred. Strong answers include a point where you stopped and handed a decision to someone who knew more, and why that was the right call. Autonomy without knowing your limits reads as risk.

Close with the outcome and the lesson. What shipped, what broke, what you would now check earlier.

Follow-ups: What is a domain where you would refuse to work this way? How do you tell the difference between a model being confidently right and confidently wrong in a field you do not know?

BehavioralMediumAI Engineer

2. Alibaba open-sources Qwen under Apache 2.0 while running a commercial cloud business.

The full question

Alibaba open-sources Qwen under Apache 2.0 while running a commercial cloud business. Walk me through the strategy, and tell me about a time you owned an ambiguous technical decision end to end.

Model answer

This is a hybrid strategy-plus-behavioural question, and both halves matter here. On strategy, the honest business case for open weights from a cloud company: open models are demand generation for the cloud. Every developer who builds on Qwen is a candidate to run it on Alibaba Cloud (Model Studio / DashScope) or to buy the hosted, larger, or managed variants. Open weights build ecosystem gravity, standardise the world on your architecture and tokenizer, attract external contributions and talent, and generate an enormous free evaluation and bug-finding loop. It also reframes competition: against closed frontier labs, being the default open choice is a defensible position, especially internationally and in markets sensitive to vendor lock-in. The tradeoff is that you give away the weights, so your moat shifts to scale, serving efficiency, the newest/biggest closed variants, data, and the cloud platform around the model. Being able to argue this crisply signals you understand why your work exists commercially, not just technically.

For the ownership half, use STAR and pick a real story where the requirements were genuinely unclear and you drove to a decision: frame the ambiguity (conflicting goals, no obvious right answer), the options you weighed with their tradeoffs, the data or experiments you used to decide, the call you made, and - critically - how you measured whether it worked and what you would change. Large AI orgs weight ownership, cross-team collaboration, and incident handling heavily in the hiring-manager round, so land the "I owned it, including the parts that went wrong" note rather than a tidy success story. Keep it concrete and quantified.

Follow-ups: Where is the open-weight strategy weakest, and what would change your mind? Tell me about a decision of yours that turned out wrong and how you caught it.

BehavioralMediumAI Engineer

3. Tell me about a time a technical decision you championed turned out to be wrong.

The full question

Tell me about a time a technical decision you championed turned out to be wrong. What happened, and what did you change afterward?

Model answer

Microsoft interviews behavioural competencies deliberately - official guidance names growth mindset and recommends STAR(R): Situation, Task, Action, Result, and Reflection. The final R is what separates a Microsoft-calibrated answer: they are explicitly hiring "learn-it-alls," so the learning must be concrete and durable, not a platitude.

Strong answer structure: (1) Situation/Task - one crisp sentence of context with real stakes ("I pushed us to fine-tune a model for our support-bot rather than build retrieval, because I'd seen fine-tuning work at my previous team"). (2) Action - what you decided and, importantly, why it was reasonable at the time; owning a wrong-but-defensible decision reads better than a strawman. (3) Result - quantify the miss honestly ("three weeks in, eval accuracy plateaued below the RAG baseline and every content update needed a retrain; we'd burned a month"). (4) Reflection + changed behaviour - the graded part: what generalisable lesson, and what you now do differently ("I now insist on a cheapest-viable-baseline spike before committing to the heavier approach, and I wrote the decision doc template my team still uses"). Bonus signal: how you handled being wrong publicly - informing stakeholders early, crediting the teammate who was right.

Anti-patterns: a "failure" that's secretly a win, blaming externals, lessons with no behaviour change, or a story with no stakes. Prepare 3-4 such stories mapped against their six competencies (collaboration, drive for results, customer focus, influencing for impact, judgment, adaptability) - most behavioural prompts are one of these in disguise.

Follow-ups: What did the teammate who disagreed with you see that you didn't? How do you decide when to reverse a decision versus push through?

BehavioralMediumAI Engineer

4. Tell me about a time you disagreed with a researcher (or tech lead) about priorities, and what happened.

Model answer

This is the culture round; at DeepMind and Google generally the evaluated axes are collaboration, intellectual humility, and data-driven disagreement ("Googleyness" on the Google side). Structure as STAR, but the content interviewers listen for:

  • You disagreed on substance, respectfully. Example shape: a researcher wanted a bespoke experimental fork of the training stack for one paper; you believed the fork would rot and pushed for landing the feature behind a flag in mainline. Present the researcher's position as legitimate - deadline pressure before a conference is real - not as an obstacle.
  • You reduced the disagreement to evidence. You quantified the cost (past forks took N weeks to reconcile; the flagged path added two days now) and timeboxed a spike to de-risk the mainline approach rather than debating in the abstract.
  • Clean resolution, including the case where you lost. "Disagree and commit" is explicitly valued: if the decision went the other way, show you executed it wholeheartedly and set a checkpoint to revisit with data - not that you relitigated.
  • Relationship outcome. The researcher-engineer partnership is the core working unit at a lab; end with how the collaboration got stronger (they brought you in earlier on the next project).

Anti-patterns: villainising the counterpart, "I was right all along" endings, escalating to management as the first move, and conflicts that are really just communication failures you didn't own. Pick a story with genuine technical stakes - evaluation validity, reproducibility, infra debt - because those map directly onto what REs at a lab negotiate weekly.

Follow-ups: What would have made you change your mind? Describe a time you were wrong in a technical disagreement - how did you find out?

BehavioralMediumAI Engineer

5. Why do you want to work at a safety-focused lab - and where do you disagree with Anthropic?

Model answer

This is the values round in miniature, and public reports consistently describe it as a round where many candidates fail. Two failure modes to avoid: recited enthusiasm ("AI is the most important technology of our time") and performed doom. Anthropic's own culture language is "light and shade" - holding genuine optimism and genuine concern simultaneously.

What a strong answer contains:

  • Specific engagement with their actual positions. You've read "Core Views on AI Safety" or the RSP and can reference an argument - ideally including one you find weak or uncertain. Genuine engagement includes disagreement; the question invites it, and dodging it ("I agree with everything") signals you haven't thought hard.
  • A personal, non-borrowed reason. Something from your own experience building or deploying AI systems - a moment where capability outran your ability to evaluate or control it - beats any philosophical citation.
  • Consistency under probing. Interviewers reportedly push: "Would you delay a launch that costs the company revenue if evals showed marginal risk?" The tested property isn't your answer; it's whether your reasoning stays coherent when the scenario mutates. Decide your actual views beforehand, because improvised ethics collapses under two follow-ups.
  • Epistemic honesty. "I don't know" and "I changed my mind about X when Y" are assets. Reports describe this round as unusually personal - closer to a candid conversation than a behavioural checklist.

Preparation is legitimate here and their own guidance endorses using Claude for it: argue both sides of a safety question out loud until your position survives contact.

Follow-ups: Tell me about a time you were the one arguing against shipping. What would make you leave Anthropic?

CodingMediumAI EngineerPair programming

6. Paired coding: write a chunker for a legal document that never splits a clause and carries enough context that a retrieved chunk is self-contained.

Model answer

The interesting part is not the token counting, it is deciding what travels with the chunk. Talk through the boundary rules before writing code.

import re
from dataclasses import dataclass, field

CLAUSE = re.compile(r"^\s*(\d+(?:\.\d+)*)\s+(.{0,120}?)\s*$")   # "7.02  Indebtedness"
DEFINED = re.compile(r'"([A-Z][A-Za-z ]+)"')                     # "Permitted Indebtedness"

@dataclass
class Chunk:
    text: str
    heading_path: list[str] = field(default_factory=list)
    section: str = ""
    defined_terms: dict[str, str] = field(default_factory=dict)
    cross_refs: list[str] = field(default_factory=list)

def chunk(lines, definitions, max_tokens, count):
    """Split on clause boundaries. Oversized clauses split on sentences,
    keeping the heading on every piece so no fragment is orphaned."""
    out, buf, path, section = [], [], [], ""
    for line in lines:
        m = CLAUSE.match(line)
        if m:                                   # boundary: flush what we have
            if buf:
                out += emit(buf, path, section, definitions, max_tokens, count)
            section, path, buf = m.group(1), update_path(path, m), [line]
        else:
            buf.append(line)
    if buf:
        out += emit(buf, path, section, definitions, max_tokens, count)
    return out

def emit(buf, path, section, definitions, max_tokens, count):
    body = "\n".join(buf)
    pieces = [body] if count(body) <= max_tokens else split_sentences(body, max_tokens, count)
    return [
        Chunk(
            text=p,
            heading_path=list(path),
            section=section,
            # only the definitions this piece actually uses, so context stays small
            defined_terms={t: definitions[t] for t in DEFINED.findall(p) if t in definitions},
            cross_refs=re.findall(r"Section\s+(\d+(?:\.\d+)*)", p),
        )
        for p in pieces
    ]

Points worth saying out loud: the definitions dictionary is built in a prior pass over the definitions article plus inline ("Borrower") patterns, and only the terms a chunk actually uses are attached, so context stays small. cross_refs are stored rather than inlined, so the retriever can expand one hop at query time instead of duplicating text into every chunk. The heading path is repeated on every split piece, which is the cheapest fix for orphaned fragments. And the whole thing is only meaningful with a parser upstream - if the input is a scanned PDF, the regex boundaries are only as good as the OCR.

Follow-ups: How would you extend this to a document with schedules and exhibits that are effectively separate documents? What test would tell you this chunker is better than fixed 512-token windows?

CodingMediumAI EngineerCoding

7. A standard Alibaba coding round.

The full question

A standard Alibaba coding round. Design a data structure for a fixed-capacity cache that supports get and put in O(1) and evicts the least-recently-used entry when full. Implement it in Python.

Model answer

This is the classic LRU cache, and it is worth knowing cold because the algorithmic rounds gate the whole loop regardless of how senior the model work is. The requirement is O(1) for both operations, which rules out scanning for the least-recently-used item. The standard answer is a hash map plus a doubly linked list: the map gives O(1) lookup by key, and the linked list keeps entries in recency order so eviction is O(1) at the tail and promotion on access is O(1) by splicing.

class Node:
    __slots__ = ("k", "v", "prev", "next")
    def __init__(self, k=0, v=0):
        self.k, self.v, self.prev, self.next = k, v, None, None

class LRUCache:
    def __init__(self, capacity: int):
        self.cap = capacity
        self.map = {}                      # key -> Node
        self.head, self.tail = Node(), Node()   # sentinels
        self.head.next, self.tail.prev = self.tail, self.head

    def _remove(self, n):
        n.prev.next, n.next.prev = n.next, n.prev

    def _push_front(self, n):              # most-recently-used at front
        n.prev, n.next = self.head, self.head.next
        self.head.next.prev = n
        self.head.next = n

    def get(self, key: int) -> int:
        if key not in self.map:
            return -1
        n = self.map[key]
        self._remove(n); self._push_front(n)
        return n.v

    def put(self, key: int, value: int) -> None:
        if key in self.map:
            n = self.map[key]; n.v = value
            self._remove(n); self._push_front(n)
            return
        if len(self.map) == self.cap:
            lru = self.tail.prev            # evict at tail
            self._remove(lru); del self.map[lru.k]
        n = Node(key, value)
        self.map[key] = n; self._push_front(n)

Sentinel head/tail nodes remove the null-checking edge cases that trip people up under time pressure. In Python you could also use collections.OrderedDict with move_to_end and popitem(last=False) for a shorter answer, but interviewers usually want the underlying mechanism, so I would code the linked list and mention the shortcut. Complexity: O(1) time per operation, O(capacity) space.

Follow-ups: Extend it to LFU (least-frequently-used) - what changes? How does this relate to KV-cache eviction policies in long-context serving?

CodingMediumAI Engineer

8. Qwen2.5-Coder trains with repository-level fill-in-the-middle using special tokens like < fim_prefix >, < fim_suffix >, < fim_middle >, < repo_name…

The full question

Qwen2.5-Coder trains with repository-level fill-in-the-middle using special tokens like <|fim_prefix|>, <|fim_suffix|>, <|fim_middle|>, <|repo_name|>, and <|file_sep|>. Write the function that formats a repo-level FIM training example, and explain why repo-level beats file-level for real code completion.

Model answer

FIM teaches the model to fill a gap given both left and right context, which is what an editor autocomplete actually needs (you have code before and after the cursor). The trick is to reorder: put prefix and suffix first, then have the model predict the middle. Repo-level FIM extends this across files so the model can use imports and definitions from elsewhere in the project.

def format_repo_fim(repo_name, files, target_idx, cut_a, cut_b):
    # files: list of (path, content). Split the target file into pre/mid/suf.
    path, content = files[target_idx]
    pre, mid, suf = content[:cut_a], content[cut_a:cut_b], content[cut_b:]
    parts = [f"<|repo_name|>{repo_name}"]
    for i, (p, c) in enumerate(files):
        if i == target_idx:
            parts.append(
                f"<|file_sep|>{p}\n"
                f"<|fim_prefix|>{pre}<|fim_suffix|>{suf}<|fim_middle|>{mid}"
            )
        else:
            parts.append(f"<|file_sep|>{p}\n{c}")   # context files verbatim
    return "".join(parts) + "<|endoftext|>"

At training time the loss is on the whole sequence (or focused on the middle); at inference you feed everything up to <|fim_middle|> and let the model generate the missing span. The special tokens matter because they are unambiguous structural signals the model learns to key on: <|repo_name|> and <|file_sep|> mark project and file boundaries so cross-file references are learnable, and the FIM markers tell the model this is an infilling task, not left-to-right continuation.

Why repo-level wins: real completions depend on symbols defined in other files - a function you call, a type you import, a config constant. File-level training caps the model's world at one file, so it hallucinates signatures. Repo-level pretraining (reported around 32K-token context with long-context repo data) lets the model actually read the surrounding project, which is the difference between guessing an API and using the real one. The cost is longer sequences and careful data construction (ordering files, avoiding leakage of the answer into context).

Follow-ups: How would you order context files so the most relevant ones survive context truncation? Why can naive FIM masking accidentally leak the middle into the suffix, and how do you prevent it?

CodingMediumAI EngineerLive coding

9. Live coding: build the prompt for the next turn under a fixed token budget.

The full question

Live coding: build the prompt for the next turn under a fixed token budget. The catch is our prefix cache.

Model answer

The trap is the obvious implementation: keep the last N tokens and drop the oldest. That truncates from the front, which changes the prefix on every turn and destroys the inter-turn cache from Q2 - you convert a 95% hit rate into near zero and multiply serving cost.

The correct shape orders the prompt by stability, so everything that changes lives at the end:

def build_context(char, memory, turns, user_msg, budget, tokenize):
    # Stable prefix: identical across turns and often across users.
    prefix = char.system_prompt + char.definition + char.example_dialogue
    head = tokenize(prefix)

    # Semi-stable: rolling summary plus durable facts. Changes only at
    # a compaction event, never on every turn.
    mem = tokenize(memory.summary + memory.facts)

    tail = tokenize(user_msg)
    remaining = budget - len(head) - len(mem) - len(tail) - RESERVED_FOR_REPLY
    if remaining < 0:
        raise ContextTooSmall

    # Fill backwards from the most recent turn, but emit in order.
    kept, used = [], 0
    for t in reversed(turns):
        tt = tokenize(t)
        if used + len(tt) > remaining:
            break
        kept.append(tt)
        used += len(tt)
    kept.reverse()

    return head + mem + flatten(kept) + tail, len(head) + len(mem)

Two things to say out loud. First, return the stable prefix length so the caller knows where the cache boundary sits. Second, compaction must be event-driven, not per-turn: when the recent-turn window overflows, summarise a whole block at once and accept one cache invalidation, rather than shifting the boundary every turn. Amortised, that is one expensive prefill every dozens of turns instead of one every turn.

Worth sketching. Ordering by stability is the entire design.

flowchart LR
    A["Pinned prefix<br/>persona and definition"] --> B["Rolling summary<br/>and durable facts"]
    B --> C["Recent turns verbatim"]
    C --> D["Current user message"]
    D --> E["Reply budget reserved"]
Diagram

Follow-ups: Where do you insert retrieved older turns without breaking the cache boundary? How would you unit-test that a refactor did not silently drop the hit rate?

CodingMediumAI EngineerCoding

10. Code: implement the block manager for a paged KV cache - allocate, append, free, and copy-on-write prefix sharing.

Model answer

The core is a free list of fixed-size blocks, per-sequence block tables, and reference counts for sharing:

class BlockManager:
    def __init__(self, num_blocks: int, block_size: int):
        self.block_size = block_size
        self.free = list(range(num_blocks))
        self.refs = [0] * num_blocks
        self.tables = {}        # seq_id -> list[block_id]
        self.lengths = {}       # seq_id -> tokens written

    def _alloc(self) -> int:
        if not self.free:
            raise MemoryError("KV cache exhausted")  # caller preempts/queues
        b = self.free.pop()
        self.refs[b] = 1
        return b

    def append_token(self, seq_id: int):
        table = self.tables.setdefault(seq_id, [])
        n = self.lengths.get(seq_id, 0)
        if n % self.block_size == 0:              # need a new block
            table.append(self._alloc())
        else:
            last = table[-1]
            if self.refs[last] > 1:               # shared: copy-on-write
                self.refs[last] -= 1
                new = self._alloc()
                # (kernel would copy last block's KV data here)
                table[-1] = new
        self.lengths[seq_id] = n + 1

    def fork(self, parent: int, child: int):      # share prefix
        self.tables[child] = list(self.tables[parent])
        self.lengths[child] = self.lengths[parent]
        for b in self.tables[child]:
            self.refs[b] += 1

    def free_seq(self, seq_id: int):
        for b in self.tables.pop(seq_id, []):
            self.refs[b] -= 1
            if self.refs[b] == 0:
                self.free.append(b)
        self.lengths.pop(seq_id, None)

Discussion points that separate levels: only the last block ever needs copy-on-write (earlier blocks are immutable once full); allocation failure is a scheduling event (preempt a sequence and recompute or swap its KV), not an exception to crash on; and the block table is exactly what the attention kernel consumes to gather K/V physically.

Follow-ups: How do you pick block size - what do 1-token vs 128-token blocks trade? How would you add prefix caching across requests (hash-based block reuse)?

Product & growthMediumAI Engineer

11. A generated note contains a medication the patient never mentioned.

The full question

A generated note contains a medication the patient never mentioned. Treat that as a safety incident: how do you detect it before a clinician sees it?

Model answer

First, name the categories, because "hallucination" is too coarse to act on. At minimum: fabrication (content with no source in the transcript or chart), contradiction (content that reverses what was said, typically negation flips), attribution errors (right content, wrong speaker or wrong tense), and omission (a clinically significant item the note dropped). Omission is the one candidates forget and clinicians care about most. Then weight by severity - a fabricated dose or allergy is categorically worse than a fabricated pleasantry - so your top-line number is not dominated by harmless noise.

Detection runs as a verification stage over the draft, not as a hope embedded in the prompt. Decompose the note into atomic claims, then check each claim for entailment against the transcript plus the chart context that was actually retrieved for this encounter. Do it with a mix of cheap deterministic checks and a model-based verifier: every medication, dose, unit, frequency, date, and numeric in the note should be matchable to a source span, and anything unmatched is unsupported by construction. Negation and hedging get their own checker, because entailment models are historically weak there.

Routing matters as much as detection. Unsupported spans should be suppressed or visibly flagged for review rather than silently deleted, since silent deletion converts a fabrication into an omission. Contradictions should block, not warn.

Then close the loop offline: sample production notes for clinician adjudication, treat their labels as the reference set, and track fabrication rate per thousand notes by severity class as a release gate rather than a dashboard metric.

Worth sketching. The verification stage sitting between generation and the clinician.

flowchart TD
    A["Draft note"] --> B["Decompose into atomic claims"]
    B --> C["Deterministic checks:<br/>meds, doses, numerics, dates"]
    B --> D["Entailment verifier vs<br/>transcript and chart"]
    C --> E{"Supported?"}
    D --> E
    E -->|"yes"| F["Keep with evidence link"]
    E -->|"no"| G["Flag for review or suppress"]
    E -->|"contradicted"| H["Block release"]
Diagram

Follow-ups: Your verifier itself has a false-negative rate. How do you set its operating point? How would you catch omissions, which no entailment check will ever surface?

System designMediumAI EngineerLow-level design

12. Low-level design: sketch the classes and interfaces for the tool-calling layer of an agent host, where tools can come from native code, an OpenAPI…

The full question

Low-level design: sketch the classes and interfaces for the tool-calling layer of an agent host, where tools can come from native code, an OpenAPI spec, or an MCP server.

Model answer

Start from the seam. A tool is four things: a name, a description the model reads, a parameter schema, and something to invoke. Everything else is policy layered on top.

Core types:

  • ToolDefinition - immutable value object holding name, description, JSON Schema for parameters, optional result schema. Immutable because it is shared across threads and serialised into every prompt.
  • ITool - ToolDefinition Definition { get; } plus Task<ToolResult> InvokeAsync(ToolCall call, ToolContext ctx, CancellationToken ct). Deliberately narrow: one property, one method.
  • ToolResult - a success/error union rather than thrown exceptions. The error text is fed back to the model for a corrective retry, so it has to be structured and safe to show: no stack traces, no connection strings.
  • ToolContext - caller identity, tenant, correlation id, remaining budget and deadline. Passing identity explicitly instead of reading ambient state is what makes "the tool runs as the user, never as the service" testable.
  • IToolProvider - discovery plus a factory for handlers. One adapter per source: reflection over annotated methods, an OpenAPI parser, an MCP client. A new integration is a new provider, and the orchestration loop never changes.
  • ToolRegistry - resolves names, namespaces them by provider so two sources can both ship a search, and hands each session a filtered view, meaning the allowlist for that agent rather than the global set.

Cross-cutting concerns go in decorators over ITool, not in a base class: timeout, retry, rate limit, audit log, and a RequiresApproval wrapper that returns a pending result for the host to surface to the user. Decorators compose in a declared order and each is unit-testable against a fake inner tool. The invoker validates arguments against the schema before dispatch, so a hallucinated parameter fails cheaply instead of halfway through a side effect.

Worth sketching. How one narrow interface plus decorators lets three unrelated tool sources share a single execution path.

flowchart LR
    A["Providers: native,<br/>OpenAPI, MCP"] --> B["ToolRegistry<br/>(namespaced)"]
    B --> C["Per-session allowlist"]
    C --> D["Decorators: timeout,<br/>retry, audit, approval"]
    D --> E["ITool.InvokeAsync"]
    E --> F["ToolResult<br/>(success or error)"]
    F -->|"error fed back"| G["Agent loop"]
Diagram

Follow-ups: Two providers register a tool called search. How does the registry resolve that, and what name does the model actually see? Where does per-user permission enforcement belong: the tool, a decorator, or the downstream service?

System designMediumAI Engineer

13. Design a service that turns the conversation into draft orders: labs, imaging, referrals, prescriptions.

The full question

Design a service that turns the conversation into draft orders: labs, imaging, referrals, prescriptions. Assume tool calls against the EHR.

Model answer

This is the highest-stakes surface in the product, because a note is a record while an order is an action. Design accordingly: the service proposes, the clinician disposes, and nothing reaches the EHR without explicit human acceptance.

Shape it as a bounded agent loop rather than an open-ended one. Detect order intent in the conversation ("let's get a CBC", "I'll send you for an MRI of the left knee"), which is a classifier problem with heavy negation and hypothetical handling - "we might need imaging if this doesn't settle" is not an order. For each candidate intent, resolve it against real catalogues via tools: the orderable item catalogue for this health system, which differs per site; the formulary; the patient's allergy and current medication list for interaction checks; and any required fields the order type demands (laterality, indication, priority, frequency, duration). Missing required fields are the common case, and the correct behaviour is to surface a specific gap to the clinician, not to invent a value.

Constrain the tool layer hard. Read tools are freely callable; write tools do not exist in the agent's action space at all. The loop's only output is a structured order proposal with evidence links, populated fields, unresolved gaps, and any safety flags. A separate, non-agentic path performs the write after clinician acceptance. That separation is what makes the system reviewable and what you should say first.

Reliability details worth raising: cap loop iterations and tool calls per encounter, make every tool call idempotent and logged for audit, validate the proposal against a schema before it reaches the UI, and fail closed - an order the system cannot fully resolve is shown as an incomplete suggestion rather than dropped silently, because a silently dropped order is a missed test.

Worth sketching. The bounded loop with the write path deliberately outside it.

flowchart TD
    A["Transcript segment"] --> B["Order intent detection"]
    B --> C["Resolve via read-only tools:<br/>catalogue, formulary, allergies"]
    C --> D{"Required fields complete?"}
    D -->|"no"| E["Surface specific gap"]
    D -->|"yes"| F["Structured proposal<br/>with evidence links"]
    E --> G["Clinician review UI"]
    F --> G
    G -->|"explicit accept"| H["Non-agentic write to EHR"]
Diagram

Follow-ups: The clinician mentions a drug the patient is allergic to. What does the system do? How would you evaluate this end to end, given that a missed order and a spurious order have very different costs?

System designMediumAI Engineer

14. PHI is in every audio file, transcript, and note you touch.

The full question

PHI is in every audio file, transcript, and note you touch. How does that shape the architecture, and what can you send to a third-party model API?

Model answer

Start from the legal frame, because it drives the engineering. Operating on protected health information on behalf of a health system makes you a business associate under HIPAA, which means a signed BAA with the covered entity and, critically, a BAA with every subprocessor that touches PHI. That single requirement decides your model vendor list before any benchmark does: if a provider will not sign a BAA and commit to no-training-on-your-data terms, that provider is not in the architecture, regardless of how good the model is. Abridge's public trust materials describe HIPAA business associate status, SOC 2 Type II controls, US data residency, SSO/MFA, and audit logging, with retention governed by customer agreements.

Architectural consequences. PHI-bearing data stays inside a defined boundary with encryption in transit and at rest, and inference runs either self-hosted inside that boundary or against a vendor covered by a BAA with zero-retention terms. Tenant isolation is real isolation, since one health system's audio must never influence or leak into another's context. Retention becomes a configurable per-customer contract, so audio and transcripts age out on the customer's schedule and every derived artefact respects the same clock. Audit logging is append-only and covers reads, not just writes, because "who listened to this patient's recording" is a question a compliance officer will ask.

De-identification deserves a nuanced answer rather than a reflexive one: it is useful for building research and evaluation datasets, but it does not help the production path, since the note has to name the patient's actual medications and history. Do not claim it as a way around the BAA requirement. And minimum necessary applies to context assembly - retrieve the chart slices this encounter needs, not the whole record.

Worth sketching. Where the PHI boundary sits and what crosses it.

flowchart LR
    A["Clinic device"] -->|"encrypted"| B["PHI boundary:<br/>ingest, ASR, storage"]
    B --> C["Context assembly<br/>(minimum necessary)"]
    C --> D["Inference: self-hosted<br/>or BAA-covered vendor"]
    D --> B
    B --> E["EHR write-back"]
    B --> F["Append-only audit log"]
    G["De-identified corpus"] -.->|"eval and research only"| H["Offline model work"]
    B -.-> G
Diagram

Follow-ups: A new frontier model would clearly improve note quality but the vendor will not sign a BAA. What are your options? How would you prove to an auditor that a specific patient's audio was deleted on schedule?

System designMediumAI Engineer

15. Design discovery and search across millions of user-created characters.

Model answer

This is a marketplace ranking problem with three properties that make it unlike standard content recommendation: the catalogue is user-generated and unbounded, quality is enormously skewed, and engagement labels are contaminated by exactly the wellbeing tension in Q11.

Eligibility first. Before ranking, filter on safety and quality. Character definitions are user-authored text that will be shown to other users and injected into prompts, so they need classification for policy violations and for prompt-injection attempts. Age-appropriate filtering has to be a hard gate tied to the account's assurance state, not a ranking feature.

Candidate generation. Two complementary sources. Embedding retrieval over character definitions and tags handles semantic and cold-start cases, and a co-engagement model (users who talked to A also talked to B) handles the head. Union them, a few hundred candidates.

Ranking. Predict a multi-task target rather than click: probability of starting a conversation, expected conversation depth, and a return-within-a-week term. Depth matters more than clicks here, because a character that gets opened and abandoned is a failure that click-through rewards.

Cold start is the structural problem. Most characters are created and never surface. Without deliberate exploration the system ossifies around the first popular thousand and creators leave. Reserve exploration slots, use embedding similarity to bootstrap new characters near proven ones, and measure creator-side outcomes (fraction of new characters getting a first real conversation) as a first-class metric.

Search is a different intent: users arrive with a name, a fandom, or a description. Hybrid lexical plus semantic retrieval, with the same eligibility gate, and heavy handling of near-duplicate characters, since popular concepts have thousands of near-identical copies.

Worth sketching. The eligibility gate sits before ranking, not after.

flowchart LR
    A["Millions of<br/>user-made characters"] --> B["Safety and quality<br/>eligibility gate"]
    B --> C["Candidate generation<br/>embeddings plus co-engagement"]
    C --> D["Multi-task ranker<br/>start, depth, return"]
    D --> E["Exploration slots and<br/>duplicate collapsing"]
    E --> F["Feed and search results"]
Diagram

Follow-ups: How do you collapse thousands of near-duplicate versions of the same popular character without erasing the creators who made the good ones? What offline metric would you trust enough to gate a launch here?

System designMediumAI Engineer

16. A user says "send Maya the photos from Saturday's hike." Design the on-device path from that utterance to a structured app action with resolved par…

The full question

A user says "send Maya the photos from Saturday's hike." Design the on-device path from that utterance to a structured app action with resolved parameters.

Model answer

Treat this as three staged problems, not one prompt: action selection, parameter resolution, and commitment.

Action selection. The device holds a registry of app-declared actions - Apple's App Intents framework is the public surface, with typed AppIntent parameters and AppEntity types for each app's domain objects. Do not stuff every installed app's intents into the prompt; with hundreds of apps that blows both the on-device context limit and the TTFT budget. Retrieve first: embed the utterance, run semantic retrieval over intent descriptions and app-shortcut phrases, keep ~10 candidates, then let the model select one and fill parameters under constrained decoding against that intent's schema, so the call parses by construction.

Parameter resolution is the harder half, and mostly not the model's job. "Maya" resolves against contacts, "Saturday's hike" against photo-library date and scene metadata. Those are on-device indexes; the model proposes a query, the resolver returns ranked candidates with confidence. That structure is also why personal data never enters a prompt - you resolve references locally instead of shipping the user's contact graph anywhere.

Ambiguity and commitment. Two Mayas, or three matching date ranges, is a disambiguation turn, not a guess; framework-level disambiguation is cheaper and more predictable than model-level guessing. Then gate on consequence: read-only actions can just run, while mutating or externally visible ones (sending, sharing, purchasing, deleting) need confirmation that displays the resolved entities. The asymmetry is the point - a wrong recipient on a photo share is unrecoverable, a wrong timer is not.

Speech and latency. Stream ASR so intent retrieval starts before endpointing, and budget the whole turn - a spoken assistant feels broken past roughly a second of dead air.

Worth sketching. The staged pipeline, and where the two user-facing gates sit.

flowchart TD
    A["Utterance<br/>streaming ASR"] --> B["Retrieve top-10<br/>candidate app intents"]
    B --> C["On-device model picks intent,<br/>fills parameters"]
    C --> D["Resolve entities against<br/>contacts, photos, calendar"]
    D --> E{"Unambiguous?"}
    E -->|"no"| F["Ask user to disambiguate"]
    F --> D
    E -->|"yes"| G{"Consequential action?"}
    G -->|"yes"| H["Confirm resolved entities"]
    G -->|"no"| I["Execute"]
    H --> I
Diagram

Follow-ups: The utterance matches no installed intent - what does the system do, and what does the user hear? How would you measure end-to-end action accuracy across third-party apps without logging what people actually asked for?

TechnicalMediumAI Engineer

17. Serve a DeepSeek-V3-class open MoE model (671B total parameters, ~37B active per token) on Blackwell.

The full question

Serve a DeepSeek-V3-class open MoE model (671B total parameters, ~37B active per token) on Blackwell. How do you lay it out, and why does expert parallelism change the economics?

Model answer

Separate memory from compute. Every expert must be resident, so FP8 weights alone are ~671 GB: one 8×B200 node (~180 GB each, ~1.4 TB total) holds them with room for KV cache, 8×H100 does not. Per-token compute is set by the ~37B active parameters, so decode FLOPs look like a mid-size dense model while memory looks like a giant one.

The trap is arithmetic intensity per expert. Each token picks 8 of 256 routed experts per layer, so with a global batch of B tokens each expert sees roughly B/32. At batch 64, every expert's weights are read to serve about two tokens: pure bandwidth-bound decode. Reaching the ~150-300 tokens per weight read the roofline wants means thousands of concurrent tokens per step. Hence wide expert parallelism: spread experts across many GPUs, aggregate a very large batch, and move tokens to experts rather than weights to tokens.

Communication becomes the bottleneck. Each MoE layer does an all-to-all dispatch and an all-to-all combine. A rack-scale NVLink domain (GB200 NVL72 puts 72 GPUs on one fabric) keeps that off InfiniBand, and libraries such as DeepEP plus two-micro-batch overlap hide it behind compute.

Decisions to defend:

  • Attention parallelism. MLA's latent KV cache is small, but tensor-parallel attention replicates it on every rank, so data-parallel attention with expert-parallel FFNs is the usual pairing.
  • Load balance. Routing is skewed and shifts with traffic, and a hot expert stalls the whole step. Replicate hot experts and re-place them from observed routing statistics.
  • Disaggregation. Prefill and decode want different parallelism, so run separate pools and ship KV between them over RDMA.

The economics: wide EP pays off only with large aggregate traffic, which suits a shared serverless pool. A low-traffic dedicated endpoint is better on one 8-GPU node with tensor plus expert parallelism, accepting lower utilisation.

Worth sketching. One decode step, showing that the expensive hop is moving tokens between GPUs, not loading weights.

flowchart LR
    A["Request router"] --> B["Prefill pool"]
    B -->|"KV transfer (RDMA)"| C["Decode pool<br/>DP attention"]
    C --> D["Gate: top-8 of 256 experts"]
    D -->|"all-to-all dispatch"| E["Expert GPUs<br/>(EP across the rack)"]
    E -->|"all-to-all combine"| F["Next layer or<br/>sample token"]
    F --> C
Diagram

Follow-ups: A large coding customer onboards and the routing histogram shifts sharply - what happens to p99 inter-token latency, and how do you respond? How does the layout change for a ~1T-parameter model with 384 experts?

TechnicalMediumAI Engineer

18. A whole-body controller trained entirely in simulation has to run on real hardware.

The full question

A whole-body controller trained entirely in simulation has to run on real hardware. What transfers, what does not, and how do you close the gap?

Model answer

Simulation gives you what real robots cannot: Figure's System 0 is described as trained entirely in simulation across more than 200,000 parallel environments with extensive domain randomisation, on over 1,000 hours of retargeted human motion. You cannot run 200,000 real humanoids, and you cannot let a real one fall over a million times.

What transfers well: gross kinematics and coordination, balance strategies, contact-rich locomotion where the physics is dominated by rigid-body dynamics you can model. Anything driven by geometry and gravity.

What transfers badly: actuator dynamics (backlash, friction, torque saturation, thermal derating), sensor noise and bias, communication and computation latency, deformable or granular objects, and fine contact with compliant materials. Perception transfers worst of all when the sim renderer's appearance statistics do not match real cameras.

Domain randomisation is the main tool: randomise mass and inertia, joint friction and damping, controller gains, actuator delay, sensor noise, ground friction, and external pushes. The policy is then forced to learn something that works across a whole family of dynamics, and the real robot is one more member of that family. The trade-off is real: randomise too little and you overfit to the simulator, too much and you get a conservative, sluggish policy that hedges against worlds that do not exist. The practical rule is to centre the randomisation range on measured hardware parameters and widen until real-world performance stops improving.

Complements worth naming: system identification to fit sim parameters to measured hardware responses, actuator networks that learn the real motor's response from bench data and replace the analytic model, and real data fine-tuning on a small amount of hardware experience. For appearance gaps, randomise textures and lighting or train perception on real data and only the dynamics in sim.

Worth sketching. The gap closes through a measure-and-retune loop, not a single transfer.

flowchart TD
    A["Retargeted human<br/>motion data"] --> B["Massively parallel<br/>sim training"]
    B --> C["Domain randomisation<br/>mass, friction, latency, gains"]
    C --> D["Policy robust across<br/>a family of dynamics"]
    D --> E["Deploy on hardware"]
    E -->|"residual gap"| F["Measure real response,<br/>refit sim parameters"]
    F --> B
Diagram

Follow-ups: How would you tell an overfit-to-sim failure from a genuinely under-trained policy? Why does randomising latency matter more than it sounds like it should?

TechnicalMediumAI Engineer

19. New California laws require developers of clinical decision tools to take reasonable steps to reduce known or foreseeable bias and to protect clini…

The full question

New California laws require developers of clinical decision tools to take reasonable steps to reduce known or foreseeable bias and to protect clinicians' professional judgement. For an ambient scribe that drafts orders and diagnosis codes, what would you build to show that?

Model answer

AB 1979 and SB 503 (signed 30 September 2026) set that expectation, and the FDA's final Clinical Decision Support Software guidance (January 2026) is the federal reference for when software becomes a regulated device. A scribe that only transcribes is far from that line; one that drafts orders and codes is closer, so the engineering answer has three parts.

Know where the CDS line falls. Map every output to a category: documentation of what was said (low risk), suggestions derived from what was said (draft orders, codes), and recommendations the clinician did not raise (closer to decision support). Keep the third category explicit and small. The FDA guidance turns on whether a clinician can independently review the basis for a recommendation, which is exactly what span-level provenance (Q5) provides: every draft order cites the moment it was discussed.

Preserve professional judgement in the workflow. Draft orders are never auto-signed; each needs an explicit clinician action, and accepting all with one click should be measured and discouraged. Record what was suggested, what was accepted, edited or rejected, and by whom. Track acceptance rate by suggestion type: near-100% acceptance of a suggestion class is a warning sign of automation bias, not a success metric.

Bias evaluation by patient subgroup. The likely failure points are upstream: ASR error rates vary with accent, language, age and speech patterns, and transcription errors propagate into notes, orders and codes. Build an eval set stratified by language (across all 28+ supported), accent, age band, sex, and visit type, with enough samples per cell for confidence intervals. Measure WER on clinical terms, omission rate of discussed problems, and order and code accuracy per subgroup. Set a maximum acceptable gap and treat a breach as a release blocker. Watch interpreter-mediated visits and code-switching specifically, since they are where gaps tend to hide.

Make it continuous and documented. Re-run the stratified eval on every model change, monitor production proxies (edit rate and omission complaints by language and site), and keep a written record of known limitations and mitigations. "Reasonable steps" is easier to demonstrate with a versioned history of measurements than with a one-off audit.

Follow-ups: You lack demographic labels for most production audio - how do you build the subgroup eval ethically? A gap appears only for one language at one site - model, microphone or workflow? Where would you draw the line on suggesting an order the clinician never mentioned?

TechnicalMediumAI Engineer

20. Explain fine-tuning versus distillation.

The full question

Explain fine-tuning versus distillation. What are the limits of distilling a small model from a large one?

Model answer

A mid-2026 candidate report describes this as an opening question in a founder-led screen. It rewards precise definitions and an honest view of what distillation cannot do.

Fine-tuning updates a pretrained model's weights on task data with labels from any source: human-written answers, preference pairs, or reward signals. The goal is new behaviour or better behaviour on a target distribution. It can be full-parameter or parameter-efficient (LoRA and similar).

Distillation trains a student to imitate a teacher. Two main forms: sequence-level (generate outputs from the teacher and fine-tune the student on them, which is really fine-tuning on synthetic data) and logit-level (match the teacher's full next-token distribution with a KL loss, which transfers much more information per token because the student sees which wrong answers the teacher considered plausible). Logit distillation needs access to the teacher's logits and a shared or mapped tokenizer.

Why distil. Serving cost and latency: a small student can recover most of a large teacher's quality on a narrow distribution at a fraction of the compute per token.

The limits.

  • Capacity. A student with far fewer parameters cannot store the teacher's breadth of knowledge. It matches the teacher on the distribution you distilled on and degrades elsewhere, often sharply.
  • Coverage. The student only learns what your prompts exercise. Rare skills and long-tail facts are lost unless the distillation set covers them.
  • Reasoning depth. Copying the teacher's final answers or even its reasoning traces does not guarantee the student can carry out the same multi-step computation; it may learn the surface pattern of reasoning without the reliability.
  • Teacher errors and biases transfer, sometimes amplified, and the student cannot exceed the teacher on the teacher's own signal.
  • Gap too large. Distilling across a very large size gap often works worse than going through an intermediate model, because the student cannot fit the teacher's distribution.
  • Licensing. Many providers' terms forbid using their outputs to train competing models, so check before distilling from an API.

When to choose which. Fine-tune when you have good task labels and want new behaviour; distil when you already have a model that does the job and the problem is cost or latency.

Follow-ups: How would you measure what the student lost relative to the teacher? When would on-policy distillation (student generates, teacher scores) beat training on teacher outputs?

Practice these out loud, don't memorise them

Reading an answer is not the same as being able to give one under pressure. ChannelPulse plays the interviewer, asks the follow-ups, and scores each answer with feedback and a model answer so you can hear the gap between what you said and what lands.

Get ChannelPulse Browse all questions