Waymo interview questions & answers

Model answer

Situation In my role as a software engineer at a mid-sized tech company, I was part of a team working on a new feature for our flagship product. Midway through the development cycle, the product manager informed us of a significant change in the project requirements due to a shift in market demands. This change required us to pivot from our original plan and integrate a new third-party API, which was crucial for maintaining our competitive edge.

Task My specific responsibility was to lead the integration of this new API into our existing codebase. The challenge was to adapt quickly to the new requirements while ensuring minimal disruption to our timeline, as we were already operating under a tight deadline.

Action

• I immediately organized a meeting with the team to discuss the new requirements and assess the impact on our current progress. This helped us identify the areas that needed immediate attention and re-prioritize tasks.
• Recognizing the need for rapid upskilling, I coordinated with the team to divide the learning tasks. I personally took charge of understanding the API documentation and identifying potential integration challenges.
• To ensure smooth communication and progress tracking, I set up a Trello board to manage tasks related to the API integration. This visual tool helped the team stay aligned and allowed us to quickly adapt to any further changes.
• I also maintained regular communication with the product manager to provide updates and receive feedback on our approach, ensuring that we were aligned with the business goals.
• As we progressed, I encouraged the team to conduct daily stand-up meetings. This practice allowed us to address roadblocks promptly and make necessary adjustments to our strategy.

Result Through these efforts, we successfully integrated the new API within the revised timeline. The feature was launched on schedule, and it received positive feedback from both the client and end-users. This experience reinforced the importance of adaptability and proactive communication in managing changing project requirements. It taught me the value of effective collaboration and the need to embrace change as an opportunity for growth and innovation.

Model answer

Situation: In my role as a software engineer at a tech company, I was part of a team working on a project to enhance our autonomous vehicle software. During a routine code review, I noticed a recurring issue with the way our data processing pipeline was handling sensor data. The pipeline was not efficiently managing the data load, leading to delays in processing and potential inaccuracies in real-time decision-making. This was critical as the reliability of sensor data directly impacted the vehicle's ability to make safe navigational decisions.

Task: My responsibility was to identify the root cause of the inefficiencies and propose a solution that would optimize the data processing without compromising the accuracy or increasing the system's complexity.

Action:

• I began by conducting a thorough analysis of the data processing pipeline, focusing on the areas where bottlenecks were most likely to occur. This involved reviewing logs and performance metrics to pinpoint where delays were happening.
• I discovered that the issue stemmed from a lack of parallel processing capabilities in our existing setup. The system was processing sensor data sequentially, which was not scalable given the volume of data being generated.
• To address this, I proposed implementing a parallel processing framework that could handle multiple data streams simultaneously. I researched various frameworks and chose Apache Kafka due to its ability to manage real-time data streams efficiently.
• I organized a meeting with the team to present my findings and proposed solution. I demonstrated how Kafka could be integrated into our existing architecture with minimal disruption and highlighted the expected improvements in processing speed and reliability.
• After gaining buy-in from the team, I led the implementation of the new framework, ensuring thorough testing and validation to confirm that the changes met our accuracy and performance requirements.

Result: The implementation of the parallel processing framework resulted in a significant reduction in data processing time, improving the system's overall efficiency and reliability. The enhanced pipeline allowed the autonomous vehicles to process sensor data in real-time, leading to safer and more accurate navigational decisions. This experience reinforced the importance of proactive problem identification and the value of leveraging appropriate technologies to address system inefficiencies. It also highlighted the impact of effective communication and collaboration in driving successful project outcomes.

Model answer

Situation In my role as a product manager at a tech company, we recently launched a change to our app's wait-time display, making it show more conservative estimates. Post-launch, we observed a 5% drop in conversion rates, while the actual time to pickup decreased by 25%. A stakeholder suggested that this indicated improved fleet efficiency. However, I needed to analyze the situation further to understand the underlying causes and implications.

Task My goal was to determine whether the decrease in pickup time truly reflected improved fleet efficiency or if there were other factors at play. Additionally, I needed to assess how the change in conversion rates might influence our interpretation of these results.

Action

• I started by gathering data on fleet operations and customer interactions both before and after the change. This included metrics on driver availability, route optimization, and customer feedback.
• I conducted a detailed analysis of the data, looking for patterns or anomalies that could explain the reduced pickup times. I considered factors such as changes in traffic patterns, driver behavior, or even external events that might have influenced the results.
• To understand the conversion drop, I organized a focus group with users to gather qualitative insights. We discovered that the more conservative wait-time estimates might have led to a perception of longer waits, discouraging some users from completing their bookings.
• I collaborated with the data science team to run A/B tests, comparing user behavior with different wait-time displays. This helped us isolate the effect of the display change from other variables.
• I presented my findings to the stakeholders, highlighting that while fleet efficiency might have played a role, the change in user perception due to conservative estimates was a significant factor in both the pickup time reduction and conversion drop.

Result Through this analysis, we concluded that the decrease in pickup time was not solely due to improved fleet efficiency. The conservative wait-time estimates altered user behavior, leading to fewer bookings and thus less congestion, which inadvertently improved pickup times. This insight prompted us to adjust our display strategy to balance accuracy with user expectations. The experience reinforced the importance of data-driven decision-making and considering user perception in product changes.

Model answer

Situation In my previous role as a software engineer at a tech company, I was part of a cross-functional team tasked with launching a new feature for our flagship product. The team included members from engineering, product management, and design. The stakes were high as this feature was crucial for improving user engagement and was a key deliverable for the quarter. My role was to lead the engineering efforts and ensure seamless integration with existing systems.

Task My specific goal was to coordinate with the product manager and designers to align on requirements and timelines. The key challenge was managing differing priorities and ensuring that all team members were on the same page, especially given the tight deadline.

Action

• I organized a kickoff meeting to establish clear communication channels and set expectations. This included defining roles, responsibilities, and a shared timeline.
• To address potential misalignments, I set up regular check-ins with the product manager and design lead. This ensured that any changes in requirements or design were communicated promptly to the engineering team.
• I facilitated a collaborative workshop where team members could voice their concerns and suggestions. This fostered a sense of ownership and helped us identify potential roadblocks early.
• I implemented a shared project management tool to track progress and dependencies. This transparency allowed everyone to stay informed and adjust their work as needed.
• When conflicts arose, such as differing opinions on feature prioritization, I mediated discussions to find a compromise that balanced technical feasibility with user needs.

Result The feature was successfully launched on time and received positive feedback from users, resulting in a 15% increase in user engagement. The collaborative approach not only ensured the project's success but also strengthened the team's ability to work together effectively. I learned the importance of proactive communication and the value of diverse perspectives in achieving a common goal.

Model answer

// Function to compute the probability that a match goes to 5 games
function probabilityOfFiveGames(p) {
  // Probability that the match goes to 5 games
  // This happens if both players win 2 games each in the first 4 games
  const prob = 6 * Math.pow(p, 2) * Math.pow(1 - p, 2);
  return prob;
}

// Monte Carlo simulation to estimate the probability of a match going to 5 games
function simulateMatches(numMatches, p) {
  let fiveGameMatches = 0;

  for (let i = 0; i < numMatches; i++) {
    let winsA = 0;
    let winsB = 0;
    let gamesPlayed = 0;

    while (winsA < 3 && winsB < 3) {
      gamesPlayed++;
      if (Math.random() < p) {
        winsA++;
      } else {
        winsB++;
      }
    }

    if (gamesPlayed === 5) {
      fiveGameMatches++;
    }
  }

  return fiveGameMatches / numMatches;
}

// Function to build a confidence interval for the estimated probability
function confidenceInterval(numMatches, p, confidenceLevel = 0.95) {
  const thetaEstimate = simulateMatches(numMatches, p);
  const z = 1.96; // z-score for 95% confidence
  const marginOfError = z * Math.sqrt((thetaEstimate * (1 - thetaEstimate)) / numMatches);

  return [thetaEstimate - marginOfError, thetaEstimate + marginOfError];
}

// Example usage
const p = 0.5; // Probability that player A wins a game
const numMatches = 10000; // Number of simulated matches
console.log("Probability of 5 games (analytical):", probabilityOfFiveGames(p));
console.log("Probability of 5 games (simulation):", simulateMatches(numMatches, p));
console.log("95% Confidence Interval:", confidenceInterval(numMatches, p));

• Analytical Calculation: The probability that a match goes to 5 games is calculated by considering the scenarios where both players win exactly 2 games each in the first 4 games. This is given by \(6 \times p^2 \times (1-p)^2\), where 6 is the number of ways to arrange 2 wins for each player in 4 games.
• Monte Carlo Simulation: Simulate a large number of matches, counting how many go to 5 games. For each match, simulate games until one player wins 3 games. Count matches where exactly 5 games are played.
• Confidence Interval: Use the normal approximation for the binomial distribution to calculate the confidence interval for the estimated probability. The margin of error is calculated using the standard error and the z-score for the desired confidence level.

Complexity:

• Time: \(O(M)\) for the simulation, where \(M\) is the number of matches.
• Space: \(O(1)\), constant space usage.

Model answer

import numpy as np

def kmeans(X, k, max_iters=100, tol=1e-4):
    # Step 1: Initialize k centroids randomly from the dataset
    n_samples, n_features = X.shape
    centroids = X[np.random.choice(n_samples, k, replace=False)]

    for _ in range(max_iters):
        # Step 2: Assign each point to the nearest centroid
        distances = np.linalg.norm(X[:, np.newaxis] - centroids, axis=2)
        labels = np.argmin(distances, axis=1)

        # Step 3: Recompute centroids as the mean of assigned points
        new_centroids = np.array([X[labels == i].mean(axis=0) for i in range(k)])

        # Check for convergence (if centroids do not change significantly)
        if np.all(np.linalg.norm(new_centroids - centroids, axis=1) < tol):
            break

        centroids = new_centroids

    return centroids, labels

def kmeans_plus_plus(X, k):
    n_samples, n_features = X.shape
    centroids = np.empty((k, n_features))
    
    # Initialize the first centroid randomly
    centroids[0] = X[np.random.choice(n_samples)]
    
    # Initialize the remaining centroids
    for i in range(1, k):
        distances = np.min(np.linalg.norm(X[:, np.newaxis] - centroids[:i], axis=2), axis=1)
        probabilities = distances / np.sum(distances)
        centroids[i] = X[np.random.choice(n_samples, p=probabilities)]

    return centroids

# Example usage
X = np.random.rand(100, 2)  # Example dataset
k = 3
initial_centroids = kmeans_plus_plus(X, k)
final_centroids, assignments = kmeans(X, k)

• K-means Objective: K-means aims to minimize the variance within each cluster, effectively minimizing the sum of squared distances between data points and their respective cluster centroids.
• Lloyd’s Algorithm: 1. Initialize k centroids randomly. 2. Assign each data point to the nearest centroid. 3. Update centroids by calculating the mean of all points assigned to each centroid. 4. Repeat steps 2 and 3 until convergence or a maximum number of iterations is reached.
• Improved Initialization (K-means++):
• Select the first centroid randomly.
• For each subsequent centroid, choose a data point with a probability proportional to its distance squared from the nearest existing centroid.
• This helps in spreading out the initial centroids, leading to faster convergence and better clustering results.
• Handling Empty Clusters: If a cluster becomes empty, reinitialize its centroid to a random data point.
• Stopping Criteria: Convergence is typically determined by checking if the centroids do not change significantly between iterations.

Complexity:

• Time: O(n k t * d), where n is the number of points, k is the number of clusters, t is the number of iterations, and d is the dimensionality.
• Space: O(n d) for storing the dataset and O(k d) for the centroids.

Model answer

class ListNode {
  constructor(value = 0, next = null) {
    this.value = value;
    this.next = next;
  }
}

function mergeTwoLists(l1, l2) {
  // Create a dummy node to act as the start of the merged list
  let dummy = new ListNode();
  let current = dummy;

  // Iterate while both lists have nodes
  while (l1 !== null && l2 !== null) {
    if (l1.value < l2.value) {
      current.next = l1; // Point to the smaller node
      l1 = l1.next; // Move to the next node in list l1
    } else {
      current.next = l2; // Point to the smaller node
      l2 = l2.next; // Move to the next node in list l2
    }
    current = current.next; // Move the current pointer
  }

  // If there are remaining nodes in l1 or l2, append them
  if (l1 !== null) {
    current.next = l1;
  } else {
    current.next = l2;
  }

  // Return the merged list, which starts at dummy.next
  return dummy.next;
}

• Approach:
• Use a dummy node to simplify edge cases and act as the head of the merged list.
• Compare the current nodes of both lists, appending the smaller one to the merged list.
• Continue until one list is exhausted, then append the remaining nodes of the other list.
• Return the merged list starting from dummy.next.
• Complexity:
• Time: O(n + m), where n and m are the lengths of the two lists. Each node is processed exactly once.
• Space: O(1), as the merging is done in place without using additional space for data structures.

Model answer

function twoSum(nums, target) {
    // Create a map to store the difference and its index
    const numMap = new Map();

    // Iterate over the array
    for (let i = 0; i < nums.length; i++) {
        const complement = target - nums[i];

        // Check if the complement exists in the map
        if (numMap.has(complement)) {
            // Return the indices of the two numbers
            return [numMap.get(complement), i];
        }

        // Store the number and its index in the map
        numMap.set(nums[i], i);
    }

    // Return an empty array if no solution is found
    return [];
}

// Example usage:
// const result = twoSum([2, 7, 11, 15], 9);
// console.log(result); // Output: [0, 1]

• Approach: Use a hash map to store each number's complement (target - current number) and its index. As you iterate through the array, check if the current number's complement is already in the map. If it is, return the indices of the complement and the current number.
• Complexity:
• Time: O(n), where n is the number of elements in the array. We traverse the array once.
• Space: O(n), for storing elements in the hash map.

Model answer

Clarify & scope: My favorite product is Spotify, a music streaming service. The goal is to improve the user experience for discovering new music. Assume Spotify's current features include playlists, recommendations, and radio.

User segments & pain points: Focus on casual listeners who struggle to find new music that aligns with their tastes.

Goals & success metrics: The North Star metric is user engagement with new music. Guardrails include user retention and satisfaction.

Solutions:

1. Enhanced Discovery Algorithm: Improve algorithms to better match user preferences.
2. Social Discovery Features: Allow users to see and share what friends are listening to.
3. Personalized Listening Sessions: Curate sessions based on mood and past listening habits.

Recommendation: Implement social discovery features to leverage network effects and increase engagement.

Prioritization & trade-offs: Social features have a high potential impact with moderate development effort, making them a priority.

MVP, measurement & rollout: Launch a beta version of social features, measure engagement metrics, and iterate based on user feedback.

Model answer

Clarify & scope: The goal is to enhance the user experience for elderly users of Waymo's autonomous ride-hailing service. Assume the service is operational in urban areas with a focus on safety and ease of use.

User segments & pain points: Focus on elderly users who may face challenges with technology, mobility, and trust in autonomous systems.

Goals & success metrics: The North Star metric is user satisfaction among elderly riders. Guardrails include ride completion rates and safety incident reports.

Solutions:

1. Simplified App Interface: Create a user-friendly app version with larger text and voice commands.
2. In-Vehicle Assistance: Equip vehicles with a virtual assistant to guide users through the ride.
3. Pre-Ride Tutorials: Offer optional tutorials on how to use the service.

Recommendation: Implement the simplified app interface first, as it directly addresses usability.

graph LR
A[User opens app] --> B[Selects destination]
B --> C[Confirms ride]
C --> D[Virtual assistant guides]

Prioritization & trade-offs: Using RICE, the simplified app scores highest due to its broad impact and relatively low effort.

MVP, measurement & rollout: Launch a pilot with the simplified app in one city, gather feedback, and iterate based on user satisfaction scores.

Model answer

Clarify: The goal is to evaluate the success of Waymo's autonomous ride-hailing service. Assume the service is operational across multiple cities.

Define metric(s): Key metrics include ride completion rate, user satisfaction score, safety incident rate, and market penetration.

Break down:

• Ride Completion Rate: Measures operational reliability.
• User Satisfaction Score: Reflects customer experience.
• Safety Incident Rate: Indicates safety performance.
• Market Penetration: Shows market share and growth.

funnel
    subgraph Waymo Metrics
    direction TB
    Rides Requested --> Rides Completed
    Rides Completed --> User Satisfaction
    User Satisfaction --> Market Penetration
    end

Ranked hypotheses:

1. High completion rate correlates with high user satisfaction.
2. Low safety incident rate boosts user trust and satisfaction.
3. High market penetration indicates competitive success.

How to investigate: Use A/B testing to explore factors affecting user satisfaction and analyze safety data trends.

Decision & guardrails: Focus on improving metrics with the most significant impact on user satisfaction and safety, ensuring regulatory compliance and operational efficiency.

Model answer

Clarify & scope: The goal is to design a feature for Waymo's app that encourages repeat usage. Assume the app is used for booking and managing autonomous rides.

User segments & pain points: Focus on occasional users who may not see the value in frequent use.

Goals & success metrics: The North Star metric is the frequency of ride bookings per user. Guardrails include user satisfaction and retention rates.

Solutions:

1. Loyalty Program: Offer rewards for frequent usage, such as discounts or priority booking.
2. Personalized Ride Suggestions: Use AI to suggest rides based on user habits and preferences.
3. Gamified Experience: Introduce a points system for completed rides that unlocks perks.

Recommendation: Implement a loyalty program as it directly incentivizes increased usage.

graph LR
A[User opens app] --> B[Checks ride history]
B --> C[Receives loyalty points]
C --> D[Books next ride]

Prioritization & trade-offs: The loyalty program offers a high impact on engagement with manageable effort, making it a priority.

MVP, measurement & rollout: Launch a pilot of the loyalty program in select cities, measure booking frequency, and adjust based on user feedback.

Model answer

1. Requirements & scale

Functional Requirements:

• Calculate optimal routes for self-driving cars.
• Provide real-time updates on route changes due to traffic or road conditions.
• Estimate time of arrival (ETA) at the destination.
• Handle multiple vehicles and routes simultaneously.

Non-Functional Requirements:

• High availability and fault tolerance.
• Low latency for real-time route updates.
• Scalability to handle a growing number of vehicles and routes.
• Consistency in route data and ETA calculations.

Back-of-the-Envelope Estimates:

• Vehicles: Assume 10,000 self-driving cars.
• Requests per second (QPS): If each car requests route updates every 10 seconds, we have 1,000 QPS.
• Storage: Assuming each route data is 10KB, storing routes for 10,000 vehicles requires 100MB.
• Bandwidth: 1,000 QPS * 10KB = 10MB/s.

2. High-level architecture

flowchart TD
    subgraph Client
        A[Self-driving Car]
    end

    subgraph Edge/CDN
        B[Edge Server]
    end

    subgraph Load Balancer
        C[Load Balancer]
    end

    subgraph API / Services
        D[Route Service]
        E[Traffic Service]
        F[Map Service]
    end

    subgraph Cache
        G[Route Cache]
    end

    subgraph Datastores
        H["Geospatial DB"]
        I["Traffic DB"]
    end

    subgraph Message Queue
        J[Update Queue]
    end

    subgraph Workers
        K[Route Calculation Worker]
    end

    A -->|Route Request| B
    B -->|Route Request| C
    C -->|Route Request| D
    D -->|Fetch Traffic| E
    D -->|Fetch Map Data| F
    E -->|Traffic Data| G
    F -->|Map Data| G
    D -->|Route Calculation| K
    K -->|Route Update| J
    J -->|Route Update| A
    D -->|Store Route| H
    E -->|Store Traffic| I

3. API design

• GET /route
• Purpose: Retrieve the optimal route for a given start and end location.
• Parameters: start_location, end_location, vehicle_id.
• POST /traffic/update
• Purpose: Update traffic conditions in the system.
• Body: { location, traffic_info }.
• GET /eta
• Purpose: Get the estimated time of arrival for a given route.
• Parameters: route_id, vehicle_id.

4. Data model & storage

Datastores:

• Geospatial DB (SQL/NoSQL): Used for storing map data and routes. A NoSQL database like MongoDB with geospatial indexing can efficiently handle location-based queries.
• Traffic DB (SQL/NoSQL): Stores real-time traffic data. A time-series database like InfluxDB can be used for efficient storage and retrieval of traffic updates.

Key Tables:

• Routes Table:
• route_id (Primary Key)
• start_location
• end_location
• path (Geospatial data)
• eta
• Traffic Table:
• location (Geospatial index)
• traffic_info
• timestamp

5. Deep dive

The core of the route optimization system is the route calculation algorithm, which must efficiently compute the optimal path given real-time traffic data and map information.

sequenceDiagram
    participant Car as Self-driving Car
    participant Edge as Edge Server
    participant LB as Load Balancer
    participant Route as Route Service
    participant Traffic as Traffic Service
    participant Map as Map Service
    participant Worker as Route Calculation Worker

    Car->>Edge: Request Route
    Edge->>LB: Forward Request
    LB->>Route: Route Request
    Route->>Traffic: Fetch Traffic Data
    Route->>Map: Fetch Map Data
    Traffic-->>Route: Return Traffic Data
    Map-->>Route: Return Map Data
    Route->>Worker: Calculate Optimal Route
    Worker-->>Route: Return Route
    Route->>Edge: Send Route Update
    Edge->>Car: Route Update

6. Scale, bottlenecks & trade-offs

Scalability:

• Sharding: Geospatial DB and Traffic DB can be sharded based on geographical regions to distribute load and improve query performance.
• Caching: Frequently accessed routes and traffic data can be cached to reduce latency and load on databases.

Bottlenecks:

• Traffic Data Updates: High frequency of traffic updates can overwhelm the system. Implementing a message queue can help in processing updates asynchronously.
• Route Calculation: Computationally intensive; can be offloaded to dedicated workers.

Trade-offs:

• Consistency vs. Availability (CAP Theorem): Prioritize availability and eventual consistency for traffic data to ensure the system remains responsive.
• Push vs. Pull: Use a push model for critical route updates to ensure timely delivery, while less critical updates can be pulled by the vehicle periodically.

This design ensures a robust, scalable, and efficient route optimization system for self-driving cars, capable of handling real-time updates and providing optimal routes.

Model answer

1. Requirements & scale

Functional Requirements:

• Collect and store 2D turning trajectory data for vehicles at intersections.
• Differentiate between Waymo autonomous vehicles and other vehicles.
• Analyze and quantify differences in turning behavior between Waymo and other vehicles.

Non-Functional Requirements:

• Ensure data processing is efficient and scalable to handle large volumes of trajectory data.
• Maintain high availability and reliability of the analysis system.
• Provide results with statistical significance and confidence.

Scale Estimates:

• Assume 100 intersections, each with an average of 1000 turns per day.
• Total turns per day = 100 intersections * 1000 turns = 100,000 turns.
• If each trajectory consists of 100 data points, total data points per day = 100,000 turns * 100 points = 10 million data points.
• Storage: Assuming each data point requires 100 bytes (including metadata), daily storage needs = 10 million * 100 bytes = 1 GB.

2. High-level architecture

flowchart TD
    subgraph Client
        A[Drone Data Collector]
    end
    subgraph Edge/CDN
        B[Data Ingestion Service]
    end
    subgraph Load Balancer
        C[Load Balancer]
    end
    subgraph API / Services
        D[Trajectory Analysis API]
    end
    subgraph Cache
        E[In-Memory Cache]
    end
    subgraph Datastores
        F[Time-Series DB]
        G[Relational DB]
    end
    subgraph Workers
        H[Data Processing Workers]
    end
    subgraph Message Queue
        I[Message Queue]
    end

    A -->|Raw Data| B
    B -->|Processed Data| C
    C --> D
    D -->|Query Results| E
    D -->|Store Trajectories| F
    D -->|Store Metadata| G
    D -->|Analysis Requests| I
    I --> H
    H -->|Processed Results| G

3. API design

• POST /trajectories: Accepts raw trajectory data for processing and storage.
• GET /analysis/waymo-vs-others: Returns statistical analysis comparing Waymo and other vehicles.
• GET /trajectories/{intersection_id}: Retrieves trajectories for a specific intersection.

4. Data model & storage

Datastores:

• Time-Series DB: Used for storing trajectory data due to its time-indexed nature. Suitable for high write throughput and efficient querying of time-based data.
• Relational DB: Stores metadata and analysis results, facilitating complex queries and joins.

Key Tables:

• Trajectories Table: intersection_id, vehicle_type, turn_type, timestamp, x, y.
• Analysis Results Table: intersection_id, comparison_metric, p_value, effect_size.

Partition Key:

• Use intersection_id as the partition key to distribute data across multiple nodes efficiently.

5. Deep dive

To quantify differences in turning behavior, we can use a statistical approach such as the Kolmogorov-Smirnov (K-S) test to compare the distributions of trajectory features like speed, heading, and curvature between Waymo and other vehicles.

Trajectory Representation:

• Use derived features: speed v(t), heading θ(t), curvature κ(t), and acceleration a(t).
• Normalize these features to account for different scales and units.

Distributional Assumptions:

• Assume that trajectory features follow a continuous distribution.
• The K-S test does not assume a specific distribution, making it suitable for this analysis.

Hypothesis Test:

• Null Hypothesis (H0): There is no difference in the distribution of trajectory features between Waymo and other vehicles.
• Alternative Hypothesis (H1): There is a significant difference in the distribution.

Effect Size:

• Report the effect size using metrics like the difference in means or medians of the distributions.

sequenceDiagram
    participant Client as Drone Data Collector
    participant API as Trajectory Analysis API
    participant DB as Time-Series DB
    participant Worker as Data Processing Workers

    Client->>API: Send Trajectory Data
    API->>DB: Store Trajectory Data
    API->>Worker: Trigger Analysis
    Worker->>DB: Retrieve Trajectories
    Worker->>API: Return Analysis Results
    API->>Client: Send Analysis Results

6. Scale, bottlenecks & trade-offs

Replication and Sharding:

• Use sharding on the intersection_id to distribute data across multiple database nodes.
• Replicate data to ensure high availability and fault tolerance.

Caching:

• Implement write-through caching for frequently accessed analysis results to reduce latency.

Single Points of Failure:

• Use a load balancer to distribute requests and prevent single points of failure.
• Ensure redundancy in the message queue and worker nodes.

Trade-offs:

• Consistency vs. Availability: Prioritize consistency in analysis results using write-through caching, accepting some latency.
• Push vs. Pull: Use a push model to trigger analysis immediately after data ingestion, ensuring timely results.
• SQL vs. NoSQL: Use a combination of SQL (for metadata) and NoSQL (for time-series data) to balance complex queries and high write throughput.

Model answer

1. Requirements & scale

Functional Requirements:

• add_link(a_id, b_id): Link an A ID to a B ID, indicating they refer to the same object.
• add_observation(observation): Add an observation with system ID, object ID, timestamp, and metadata.
• get_history(system, object_id): Retrieve all observations for a real object, ordered chronologically.

Non-Functional Requirements:

• Low latency for get_history queries.
• High availability and consistency in linking and retrieving observations.
• Efficient memory usage for in-memory storage.

Scale Estimates:

• Assume 10,000 unique objects, each with 100 observations.
• Each observation is approximately 1 KB.
• Total memory requirement: 10,000 objects 100 observations 1 KB = 1 GB.
• Assume 100 QPS for add_observation and 50 QPS for get_history.

2. High-level architecture

flowchart TD
    subgraph Client
        A["User Interface"]
    end

    subgraph API / Services
        B["ObjectTracker Service"]
    end

    subgraph Cache
        C["In-Memory Store (e.g., Redis)"]
    end

    A -->|API Calls| B
    B -->|Read/Write| C

3. API design

• POST /add_link: Links an A ID to a B ID.
• POST /add_observation: Adds an observation with a system ID, object ID, timestamp, and metadata.
• GET /get_history: Retrieves the observation history for a given system and object ID.

4. Data model & storage

Datastore Choice:

• Use an in-memory store like Redis for fast access and storage of observations and links.

Data Structures:

• Observations Table:
• Key: system:object_id
• Value: List of observations (timestamp, metadata)
• Links Table:
• Key: a_id or b_id
• Value: real_object_id

Partitioning:

• Partition observations by system:object_id to distribute load evenly.

5. Deep dive

The core challenge is efficiently linking IDs from two systems and retrieving observations across these links. The add_link operation updates the internal mapping to ensure that any subsequent get_history call retrieves observations from both systems.

sequenceDiagram
    participant Client
    participant ObjectTracker
    participant InMemoryStore

    Client->>ObjectTracker: add_link(a_id, b_id)
    ObjectTracker->>InMemoryStore: Store link (a_id, b_id)
    
    Client->>ObjectTracker: add_observation(observation)
    ObjectTracker->>InMemoryStore: Store observation
    
    Client->>ObjectTracker: get_history(system, object_id)
    ObjectTracker->>InMemoryStore: Retrieve linked object IDs
    InMemoryStore-->>ObjectTracker: Return linked IDs
    ObjectTracker->>InMemoryStore: Retrieve observations for all linked IDs
    InMemoryStore-->>ObjectTracker: Return observations
    ObjectTracker-->>Client: Return ordered observations

6. Scale, bottlenecks & trade-offs

Replication & Sharding:

• Use Redis clustering to distribute data across multiple nodes, ensuring high availability and fault tolerance.

Caching:

• Cache frequently accessed observation histories to reduce read latency.

Single Points of Failure:

• Redis clustering mitigates single points of failure by providing data replication and failover capabilities.

Trade-offs:

• Consistency vs. Availability: Prioritize consistency in linking operations to ensure accurate history retrieval. This may slightly impact availability during link updates.
• Memory Usage vs. Performance: Storing all data in-memory offers fast access but requires careful management of memory resources. Consider eviction policies for stale data.
• Push vs. Pull: Observations are pushed to the system, while history retrieval is a pull operation. This design balances real-time updates with efficient querying.

By structuring the system to handle identity resolution separately from observation storage, we ensure that linking operations do not block observation recording, maintaining high throughput and low latency.

Model answer

1. Requirements & scale

Functional Requirements:

• Real-time tracking of autonomous vehicle locations.
• Ability to handle location updates from a large fleet of vehicles.
• Provide location data to clients with minimal latency.
• Ensure data accuracy and consistency.

Non-Functional Requirements:

• High availability and fault tolerance.
• Scalability to accommodate an increasing number of vehicles.
• Low latency for real-time updates.
• Secure data transmission.

Estimates:

• Assume 10,000 autonomous vehicles, each sending location updates every second.
• Location update size: ~200 bytes (latitude, longitude, timestamp, vehicle ID).
• QPS (Queries Per Second): 10,000 updates/second.
• Bandwidth: 10,000 vehicles * 200 bytes = 2 MB/s.
• Storage: 10,000 vehicles 200 bytes 60 seconds 60 minutes 24 hours = ~17.28 GB/day.

2. High-level architecture

flowchart TD
    subgraph Client
        A[Mobile App]
        B[Web App]
    end

    subgraph Edge/CDN
        C[CDN]
    end

    subgraph Load Balancer
        D[Load Balancer]
    end

    subgraph API / Services
        E[Location Update Service]
        F[Location Query Service]
    end

    subgraph Cache
        G[Redis Cache]
    end

    subgraph Datastores
        H["Time-series DB (InfluxDB)"]
        I["SQL DB (PostgreSQL)"]
    end

    subgraph Message Queue
        J["Kafka"]
    end

    subgraph Workers
        K[Data Processor]
    end

    A --> C
    B --> C
    C --> D
    D --> E
    D --> F
    E --> J
    F --> G
    G --> F
    J --> K
    K --> H
    K --> I

3. API design

• POST /vehicles/{vehicleId}/location: Receive location updates from vehicles.
• GET /vehicles/{vehicleId}/location: Retrieve the latest location of a specific vehicle.
• GET /vehicles/nearby: Retrieve vehicles within a certain radius of a given location.

4. Data model & storage

Datastores:

• Time-series DB (InfluxDB): For storing historical location data due to its efficiency in handling time-series data.
• SQL DB (PostgreSQL): For storing vehicle metadata and current status.

Key Tables:

• VehicleLocations: (vehicleId, latitude, longitude, timestamp) - Partitioned by time for efficient querying.
• Vehicles: (vehicleId, status, lastKnownLocation) - Indexed by vehicleId for quick access.

5. Deep dive

The core of this system is efficiently processing and storing real-time location updates while ensuring low-latency access to this data.

sequenceDiagram
    participant V as Vehicle
    participant E as Location Update Service
    participant J as Kafka
    participant K as Data Processor
    participant H as Time-series DB
    participant G as Redis Cache

    V->>E: POST /vehicles/{vehicleId}/location
    E->>J: Publish location update
    J->>K: Consume location update
    K->>H: Store in Time-series DB
    K->>G: Update Redis Cache

1. Location Update Service receives updates from vehicles and publishes them to a Kafka topic.
2. Data Processor consumes updates from Kafka, storing them in a time-series database and updating the cache for quick access.
3. Redis Cache holds the latest location data for fast retrieval by the Location Query Service.

6. Scale, bottlenecks & trade-offs

Scalability:

• Use Kafka for decoupling producers and consumers, allowing the system to scale horizontally.
• Partition the time-series database by time to handle large volumes of data efficiently.

Bottlenecks:

• Network latency can affect real-time updates; use CDNs and edge servers to reduce latency.
• Ensure that the cache is updated frequently to maintain data freshness.

Trade-offs:

• Consistency vs. Availability: Prioritize availability and eventual consistency for location data to ensure the system remains responsive.
• Push vs. Pull: Use a push model for vehicle updates to reduce server load and latency.
• SQL vs. NoSQL: Use a hybrid approach with SQL for metadata and NoSQL (time-series DB) for location data to balance consistency and scalability.

By leveraging a microservices architecture with appropriate data stores and caching strategies, this system can efficiently track and provide real-time location data for autonomous vehicles.

Model answer

To estimate the total number of people who would show up across all 10,000 reservations, we can use the Horvitz-Thompson estimator, which is suitable for size-biased sampling. This estimator allows us to account for the varying probabilities of selection based on reservation size.

Steps to Estimate Total Show-Up

1. Define the Sampling Probability: - For each reservation \( j \), the probability of being sampled is proportional to its reserved party size \( R_j \). - Let \( \pi_j \) be the probability that reservation \( j \) is included in the sample. We have: \[ \pi_j = \frac{R_j}{\sum_{k=1}^{10000} R_k} \] - Since we have a sample size of 1,000 out of 10,000, the actual probability of including a reservation in the sample is: \[ \pi_j = \frac{1000 \times R_j}{\sum_{k=1}^{10000} R_k} \]
2. Apply the Horvitz-Thompson Estimator: - For each sampled reservation \( i \), calculate the contribution to the total show-up estimate: \[ \hat{S}_i = \frac{S_i}{\pi_i} \] - The estimated total number of people who show up, \( T_S \), is the sum of these contributions: \[ T_S = \sum_{i=1}^{1000} \hat{S}_i \]
3. Assumptions: - The sample is representative of the population in terms of the relationship between \( R \) and \( S \). - The sum of reserved sizes \( \sum_{k=1}^{10000} R_k \) is known or can be estimated accurately.
4. Quantifying Uncertainty: - To quantify uncertainty, we can calculate a confidence interval for the estimate \( T_S \). - Use the variance of the Horvitz-Thompson estimator: \[ \text{Var}(T_S) = \sum_{i=1}^{1000} \left( \frac{S_i^2}{\pi_i^2} \right) \left( 1 - \pi_i \right) \] - The standard error is the square root of this variance. - Construct a confidence interval using the normal approximation: \[ \text{CI} = \left( T_S - z \times \text{SE}, T_S + z \times \text{SE} \right) \] - Here, \( z \) is the z-score corresponding to the desired confidence level (e.g., 1.96 for 95% confidence).

By following this approach, we can estimate the total number of people who will show up across all reservations, accounting for the size-biased sampling and quantifying the uncertainty of our estimate.

Model answer

1. Appropriate Metrics for Rare-Event Detection

• Precision and Recall: In rare-event detection, precision (the proportion of true positives among all positive predictions) and recall (the proportion of true positives among all actual positives) are crucial. They provide a clear picture of how well the model identifies rare events without being overwhelmed by false positives.
• F1 Score: This is the harmonic mean of precision and recall. It balances the trade-off between precision and recall, which is particularly important when dealing with rare events where one metric might dominate the other.
• Why Not Accuracy/ROC-AUC Alone: Accuracy can be misleading in rare-event settings because a model can achieve high accuracy by simply predicting the majority class (negatives). ROC-AUC, while useful, can be less informative in skewed datasets because it considers both true positive and false positive rates, which might not reflect the model's performance on the rare positive class.

2. Comparing Models with Statistical Uncertainty

• Distributions and Assumptions:
• Use a binomial distribution to model the number of true positives and false positives, given the small number of events.
• Assume that the predictions from both models are independent unless otherwise stated (unpaired evaluation).
• Confidence Intervals:
• Compute confidence intervals for precision and recall using a binomial proportion confidence interval (e.g., Wilson score interval) to account for the small sample size.
• Significance Testing:
• Perform a McNemar's test if the models are evaluated on the same dataset (paired evaluation) to test if there is a significant difference in their performance.
• For unpaired evaluations, consider using a bootstrap method to estimate the variance and compare the models' performance metrics.

3. Decision-Making with Limited Data

• Thresholds and Calibration:
• Use a fixed threshold that balances precision and recall based on the specific cost of false positives vs false negatives.
• Consider calibrating the model outputs to improve the reliability of probability estimates, especially if the models are overconfident or underconfident.
• Cost and Alert-Budget Considerations:
• Define a cost matrix that quantifies the impact of false positives and false negatives. This helps in setting a threshold that minimizes expected cost.
• Establish an alert budget that limits the number of false positives to a manageable level, ensuring that alerts remain actionable.
• Assumptions:
• Assume a fixed threshold for evaluation if only a few numbers are available, as this simplifies comparison.
• If possible, gather additional data or perform simulations to better understand the models' behavior under different scenarios.

By focusing on precision, recall, and F1 score, and using statistical methods to account for uncertainty, you can make a more informed decision about which model performs better in a rare-event detection context.

Model answer

Primary Differences Between a List and a Set in Python

1. Order: - List: Maintains the order of elements. Elements can be accessed by their index. - Set: Does not maintain any order. Elements are stored in an unordered fashion, and there is no index-based access.
2. Mutability: - Both lists and sets are mutable, meaning you can add, remove, or change elements after creation.
3. Duplicates: - List: Allows duplicate elements. You can have multiple occurrences of the same value. - Set: Does not allow duplicate elements. Each element must be unique.
4. Performance: - List: Generally slower for membership tests (checking if an element is in the list) because it requires a linear search. - Set: Optimized for membership tests using a hash table, making it significantly faster for this purpose.
5. Use Cases: - List: Ideal when you need to maintain the order of elements or require duplicate entries. - Set: Best used when you need to ensure all elements are unique and order is not important.
6. Syntax: - List: Created using square brackets [] or the list() constructor. - Set: Created using curly braces {} or the set() constructor. Note that an empty set must be created with set() because {} creates an empty dictionary.

Example Code

# List example
my_list = [1, 2, 2, 3, 4]
print(my_list)  # Output: [1, 2, 2, 3, 4]

# Set example
my_set = {1, 2, 3, 4}
print(my_set)  # Output: {1, 2, 3, 4}

# Adding elements
my_list.append(5)
my_set.add(5)

# Checking membership
print(3 in my_list)  # Output: True
print(3 in my_set)   # Output: True

# Removing elements
my_list.remove(2)  # Removes the first occurrence of 2
my_set.remove(2)   # Removes 2 from the set

Complexity:

• List: Access and membership test operations are O(n).
• Set: Membership tests and add/remove operations are average O(1) due to hashing.

Model answer

Role of Lidar in Waymo's Mapping and Navigation

1. High-Resolution Mapping - Lidar (Light Detection and Ranging) is crucial in creating high-resolution 3D maps. It emits laser beams and measures the time it takes for each beam to reflect back, allowing the system to calculate precise distances to surrounding objects. - These measurements help construct detailed 3D models of the environment, capturing features like road edges, lane markings, and static obstacles with high accuracy.
2. Real-Time Navigation and Obstacle Detection - During navigation, Lidar continuously scans the environment to detect dynamic obstacles such as pedestrians, vehicles, and cyclists. - The real-time data from Lidar is compared against pre-existing maps to identify any changes or new obstacles, ensuring the vehicle can navigate safely and efficiently.
3. Localization - Lidar assists in vehicle localization by comparing the real-time Lidar scans with the high-resolution maps. This helps the vehicle determine its precise position on the map, even in challenging conditions like poor GPS signal areas. - This precise localization is essential for maintaining lane position and executing maneuvers accurately.
4. Environmental Understanding - Lidar provides a comprehensive understanding of the vehicle's surroundings, including the detection of road signs, traffic lights, and other critical infrastructure. - The 360-degree field of view offered by Lidar sensors allows the vehicle to perceive its environment in all directions, enhancing situational awareness.
5. Redundancy and Safety - Lidar adds a layer of redundancy to the perception system, complementing other sensors like cameras and radar. This redundancy is vital for ensuring safety, as it allows the vehicle to cross-verify information from multiple sources. - In scenarios where visual data may be compromised (e.g., low light or adverse weather), Lidar provides reliable distance and shape information.
6. Data Fusion - The data from Lidar is fused with inputs from other sensors to create a comprehensive model of the environment. This fusion process helps in making informed decisions regarding path planning and obstacle avoidance. - Advanced algorithms process this data to predict the behavior of moving objects, enhancing the vehicle's ability to navigate complex urban environments.

By integrating Lidar technology, Waymo enhances the accuracy, safety, and reliability of its autonomous vehicles, enabling them to navigate complex environments with a high degree of precision.