AI & ML Interview Questions 2026

Explain the bias-variance tradeoff and how you diagnose which problem you're facing in a deployed model.

Tip: Interviewers want to see you connect theory to practice: use learning curves, train/val loss gaps, and know the levers (regularization, more data, simpler model).

What are the differences between L1 and L2 regularization, and in what scenarios would you prefer one over the other?

Tip: Cover sparsity (L1 for feature selection) vs. weight shrinkage (L2). Mention Elastic Net as a middle ground.

How does gradient descent with momentum differ from Adam, and when might Adam underperform a well-tuned SGD?

Tip: Discuss adaptive learning rates, the generalization gap (Adam can converge to sharp minima), and when SGD + momentum with LR scheduling wins on large image models.

Describe the transformer self-attention mechanism mathematically. Why does it scale quadratically with sequence length, and what approaches mitigate this?

Tip: Write out Q, K, V and the softmax(QK^T/√d)V formula. Mention Flash Attention, sparse attention, and linear attention as mitigations.

Implement k-means clustering from scratch in Python. How would you choose the optimal k?

Tip: Show the update loop, centroid assignment, and convergence check. Mention elbow method, silhouette score, and BIC for choosing k.

Write a function to compute cross-entropy loss with numerical stability (avoid log(0)) and its gradient with respect to the logits.

Tip: Use the log-sum-exp trick. They're testing both numerical correctness and whether you understand the gradient flows into backprop.

Given a dataset with a severe class imbalance (1:100 positive:negative), write the training loop modifications and evaluation code you would use.

Tip: Cover weighted loss, oversampling (SMOTE), undersampling, threshold tuning, and why you'd use F1/AUC-PR over accuracy.

Design a real-time recommendation system for an e-commerce platform that must serve 100k requests/second with sub-100ms p99 latency.

Tip: Split into candidate generation (ANN index like FAISS/ScaNN) and ranking (lightweight DNN). Discuss feature stores, caching, model serving, and A/B testing infrastructure.

How would you design the ML pipeline for a fraud detection system, from data ingestion through model retraining and monitoring?

Tip: Cover streaming features (Flink/Kafka), label acquisition challenges, concept drift detection, shadow deployment, and champion/challenger patterns.

Tell me about a time a model you shipped performed well offline but degraded in production. What did you do?

Tip: Structure with STAR. Highlight data distribution shift diagnosis, monitoring, and the fix (retraining schedule, feature re-engineering, or fallback logic).

Describe a situation where you had to push back on a stakeholder's request for a specific ML approach. How did you handle it?

Tip: They want to see technical confidence + communication skills. Show you led with data (simple baselines, ROI estimate) not just intuition.

How do you stay current with the rapid pace of ML research? Give an example of a paper you read recently and how it influenced your work.

Tip: Have a specific paper ready. Show you can extract practical takeaways, not just summarize abstracts.

Explain the score-based diffusion model framework. How does DDPM relate to score matching, and what are the training and sampling trade-offs versus GANs?

Tip: Cover denoising score matching, the forward/reverse SDE, DDPM's simplified loss, and why diffusion models achieve better mode coverage than GANs.

What is the Neural Tangent Kernel, and what does it tell us about the training dynamics of wide neural networks?

Tip: Explain the infinite-width limit where training is equivalent to kernel regression. Discuss its limitations for finite-width practical networks.

Compare RLHF, RLAIF, and DPO for aligning language models. What are the theoretical and practical differences?

Tip: Discuss reward model training, KL divergence penalties, and why DPO directly optimizes the Bradley-Terry preference model without a separate RM.

Explain mixture-of-experts (MoE) architectures. What are the challenges in training sparse MoE transformers, and how does load balancing work?

Tip: Describe the gating network, top-k routing, auxiliary load balancing loss, and expert capacity buffers. Reference Mixtral or GPT-4 MoE design.

Implement a simple version of the attention mechanism in PyTorch including scaled dot-product attention and multi-head attention.

Tip: Show masking, the scaling factor, and how heads are split/merged. Bonus: mention torch.nn.functional.scaled_dot_product_attention for Flash Attention.

Write code to evaluate a generative model using FID (Fréchet Inception Distance). What does FID measure and what are its limitations?

Tip: Explain that FID computes Fréchet distance between Inception feature distributions. Limitations: Inception bias, sensitivity to sample count, doesn't catch mode collapse.

Design an experiment to rigorously compare two LLM fine-tuning strategies. How do you control for confounds and ensure your results generalize?

Tip: Cover dataset splits, evaluation benchmarks (not just perplexity), statistical significance, multiple random seeds, and diverse task coverage.

How would you set up a scalable distributed training infrastructure for a 70B parameter model? Walk through your parallelism strategy.

Tip: Discuss tensor parallelism, pipeline parallelism, and data parallelism (3D parallelism). Mention ZeRO stages, gradient checkpointing, and mixed precision.

Walk me through your most significant research contribution. What was the core insight, and what would you do differently in hindsight?

Tip: Clarity and intellectual honesty matter here. Show you can articulate the insight at multiple abstraction levels and that you reflect critically on your own work.

How do you approach a research problem where you have a strong intuition but no theoretical justification yet?

Tip: Show empirical discipline: ablations, control experiments, trying to falsify your own hypothesis before spending compute.

Describe a time your research hypothesis was proven wrong. How did you respond, and what did you learn?

Tip: Intellectual humility is valued. Show you pivoted quickly, documented the failure, and extracted useful signal from the negative result.

What is Simpson's paradox? Give a concrete example and explain how you would detect and handle it in an A/B test analysis.

Tip: Use a concrete example (e.g., treatment appears effective overall but negative in every subgroup). Resolution: stratified analysis, regression adjustment for confounders.

Explain p-hacking and the multiple comparisons problem. How do you control false discovery rate in a large-scale experiment?

Tip: Cover Bonferroni, Benjamini-Hochberg FDR, and sequential testing (always-valid inference). Emphasize pre-registering hypotheses.

Compare random forests and gradient boosting machines. In what cases does XGBoost outperform a well-tuned random forest?

Tip: RF: parallel, low correlation between trees. GBM: sequential, lower bias. XGBoost wins when signal-to-noise is high and you can tune learning rate + depth.

What is causal inference and when is it more appropriate than standard ML prediction? Explain the potential outcomes framework.

Tip: Cover the SUTVA assumption, ATE/ATT, propensity score matching, difference-in-differences, and when to use instrumental variables vs. RCTs.

Write a Python function to perform a two-sample t-test from scratch (without scipy), then show how you'd use it in an A/B test context with proper effect size reporting.

Tip: Show the pooled standard error, t-statistic, and degrees of freedom. They want to see you understand what the test actually computes, not just call a library.

Given a DataFrame with missing values, outliers, and mixed types, write the preprocessing pipeline you would apply before modeling. Justify each step.

Tip: Cover imputation strategy (MCAR vs. MAR vs. MNAR), outlier treatment (winsorizing vs. removal), encoding, and why you fit transforms on train only.

Design a customer churn prediction system for a SaaS company. Cover the full lifecycle from problem framing to business impact measurement.

Tip: Start with the business metric (NRR, LTV). Cover label definition (30/60/90 day churn), feature engineering, model choice, threshold optimization, and post-deployment measurement.

How would you design an experimentation platform that supports 100 simultaneous A/B tests without interference effects?

Tip: Discuss mutual exclusion, holdout groups, hash-based assignment, network effects/SUTVA violations, and switchback designs for marketplace experiments.

Tell me about an analysis you did that changed a product or business decision. How did you communicate uncertainty to non-technical stakeholders?

Tip: Show you can distill statistical nuance into business language. Confidence intervals > p-values when talking to product managers.

Describe a time you discovered your dataset had a serious data quality issue mid-project. What did you do?

Tip: Show systematic debugging (distribution checks, row-level audits), proactive communication, and that you fixed the root cause rather than just the symptom.

How do you scope a data science project when requirements are ambiguous? Walk me through your process.

Tip: Discuss clarifying the business question, defining success metrics upfront, building a quick baseline first, and agreeing on the 'good enough' threshold before deep work.

Explain Retrieval-Augmented Generation (RAG). What are the key failure modes, and how would you debug a RAG pipeline that gives factually incorrect answers?

Tip: Separate retrieval failures (wrong chunks) from generation failures (correct chunks, wrong answer). Debug with retrieved-context evaluation before end-to-end eval.

Compare full fine-tuning, LoRA, QLoRA, and prefix tuning. When would you choose each, and what are the memory/compute trade-offs?

Tip: LoRA: low-rank adapters, 90%+ parameter reduction. QLoRA: 4-bit quantized base + LoRA. Prefix tuning: task-specific soft tokens, good for multi-task. Full FT: maximum capacity but expensive.

What is speculative decoding, and how does it reduce LLM inference latency? What are its limitations?

Tip: A small draft model generates k tokens; the large model verifies in parallel. Speedup depends on acceptance rate (draft quality) and how often it matches the target distribution.

Explain the difference between semantic search with dense embeddings and BM25 sparse retrieval. When would you use a hybrid approach?

Tip: Dense: captures meaning, poor at exact keyword match. BM25: great for rare keywords, no semantic generalization. Hybrid (RRF or learned combination) wins in practice.

Implement a basic RAG pipeline in Python using any embedding model and vector store. Include chunking strategy, retrieval, and prompt construction.

Tip: Show chunk size decisions (512-1024 tokens with overlap), embedding call, top-k retrieval, and a grounded prompt template. Interviewers want to see you think about chunk quality.

Write a structured output extraction system using an LLM that reliably returns valid JSON even when the model occasionally fails to follow the schema.

Tip: Use JSON mode / function calling where available, add retry logic, validate with Pydantic, and implement a fallback extraction prompt for repair.

Given an LLM-based agent that must use tools (web search, code execution, calculator), implement the agent loop with proper error handling and token budget management.

Tip: Show tool definitions, the parse-tool-call / execute / inject-result loop, max-step guard, and graceful degradation when a tool fails.

Design a production RAG system for a legal document Q&A product. The corpus has 10M documents and must return answers with citations in under 2 seconds.

Tip: Cover corpus preprocessing, chunking + metadata tagging, embedding model choice, vector DB (Pinecone/Weaviate/pgvector), reranking (cross-encoder), prompt with citations, and eval with RAGAS.

How would you design an LLM evaluation framework to measure and track quality regressions across model versions?

Tip: Distinguish automated metrics (ROUGE, BERTScore, LLM-as-judge) from human eval. Discuss golden datasets, regression test suites, and the pitfalls of LLM-as-judge bias.

Describe a GenAI feature or product you shipped. What was harder than expected and how did you handle it?

Tip: Common hard parts: eval setup, hallucination mitigation, latency, and prompt brittleness. Show you iterated systematically and measured improvement quantitatively.

How do you approach prompt engineering for a production system where reliability matters more than peak performance?

Tip: Show you treat prompts as code: version control, regression tests, few-shot example curation, and deliberate few-shot ordering experiments.

What is data drift versus concept drift? How do you detect each in a deployed model, and what automated responses do you trigger?

Tip: Data drift: input distribution shifts (PSI, KS test). Concept drift: p(y|x) changes. Responses: alert only, auto-retrain, or fallback to simpler rule-based model.

Explain the differences between shadow deployment, canary deployment, and blue-green deployment for ML models. When would you use each?

Tip: Shadow: compare outputs without serving live traffic. Canary: incremental traffic shift. Blue-green: instant cutover with quick rollback. Shadow is best for high-risk changes.

What are the challenges of feature stores in ML systems, and how do they solve training-serving skew?

Tip: Training-serving skew occurs when offline and online feature computation diverge. A feature store unifies the transformation code and provides point-in-time correct lookups.

Write a Docker + FastAPI setup for serving a scikit-learn model with a /predict endpoint, /health check, and proper logging.

Tip: Show model loading at startup (not per-request), input validation with Pydantic, structured JSON logging, and a liveness probe endpoint.

Implement a simple model monitoring script that computes Population Stability Index (PSI) between a reference dataset and a new batch, and triggers an alert if PSI > 0.2.

Tip: Show the binning logic, PSI formula (sum of (A-E)*ln(A/E)), and how you'd integrate this into a batch monitoring job with alerting.

Design the ML platform for a company that trains 50 models per week, with different team owners, frameworks (PyTorch, sklearn, XGBoost), and SLAs.

Tip: Cover experiment tracking (MLflow/W&B), model registry, automated retraining triggers, multi-framework serving (Triton/BentoML), and per-team resource quotas.

How would you design a low-latency model serving system for an online ad ranking model that must respond in under 10ms at 500k QPS?

Tip: Discuss model quantization (INT8), TorchScript/ONNX, Triton Inference Server, batching strategy, GPU vs. CPU trade-off at sub-10ms, and caching frequently-seen user embeddings.

Tell me about a production ML incident you led the response to. How did you triage it, and what did you change to prevent recurrence?

Tip: Show a blameless postmortem mindset. Cover detection lag, root cause, mitigation, and the monitoring or process change you implemented.

How do you build alignment between ML engineers (who want fast iteration) and platform engineers (who want stability)?

Tip: Show you've brokered tradeoffs: experiment environments, clear promotion gates, and SLA tiers so teams know when 'move fast' is OK vs. not.

Describe how you approach reducing technical debt in an ML codebase that has grown organically.

Tip: Show prioritization (debt that blocks safety/reliability first), incremental refactoring, and how you measure success (build time, deploy frequency, incident rate).

Explain how modern object detection models like YOLO v8 and DETR differ architecturally. What are the trade-offs in terms of speed, accuracy, and training complexity?

Tip: YOLO: single-stage anchor-based, fast inference. DETR: Transformer-based, end-to-end no NMS, slower to train but no hand-engineered anchors. Discuss Hungarian matching loss in DETR.

What is the role of non-maximum suppression (NMS) in object detection, and why are transformer-based detectors able to remove it?

Tip: NMS resolves duplicate predictions using IoU thresholds. DETR uses bipartite matching so each object query can only match one ground truth, eliminating duplicates by design.

Explain the Vision Transformer (ViT) architecture. How does it handle the inductive biases that CNNs have built in, and when does ViT outperform CNNs?

Tip: ViT patches vs. CNN receptive fields. ViT lacks translation equivariance and locality — it needs more data or pretraining. Outperforms CNNs at scale (large datasets, large model).

How does instance segmentation differ from semantic segmentation? Compare Mask R-CNN with SAM (Segment Anything Model).

Tip: Semantic: per-pixel class. Instance: per-object mask. Mask R-CNN extends Faster R-CNN with a mask head. SAM uses prompt-driven zero-shot segmentation with a ViT backbone.

Implement Intersection over Union (IoU) for axis-aligned bounding boxes and use it to implement a basic NMS function in NumPy.

Tip: Clean vectorized implementation is expected. Show the intersection area computation, then the suppression loop sorting by confidence score.

Write a PyTorch data augmentation pipeline for a medical imaging classification task where you must be careful not to introduce unrealistic artifacts.

Tip: Discuss which augmentations are domain-safe (flip, small rotation, brightness) vs. dangerous (heavy distortion, hue shift). Use torchvision.transforms or Albumentations.

Design a real-time defect detection system for a manufacturing line running at 200 frames/second. Latency must be under 5ms per frame.

Tip: Cover edge GPU deployment (Jetson), model quantization, pipelining (async preprocessing), TensorRT optimization, and how to handle false-positive/false-negative costs asymmetrically.

Describe a computer vision project where your model performed well in the lab but struggled in the real-world deployment environment. What caused this gap and how did you fix it?

Tip: Common causes: lighting changes, camera FOV differences, data distribution shift. Show you diagnosed the domain gap systematically and applied domain adaptation or data collection.

How do you approach dataset curation and annotation quality control for a CV project?

Tip: Cover annotator agreement metrics (Cohen's kappa), consensus labeling, active learning for hard cases, and how annotation errors propagate into model behavior.

Compare BERT-style masked language modeling with GPT-style causal language modeling. In what tasks does each architecture excel, and why?

Tip: BERT: bidirectional context, great for classification, NER, QA. GPT: unidirectional, optimized for generation. Encoder-decoder (T5) for seq2seq tasks. Discuss the pre-training objective effect on learned representations.

What is subword tokenization, and why do BPE, WordPiece, and SentencePiece exist? What are the failure modes of tokenization for certain languages or domains?

Tip: Subword balances vocabulary size vs. OOV. BPE merges frequent pairs; WordPiece maximizes LM likelihood. Problems: agglutinative languages (Finnish, Turkish), code, numbers, non-Latin scripts.

Explain Named Entity Recognition (NER) as a sequence labeling problem. How do you handle entities that span multiple tokens, and how does the BIO tagging scheme work?

Tip: BIO: B-begin, I-inside, O-outside. Multi-token spans tracked by B→I transitions. Alternatives: BIOES. Modern approach: fine-tune BERT with a token classification head, CRF layer optional.

What are the key challenges in building a high-quality text classification system for a domain with limited labeled data?

Tip: Discuss zero-shot/few-shot prompting, SetFit (few-shot fine-tuning with contrastive learning), data augmentation (back-translation, EDA), and active learning.

Implement TF-IDF from scratch and use it to build a simple document retrieval system. Then explain why dense embeddings outperform it for semantic queries.

Tip: Show term frequency, inverse document frequency, and cosine similarity retrieval. Dense embeddings beat TF-IDF on paraphrase / semantic similarity but TF-IDF wins on rare exact-match terms.

Write a preprocessing pipeline for fine-tuning a BERT model on a text classification task, including tokenization, batching with dynamic padding, and attention masks.

Tip: Use HuggingFace tokenizer with padding='longest' in the collator. Show attention mask handling and why you pad per batch not globally.

Design a multilingual customer support intent classification and entity extraction system that handles 40 languages and 500 intents.

Tip: Use a multilingual backbone (mBERT, XLM-R, or mDeBERTa). Discuss zero-shot cross-lingual transfer vs. language-specific fine-tuning, taxonomy design, and confidence thresholding for human handoff.

Tell me about a time you built an NLP system that worked well in English but failed in another language or dialect. What did you learn?

Tip: Show awareness of linguistic diversity (morphology, script, code-switching). Discuss how you diagnosed the gap and what transfer learning or data collection strategy you used.

How do you evaluate whether a text summarization model is actually producing better summaries? What limitations do you see with ROUGE?

Tip: ROUGE measures n-gram overlap but misses factual accuracy and fluency. Better: BERTScore, QA-based faithfulness evaluation, and human eval with specific rubrics (conciseness, relevance, faithfulness).

Describe your approach to handling PII and sensitive information in an NLP pipeline.

Tip: Cover regex + NER-based detection, tokenization masking, differential privacy in training, data retention policies, and downstream model audit to ensure PII isn't memorized.

You don't need to write code, but explain precision and recall to a non-technical stakeholder. Why do they matter when defining your AI product's success metrics?

Tip: Use a medical diagnosis analogy (false negatives vs. false positives). Show you understand the precision-recall tradeoff and can connect it to business impact (missed revenue vs. false alerts).

What is hallucination in an LLM context, and how would you design product guardrails and user experiences to mitigate its impact?

Tip: Discuss citation grounding, confidence signals in UI, human-in-the-loop review flows, user education, and feedback mechanisms that help you detect hallucinations in production.

What are the product tradeoffs between a fine-tuned proprietary model and an off-the-shelf API like GPT-4 or Claude?

Tip: API: fast to market, vendor dependency, data privacy concerns, recurring cost. Fine-tuned: latency control, cost at scale, domain adaptation, but ops burden. Show you can frame this as a build-vs-buy decision.

How would you design the feedback loop and continuous improvement system for a generative AI feature in a consumer product?

Tip: Cover explicit feedback (thumbs up/down), implicit signals (copy, share, session length), evaluation pipeline, A/B testing new model versions, and a staged rollout strategy.

Design the product specification and launch criteria for an AI writing assistant for enterprise customers. How do you handle the responsible AI requirements?

Tip: Cover use case clarity, accuracy / hallucination requirements, data privacy SLAs, bias testing across user demographics, admin controls, auditability, and regulatory considerations (EU AI Act).

Describe a situation where an AI feature you shipped had unintended consequences for users. What did you do?

Tip: Show you have mechanisms to detect harm early (monitoring, user reports), can make a fast product decision (kill switch, tuning), and did a blameless retrospective.

How do you prioritize the AI roadmap when engineering capacity is limited but stakeholder requests for AI features are high?

Tip: Show a framework: business impact x confidence x effort. Discuss why 'add AI' requests without a clear problem statement get deprioritized and how you educate stakeholders.

How do you work effectively with ML engineers who have different intuitions about what's technically feasible?

Tip: Show collaborative dynamic: you scope the 'what' and 'why', they own the 'how'. Reference a time you changed a product requirement based on an engineer's technical insight.

Tell me about an AI product idea you evaluated and decided NOT to build. What was your reasoning?

Tip: Show rigorous evaluation: problem-solution fit, data availability, model maturity, user trust, regulatory risk. Saying no clearly is a PM superpower.