Essays on this site

  1. Why I Mostly Switched From Claude Code to the Codex Desktop App

    How the Codex desktop app became my default coding workspace, and where Claude Code still wins.

  2. Notes on the Claude Code 2.1.0 outage

    A changelog formatting change took down Claude Code. Lessons about parsing human docs as machine data.

  3. Notes on the Eurostar chatbot “vulnerability” report

    Looking at what makes something a vulnerability versus a hardening opportunity in LLM applications.

  4. What I learned shipping 1,000+ PRs with Claude Code

    Notes from using Claude Code in parallel git worktrees: Plan Mode, ultrathink, verification loops, and Chrome automation.

Selected writing elsewhere

  1. McKinsey’s Lilli Looks More Like an API Security Failure Than a Model Jailbreak

    Why the reported Lilli incident looks like an application-security chain reaching an AI system, not a model jailbreak.

    (opens in a new tab)
  2. Promptfoo is joining OpenAI

    Announcing that Promptfoo has agreed to be acquired by OpenAI.

    (opens in a new tab)
  3. How AI Regulation Changed in 2025

    Why “AI compliance questions” appeared in security questionnaires and RFPs, and how policy becomes contract requirements.

    (opens in a new tab)
  4. Why Attack Success Rate (ASR) Isn’t Comparable Across Jailbreak Papers

    ASR isn’t portable across papers because measurement choices dominate the headline number. Includes math and a checklist for reading papers.

    (opens in a new tab)
  5. GPT-5.2 Initial Trust and Safety Assessment

    Day-zero red team of GPT-5.2 focusing on jailbreak resilience and harmful content.

    (opens in a new tab)
  6. Real-Time Fact Checking for LLM Outputs

    Introduces search-rubric, an assertion where a search-enabled judge verifies time-sensitive claims during evals and CI.

    (opens in a new tab)
  7. When AI becomes the attacker: The rise of AI-orchestrated cyberattacks

    Connects malware querying LLMs at runtime with “vibe hacking” case studies. Defense needs continuous testing.

    (opens in a new tab)
  8. Reinforcement Learning with Verifiable Rewards Makes Models Faster, Not Smarter

    RLVR gains are often “search compression” rather than new reasoning ability.

    (opens in a new tab)
  9. Prompt Injection vs Jailbreaking: What’s the Difference?

    Jailbreaking targets model safety training; prompt injection targets application trust boundaries.

    (opens in a new tab)
  10. AI Safety vs AI Security in LLM Applications: What Teams Must Know

    Safety protects people from harmful outputs; security protects systems from adversarial manipulation.

    (opens in a new tab)
  11. Promptfoo Raises $18.4M Series A

    Announcing our Series A led by Insight Partners with participation from a16z.

    (opens in a new tab)
  12. Evaluating political bias in LLMs

    Open methodology and dataset (2,500 political statements) to measure political leaning in models.

    (opens in a new tab)
  13. Celebrating 100,000 Users

    Promptfoo’s journey from prompt evaluation to AI red teaming, marking 100,000 users.

    (opens in a new tab)

Guides

  1. Testing Humanity’s Last Exam with Promptfoo

    Guide on using Promptfoo to test the HLE benchmark.

    (opens in a new tab)