Los Angeles, CA  ·  US Citizen

Kevin Bartolome

AI Engineer  ·  Forward Deployed Engineer  ·  QA Lead  ·  Software Developer

I build the systems that let an AI agent do real engineering work, and I keep a human gate on every decision that is hard to undo.

The story, in three steps

  1. 1
    The problem. I lead QA for a large AI product: hundreds of automated tests, a stream of failures every day, and one person triaging them by hand.
  2. 2
    What I built. An AI agent that does the triage, the fixes and the verification, and one screen that shows me only what needs a human, with three more I open when I want the story behind it. I approve the parts that are hard to undo. Nothing reaches the product without a recorded run.
  3. 3
    What happened. Most of the job now runs itself, the quality bar did not move, and the system writes down what it learns so the same mistake is not made twice. It also keeps its own books: every request, every reply, every dollar and every byte, derived from the record rather than typed. Everything below is that system, so you can read it, then open it.
FAILING TESTSdozens a day AI AGENTtriage ยท fix ยท verifyruns its own checks ONE SCREEN TO WATCHonly what needs a humanthree more hold the detail HUMAN GATEapproves what is hard to undo SIGNEDoff, for real every correction becomes a written rule and a test the agent runs on itself
1,600+end-to-end tests kept green across 13 product domains
82classes in the test framework the agent works inside
500+automated self-checks that guard the machinery itself
62s → 17sto inspect a live page, and 3-8k → ~1k tokens per investigation
30 → 2daily reading load: thirty failures became two decisions
2hand-offs left in a broken test's life, down from ten

Read the story

How it works

The Control Room

How I handed my daily work to an AI and kept a human gate on every decision that is hard to undo. Written so anyone can follow it, with one real day start to finish.

Read it →

See it running One screen you watch. Three more you open when you want the story behind it. Invented data; not screenshots, the real pages.

The one to watch · What needs me right now?

Headquarters

The one screen I have to look at: what needs a human, what the agent is doing, what just finished.

HQ
๐Ÿ‘‰ YOUR MOVE (2)
  1 Paste 3 files, then run acc-07
  2 Re-run these 9. No fix needed
โšก RIGHT NOW  fixing bil-19  3m 21s
Open it in a new tab →
Detail · Where does every broken test stand?

The Mission Board

Every test in the round on a ladder from "investigating" to "signed off", and whose turn it is.

Mission Board
โ™ป PASSED ON RE-RUN   chk-04   23m ago
๐Ÿ›ฐ PROVEN ON REAL MACHINES  acc-07
๐Ÿ”ง FIX READY  onb-05  waiting on you
๐ŸŸก INVESTIGATING  bil-19   3m
Open it in a new tab →
Detail · What is running, and did it pass?

Flow Ops

Several tests at once in their own lanes, plus every run that finished today and why it failed.

Flow Ops
โ–ถ IN FLIGHT (28)            32 lanes
  lane 1  bil-19   โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘  3m 21s
  lane 2  srch-02  โ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘  1m 04s
โœ“ acc-07 passed 6m 23s   โœ— adm-03 failed
Open it in a new tab →
Detail · What actually got done today?

The Workshop

The day's record: hand-overs, answers, how well the agent's calls held up, and what it learned.

Workshop
๐Ÿ† 143 signed off this week
โฑ Checkout batch  52 tests  55m
๐Ÿ“ˆ confident calls  41/43 correct
๐Ÿง  3 lessons became rules
Open it in a new tab →

The record it writes Four pages the system keeps about itself: what changed, what was asked and answered, what got done, what it learned. Derived from what happened, never typed by hand. Same invented data.

The record · What changed, and who asked for it?

The Upgrade Bay

Every feature, fix and change to the machinery itself, marked as mine or the agent's own suggestion, with a plain guide for each one that shipped.

Upgrade Bay
โœจ 12 shipped ยท 14 open ยท ๐Ÿค– suggested / ๐Ÿ‘ค asked
  H Runner swarm: one pod per run, never two   ๐Ÿ‘ค Kevin
  M Quick pull: merge, never overwrite         ๐Ÿค– Claude
  L Info tips on every panel                   ๐Ÿ‘ค Kevin
Open it in a new tab →
The record · What was asked, and what came back?

Telemetry

One row per request across four days, with the agent's replies, what each one produced, what it cost and what it takes up on disk.

Telemetry
โ–ค INBOX (60)                     2 waiting on you
โ–ธ here is the search and account batch   RUNNING
  why did the checkout batch fail first?  DONE
$169 today 612 calls                storage fine
Open it in a new tab →
The record · What got done, day by day?

The Logbook

A calendar of every batch handed over, every test signed off and every sync, from a whole year down to a single day.

Logbook
SEPTEMBER 2026          792 passed ยท 33 batches
  Mon 31  โ–  1 batch  โ–  27 passed  โ–  6 commits
  Tue  1  โ–  1 batch  โ–  52 passed  โ–  8 commits
  Thu  3  โ–  1 batch  โ–  38 passed  โ–  7 commits
Open it in a new tab →
The record · What did it learn, and did it stick?

Recalibration

The learning ledger: lessons captured from my corrections, which ones came back, which are now enforced by a check, and the two-week trend.

Recalibration
๐Ÿง  120 lessons ยท 7 recurred ยท 4 now mechanisms
  corrections per 100 msgs  โ–‡โ–‡โ–‡โ–†โ–…โ–„โ–ƒโ–‚  trending down
  ๐Ÿ“ prose-only, will recur       3
  โš™ enforced by a check          4
Open it in a new tab →

About me

Background

Resume

Six years across AI automation, QA leadership, test automation and full stack development. Readable here, or download the PDF.

View it →