Devlin Liles

The Webcam Light Question Nobody Scripted For Nobody scripted an answer for the webcam light question, so the assistant improvised. Plan for trust questions before launch. Worth a read: devlinliles.com/the-webcam-l… #AIUX #Trust

The View That Forgets SSN: Withholding Sensitive Columns at the Source Masking a column in the prompt is a request. A view that never exposes it is a guarantee. Withhold at the source. Dive in: devlinliles.com/the-view-tha… #DataSecurity #AIGovernance

The Untagged Aside: Why Source Identity Has to Be Captured at Ingestion An untagged aside gets ingested and later surfaces as fact. Capture source identity at ingestion. You can’t rebuild it later. Full post: devlinliles.com/the-untagged… #RAG #DataGovernance

The Tone-Polish Tax: Gating a New Step by What It Actually Buys Every pipeline step adds latency and cost. Gate a new one on what it measurably buys, not how it feels. Check it out: devlinliles.com/the-tone-pol… #AIEngineering #Cost

The Tokenize-and-Restore Sandwich: A Proxy That Never Sees the Real Value Tokenize the sensitive value going out, restore it coming back. The proxy in the middle never sees the real data. Dive in: devlinliles.com/the-tokenize… #DataPrivacy #LLM

The Token Scoped to Exactly One Employee ID A shared token can read everyone’s records. Scope the token to one employee ID and the boundary lives in the credential, not the prompt. Worth a read: devlinliles.com/the-token-sc… #AIAgents #Security

The Time-of-Decision Log: Scoring Judgment Before the Outcome Arrives Outcomes alone don’t grade a decision. Log what was known and chosen at decision time so you can score judgment early. Read it here: devlinliles.com/the-time-of-… #AIGovernance #Audit

The Tie Check: Refusing to Guess When the Evidence Splits Evenly When the evidence splits evenly, don’t let the system pick. Detect the tie and route it to a human. See it here: devlinliles.com/the-tie-chec… #AIAgents #HumanInTheLoop

The Ten-Run Rule: Telling a Real Regression From Ordinary Noise One bad run is not a regression. Outputs vary. Run it ten times before you roll back or blame anyone. Dive in: devlinliles.com/the-ten-run-… #AIEvals #Testing

The Suppression Test: Verifying What the Model Says Caused It A model’s stated cause is a hypothesis. Suppress it and see if the effect goes away. Explanations that survive are worth keeping. Full post: devlinliles.com/the-suppress… #Explainability #LLM

The Sprint That Was Sized for the Wrong Stage Estimates fail when the sprint is sized for the wrong project stage. Match the plan to the stage’s risk and pace. Check it out: devlinliles.com/the-sprint-t… #AIDelivery #ProjectManagement

The Sixty-Day Gap Between a Model Swap and Anyone Noticing The model changed. Nobody noticed for sixty days because nothing broke, quality just moved. Measure behavior, not uptime. Worth a read: devlinliles.com/the-sixty-da… #MLOps #AIOps

The Six-Suitcase Problem: Letting Failure Set Decomposition Depth Don’t pick decomposition depth up front. Start coarse, watch where it fails, and split only there. Dive in: devlinliles.com/the-six-suit… #AIAgents #Architecture

The Signal That Beats the Label by a Month A label confirms what already happened. An upstream signal moved a month earlier. Find it, validate it, and act on it first. Read the post: devlinliles.com/the-signal-t… #MachineLearning #Analytics

The Shared ID: One Lookup Instead of Five Disconnected Logs An incident touches five systems, each with its own log and its own ID. Stamp one ID at the start of the request and five logs become one lookup. See it here: devlinliles.com/the-shared-i… #Observability #AIOps

The Self-Graded Node: Finding a Bad Step Through Arithmetic Self-grading gives confident numbers and little signal. Compare pass rates and retry counts to baseline and the bad step shows itself. Worth a read: devlinliles.com/the-self-gra… #AIAgents #Observability

The Second Reviewer Problem: Why Adding an Agent Usually Adds Nothing A second agent on the same model with the same context shares the first one’s blind spots. Agreement is not confirmation. Dive in: devlinliles.com/the-second-r… #AIAgents #AIQuality

The Schema the Model Cannot Violate Asking a model for valid JSON is a request. Constraining decoding to a schema is a guarantee. Malformed structure becomes impossible, not unlikely. Worth a read: devlinliles.com/the-schema-t… #LLM #StructuredOutput

The Schema You Add After the Test Fails A malformed output breaks a downstream test, so someone adds a schema. That covers the failure that already happened. Write it from the consumer’s contract first. Dive in: devlinliles.com/the-schema-y… #AIEngineering #Testing

The Review Queue Sets the Speed Limit on Generation Generation scales with compute. Review scales with people. Whichever is slower sets your real throughput. Size both together. Read it here: devlinliles.com/the-review-q… #GenerativeAI #HumanInTheLoop