Posts
The AI Illusion: Why feeding an LLM without data governance is a risk you can’t audit
A lawyer opens the company’s new AI assistant, uploads an exclusivity contract, and types: “Summarize the key obligations in five lines.” Ten seconds later she has a clean summary. It’s a small, real productivity win.What she doesn’t see is what happened in those ten seconds. The application didn’t just send her question. It sent the whole contract: the counterparty, the prices, the termination clauses, the signatory’s name. All of it landed, as readable text, on a server run by someone else.The same thing happens when an HR manager asks for a summary of a performance review, or a financial analyst digs into a client’s credit file. The question is short. The context that travels with it is enormous.In nearly every executive committee, the directive for 2026 is the same: “We need AI, and we need it now.” Within weeks, API keys are created, language models are wired into internal databases, and whole workflows are handed to third-party algorithms. The demos are impressive and the productivity gains are real.This article is not an argument against that. It’s an argument that the companies that get data governance right will deploy AI faster and in more places, because the foundation underneath is safe. The companies that skip it will eventually face one simple question they cannot answer. We’ll come back to it at the end.1. The Problem: Every Request Carries Your DataEven when your contract with the AI vendor forbids training on your data, and even when the connection is fully encrypted in transit, the raw text still arrives in readable form at a third party. Contracts limit what the vendor is allowed to do with it. They don’t change what is exposed if that vendor suffers a breach, if a log is misconfigured, or if a legal request lands on their desk.The risk grows with multi-agent architectures, where a router agent delegates work to specialized sub-agents that share a common memory. A single name or revenue figure injected into that shared state is now visible to every agent in the chain, and to every log, cache and tracing tool they write to.Figure 1: Injection of raw data into a multi-agent shared memory.Sending raw business data into an LLM with no controls isn’t innovation. It’s outsourcing your company’s strategic memory to a party that will never bear the cost of your leaks.2. Why Naive Anonymization FailsSome teams, aware of the danger, put a basic filter in front of the model: generic Named Entity Recognition (NER) tools, or regular expressions that redact proper nouns. On paper, that sounds reasonable. In production it tends to fail in three ways, and the real danger is that it gives leadership a sense of safety that stops them from asking questions.Failure 1: The filter doesn’t understand your businessGeneric detection tools are trained on everyday text. They confuse company names with common words (“Services”, “Transit”, “Holding”), and they often miss a company whose name contains such a word, like “NET-POLY SERVICES”. Internal identifiers such as tax codes or registration numbers don’t match standard formats, so they slip through.Failure 2: Hiding the name isn’t hiding the companyConsider this example: Raw text: “Company X posted 4.2 billion FCFA in revenue in Abidjan in 2024.” After a naive filter: “COMPANY_1 posted 4.2 billion FCFA in revenue in Abidjan in 2024.”The name is gone, but the exact revenue and the city are enough to identify the company with a single search. A modern model, with its vast training knowledge, can often do the same in a fraction of a second.Failure 3: A prompt is a request, not a controlAn instruction like “Never repeat client names” is a probabilistic nudge. The model will usually comply, until it doesn’t. There is a subtler problem too. Depending on the vendor and the settings, some models return their intermediate reasoning in a channel separate from the final answer, a kind of scratchpad handed back to you. If your protection only inspects the final answer, raw data can travel through the scratchpad unchecked.Believing a system prompt that says “be discreet” protects your data is like locking an armored door and leaving the key in the lock outside.3. The Six Invariants of Robust GovernanceIf instructions and basic filters aren’t enough, what is? Most architects sketch a linear pipeline: data comes in, gets filtered, goes to the model, comes back. A robust system looks different. It is defense in depth, where every boundary carries its own protection, independently of the others.What follows are six invariants. They are contracts that any architect can check against their own system, whatever the stack, the model vendor or the team size.Figure 2: The six governance invariants applied to LLM workflows.Invariant 1: Minimize at the sourceWhat isn’t in the returned data can’t leak. The tools exposed to AI should never be the same ones exposed to your human developers. Pagination is enforced, volumes are capped, and unnecessary fields are removed entirely. The API doesn’t return a full customer record just to compute a ratio. It returns the three numbers actually needed: no contacts, no addresses, no names.This isn’t an AI-specific rule. It’s a data-exposure rule that AI simply makes urgent.Invariant 2: Never let a model resolve what a deterministic path can decideA ranking, a ratio lookup or a simple extraction requires no judgment, so it has no business going through your most expensive and least predictable model. The routing can be a business rule, a choice in the interface or a lightweight classifier. The principle doesn’t prescribe how.Two honest caveats. First, if the router is itself a model, it is still a model: keep it small, tightly constrained, and feed it only protected text. The goal is to shrink exposure, not to claim it disappears. Second, this invariant has a benefit that often convinces a board before security does: it’s cheaper and faster.A fun example comes from Anthropic itself. In March 2026, Claude Code’s source code was accidentally leaked through a source map published on npm. It showed that the tool detects user frustration, meaning whether someone is swearing at it, with a plain regular expression, not a model1. A regex is instant and free. An inference call is neither, for a question that needs no judgment at all.Invariant 3: Treat every text-to-model boundary as a mandatory, reversible protection pointAny data crossing the boundary between your system and a language model, inbound or outbound, should be protected by a reversible mechanism such as substitution, pseudonymization or tokenization. The common mistake is protecting only the obvious response field. Check every channel the vendor exposes, including the reasoning channel described in section 2.One caution: pseudonymized data can still count as personal data under regulations like GDPR. This invariant reduces your exposure. It doesn’t remove your legal obligations.Invariant 4: Rehydrate at the last legitimate momentSubstitution should be reversed only at the very last step, right before the answer is shown to a human who is authorized to see the real values. The model works blind from start to finish. Only the human gets the answer in the clear, and only at the end of the chain.Invariant 5: A deterministic guardrail has the final say on plausibilityThe first four invariants protect confidentiality. This one protects correctness. Any response coming out of a deterministic path must pass through a non-probabilistic check, such as a threshold or a comparison with a known reference, before it goes out. If the result looks aberrant, the system falls back to a slower, supervised path rather than letting an unverified answer through.Invariant 6: Every agent-to-agent hop is its own trust boundaryThis is the invariant most often forgotten, because it doesn’t show up in a static layered diagram. An agent’s output isn’t automatically safe for the agent that called it.Here is the scenario. A sub-agent, believing it is “cleaning up its own output” before replying, de-anonymizes data that was meant to stay protected for the agent that delegated to it. The sub-agent did exactly what it was told. It just didn’t know its answer was headed for another link in the chain, one that was supposed to keep reasoning over pseudonyms.There is an aggravating corollary. When conversation history crosses several agents, turns from different agents end up mixed in one context. Without explicit provenance tagging, you can no longer tell which trust boundary applies to which piece of text.Figure 3: Agent-to-agent trust boundaries and rehydration placement.4. For Engineers: How We Implement the InvariantsThis is one possible implementation among many. The tools are specific to our stack. If your system looks nothing like ours, the invariants still hold; only their embodiment changes. Non-technical readers can skip to section 5. Anonymization gate (invariant 3): Before anything is classified, the user’s free text passes through a strict gate that combines NLP libraries with deterministic substitutions. Names and identifiers become unique tokens (for example CORP_A83B). The correspondence table goes straight into an encrypted vault: HMAC-SHA256 for the token, Fernet encryption for the real value at rest. Router (invariant 2): The pseudonymized question goes to a deliberately small, fast, low-cost LLM whose only job is to classify intent. Ranking, ratio lookup or simple extraction takes the fast path: a tool from a bounded allowlist is called directly, and the heavy model is never invoked. Risk analysis or interpretation takes the slow path and goes to the main agent. Minimized tools (invariant 1): On both paths, internal data is reached through tools exposed via the Model Context Protocol (MCP), cleaned at the source and never the same tools as those given to human developers. Slow path (invariant 3, outbound): Tool results are pseudonymized before they enter the heavy model’s context, so it reasons and writes entirely over pseudonyms. Fast path (invariant 5): A plausibility guardrail, a simple ratio compared against a reference value, blocks any aberrant automatic answer and forces a fallback to the slow path for review. Rehydration (invariant 4): Only once the response is generated and formatted does the system query the vault and swap tokens back for real values, right before display to the authorized user.Figure 4: Fast-path / slow-path routing architecture. Observed results: (Add here your real production metrics — e.g., 72% of queries served via the fast path without heavy model invocation, 4x latency reduction, and 65% API cost reduction).5. Verify, Don’t AssumeA defense that has never been verified under real conditions is a defense you only hope exists.Here is a scenario that is far from theoretical. A control meant to transform every message before it reached the model was written, reviewed and merged. The tests passed. For weeks, the team believed the protection was active. It wasn’t. An assumption about the exact shape of the incoming data was wrong, so the control ran against the wrong structure, silently, with no error and nothing to raise an alarm.A test that passes without actually going through the protection code proves nothing. A code review is not proof of execution, and a clean architecture diagram is not proof that the path it draws is the path taken at runtime. The only proof that counts is a test replayed against the real system: the real model, the real data, the real call chain.Every claim of compliance, such as “this is anonymized” or “this data never leaves in the clear,” should come with replayable proof, available on demand. Otherwise it isn’t a guarantee. It’s a hypothesis that hasn’t been disproved yet.Figure 5: The control illusion — silent failure when data format mismatches expectations.6. The Audit TrailGovernance doesn’t stop at real-time protection. It requires traceability. When a banking regulator, a GDPR auditor or a major client reviews how AI is used in your organization, they won’t accept statements of intent. They will ask for evidence.A responsible enterprise AI system keeps a tamper-evident, append-only audit log, kept separate from ordinary conversation logs and chat history. For every event in an agent’s lifecycle, it records a pair: the raw data as originally received, the data actually transmitted to external models.The rule that matters most is that no configuration flag or single administrator can turn this log off. During load spikes, a team’s first instinct is often to disable logging “for performance”. If that switch exists, then on the day a leak happens you won’t be able to prove its scope or identify what was exposed.Figure 6: Why the audit log must be tamper-proof and impossible to bypass via configuration flags.There is a catch, and it deserves to be stated plainly: this log is itself one of the most sensitive datasets in the company, because it contains the raw data you worked so hard to protect. Treat it accordingly. Encrypt it, restrict and separate access to it, define retention limits, and work out with your legal counsel how it coexists with erasure requests under regulations like GDPR.Finally, add retroactive leak scanning: background processes that keep re-scanning archived audit history for sensitive patterns that slipped past the first filters. When one is found, the scanner never edits the original log. It alerts the security team, who can tighten the rules before a small gap becomes an incident.Figure 7: Immutable audit pipeline and asynchronous leak scanning.7. Conclusion: AI Is a Data Engineering Problem“Garbage in, garbage out” has a modern cousin: secret in, exposure out. AI isn’t a cosmetic layer you bolt onto a corner of your information system to satisfy the board. It changes how information flows, how it is segmented, how it is anonymized and how it is archived.The companies that win with AI won’t be the ones that deployed the most agents the fastest. They will be the ones that built a safe foundation first. A good brake system is what lets a car drive fast. With these controls in place, teams can say yes to more use cases, with less debate and fewer late-stage vetoes.So before you sign the next AI contract, or celebrate the rollout of your latest agent, gather your architects and ask one question: “If an external auditor sent by our biggest client showed up tomorrow at 8am, could you prove to me, line of code by line of code, that not a single name, identifier or confidential figure left our network in the clear over the past month?”If the answer is silence, or “our vendor’s clauses”, or “a prompt that tells the AI to be careful,” you have your next project.The Governance Checklist: 8 boxes to checkBefore you leave the room, eight boxes to check (the six invariants, plus the testing and auditing essentials): Do you minimize at the source? (Invariant 1) Do you keep the expensive model off simple requests? (Invariant 2) Is every text-to-model boundary protected by a reversible mechanism? (Invariant 3) Do you rehydrate only at the last legitimate moment? (Invariant 4) Does a deterministic guardrail have the final say on plausibility? (Invariant 5) Is every agent-to-agent hop its own trust boundary? (Invariant 6) Is every control verified under real conditions, not just assumed? (Section 5) Is the audit log impossible to disable by flag or by a single administrator? (Section 6) Alex Kim, “The Claude Code Source Leak: fake tools, frustration regexes, undercover mode, and more”, March 31, 2026. The leak came from a source map file published by mistake on npm. The file userPromptKeywords.ts contains the regex used for frustration detection. ↩
The Ultimate Guide to a Smooth Dev Environment
I wrote about Stack a few days ago — covering the short version of why I built it and the whole “no Docker” philosophy. But theory only gets you so far. If you’re looking for the ultimate guide to actually achieving a buttery-smooth dev environment, this is it.No elevator pitch. No marketing fluff. Just the raw, practical reality of getting a project running effortlessly, complete with every weird prompt and console output you’ll encounter along the way.If you just want the dry reference docs, you can find them at sanayavo.com/stack. Otherwise, welcome to the guided tour of a better workflow.What you’re actually gettingA smooth environment means zero bloat. Stack is just one Rust binary. There is no daemon hogging RAM in your system tray, no Docker Desktop spinning up your laptop fans, and absolutely nothing touching your global system PHP install.Instead, it acts as highly intelligent duct tape. It leans on vfox for managing PHP/Node versions, uv for Python, and Caddy for instant local domains. Stack reads a simple stack.toml file in your project root and automatically wires these three tools together.Note: Stack is Windows only right now (PowerShell or cmd). Mac and Linux support are on the roadmap.The Installation: Your one-time headacheA smooth day-to-day workflow requires a tiny bit of upfront setup. Run this to download the installer:irm https://github.com/sanayasfp/stack/releases/latest/download/stackenv-installer.ps1 | iexThen, run this command once per machine:stack setupThis handles the heavy lifting behind the scenes. It adds a hook to your PowerShell profile (so the correct PHP or Node version magically activates when you cd into a directory), installs vfox, uv, and Caddy at heavily-tested versions, and runs caddy trust. That last step is crucial for local HTTPS — more on that in a minute.$ stack setupadded the stack hook for pwshchecking vfox/uv/caddy... vfox: OK (1.0.11) uv: OK (0.11.7) caddy: OK (2.11.4) caddy: local CA trusted (https://*.localhost works with no browser warning)Restart your terminal now. If you skip this, the PowerShell hook won’t be active and you’ll wonder why nothing is working. You will never have to run stack setup again unless you buy a new computer.Starting a project friction-freeLet’s build something real to show off the workflow: a lightweight PHP API backed by MySQL.From scratch$ stack new acme-apidomain: acme-api.localhostlanguages (space to toggle, enter to confirm): [x] phpservices (space to toggle, enter to confirm): [x] mysql php version: 8.3.1 mysql version: 8.0.35created acme-api\stack.tomlnext: cd into it, add a [run] command when you know it, then `stack up`It’s an interactive terminal checklist. Space to toggle, enter to confirm. It feels a bit different the first time, but nothing commits until you press enter. If you accidentally select Node instead of PHP, just hit space again. Zero stress, no do-overs required.If you already have a projectIf your acme-api directory already has a composer.json file specifying "php": "^8.3", Stack is smart enough to skip the manual typing:$ stack initdetected from existing project files: php 8.3 (from composer.json)domain: acme-api.localhoststack init automatically parses composer.json, package.json, or pyproject.toml and pre-fills the wizard for you.The magic artifactStack generates a highly readable manifest:[project]name = "acme-api"domain = "acme-api.localhost"[language]php = "8.3.1"[service.mysql]version = "8.0.35"Commit this stack.toml file to your repo. It is your ultimate “works on my machine” artifact—the lightweight, container-free equivalent of a Dockerfile. When a teammate clones the repo and types stack up, they instantly get the exact same PHP and MySQL versions without installing anything by hand.Teaching it how to runYour stack.toml needs to know how to boot up your dev server. Open the file and add a [run] block:[run]command = "php -S 127.0.0.1:{port} -t public"The {port} placeholder dynamically swaps for an available port at runtime. (Note: If you declare [language.php] but skip the [run] section, Stack falls back to its built-in FastCGI setup using php-cgi.exe and Caddy, which is incredibly solid. But for this guide, we’ll keep it explicit.)stack up and watch it fly$ cd acme-api$ stack upLoaded C:\Users\you\acme-api\stack.toml project: acme-api domain: acme-api.localhost languages: php services: mysqlfirst run for this project — stack.toml will execute: [run] php -S 127.0.0.1:{port} -t publicTrust and run these commands? [y/N] y php: C:\Users\you\.vfox\cache\php\v-8.3.1\...\php.exe -> PHP 8.3.1 (cli) service.mysql: started (pid 41232, port 3306) schema 'acme_api' — automatic creation not yet implemented; create it manually if needed run: php -S 127.0.0.1:52140 -t public (pid 41244, port 52140) log: C:\Users\you\.stack\logs\acme-api.log routed: http://acme-api.localhost -> 127.0.0.1:52140There are two major things happening here that contribute to a seamless experience:1. The security trust promptBecause [run].command is a literal shell invocation, Stack runs it exactly as written. The first time Stack sees a new or modified command (like after a git pull), it asks you to confirm it. Say yes, and it remembers your choice in ~/.stack/trust.json. Every subsequent stack up is blissfully silent. Need to bypass it for CI? Just pass --yes.2. HTTPS works out of the boxThe console says http://, but https://acme-api.localhost works instantly. No red browser warnings. Remember that caddy trust command from the setup phase? It installs a local Certificate Authority (similar to mkcert), meaning your local environment accurately mirrors production SSL from day one.What “no containers” actually means in practiceOpen a second terminal, cd into a totally different project that also requires MySQL 8.0.35, and run stack up: service.mysql: already running, shared with other projects (pid 41232, port 3306)That’s the beauty of it. One MySQL process serving two projects, isolated by schema rather than burning your CPU and RAM on redundant containers.But here is where the “smooth” factor really shines. Open a third terminal, cd into acme-api, and just run php -v. No stack up, no extra commands:$ cd acme-api$ php -vPHP 8.3.1 (cli) (built: ...)$ cd ..$ php -v'php' is not recognized as an internal or external commandThe version switch happens ambiently on every prompt thanks to that PowerShell hook. Your IDE, composer install, and your test runners all automatically get the correctly pinned version the second they enter the folder. Leave the directory, and it vanishes. No source venv/bin/activate, no nvm use, and no polluting your global system.Fresh PHP installs that don’t suckUsually, a fresh PHP installation means spending an afternoon hunting down “PDO driver not found” or “timezone not set” errors. Stack eliminates this entirely.The first time it downloads a PHP version via vfox, it automatically patches php.ini. It turns on OPcache, bumps memory and upload limits, sets a timezone, and enables the extensions you actually need out of the box (pdo_mysql, pdo_pgsql, pdo_sqlite, sockets, sodium, etc.). It happens once, automatically, and you never have to think about it again.stack doctor — The ultimate pre-flight checkBefore you hand off a project or start debugging a strange issue, run this:$ stack doctor --projectchecking C:\Users\you\acme-api\stack.toml... language.php: OK (C:\Users\you\.vfox\cache\php\v-8.3.1\...\php.exe) service.mysql: OK (managed)This command validates your ports, paths, and {PLACEHOLDER} values against your environment (loading .env first) without starting a single service. If something is broken, you get a clean list upfront instead of discovering cryptic errors halfway through startup.Command your setup from anywhereOnce Stack knows about a project, you don’t even need to be in its folder to manage it:stack describe acme-apistack restart acme-apistack down acme-apistack describe dumps everything—resolved binary paths, the exact location of your hashed php.ini, logs, and routed domains. Stack keeps a local registry of your projects, making global management effortless.Shutting it all down cleanlyWhen the day is over:stack down --allEverything stops. Every project, every shared service, and Caddy. You get actual zero CPU usage, not a lingering phantom container you forgot to kill.A quick note on custom domainsUsing .localhost guarantees a smooth experience because browsers automatically resolve it to the loopback address (RFC 6761) without you needing to hack your hosts file. However, if you are migrating from Laragon or Herd and absolutely need .test domains, there is a one-time setup using Acrylic DNS Proxy.Your New WorkflowScaffold, run, trust once, enjoy automatic HTTPS, commit your manifest, and tear it down cleanly when you’re done.I use this exact workflow for real client work every single day. This isn’t a theoretical happy path; it’s a battle-tested blueprint for an ultimate, smooth dev environment.For the complete stack.toml reference, CLI flags, and advanced features (like using [[clone]] to bootstrap from just a manifest), check out the official docs: Why I built this — the longer rant against Docker/XAMPP for local dev Getting Started Manifest Reference CLI Reference Code: github.com/sanayasfp/stack