Open the Cases page on our website and you will notice something few of our peers do: every case is not a video but a complete AI session replay, with a .zsession file beside it that you can download and import. From the first question to the final report — how the model reasoned, which tools it called, where it failed, how it recovered — nothing is cut. Ten cases, no exceptions.
Not many teams do this, and the reason is easy to see: an AI session is the most honest log there is, and also the one most likely to leak secrets. It contains the commands that were executed, the raw output the tools returned, echoed environment variables, internal network topology, customer names — any one of these, exposed, could amount to a real security incident. So the industry norm is: demo videos get edited, screenshots get doctored, and the session itself never sees the light of day.
We made the opposite choice. What supports that choice is not courage but a data redaction engine that does its work before content leaves the machine. This post covers three things: how it works, where we got it wrong, and a fact that is easy to overlook — good redaction is not about masking as much as possible.
1. What Stays Unedited Is the Process; What Gets Masked Is the Private Data
First, a precise definition of “unedited” in the title: the working process stays exactly as it happened; the sensitive fields get masked. Every piece of reasoning, every tool call, every failed retry in the replay is the original record of what took place — while license serials, keys, and tokens were already replaced at the moment of export.
You can verify this directly. Open the session replay of the Antiy IEP EPP automated inspection case, find the inspection report section, and you will see a line like this:
License serial:
****, expires 2026-06-24 (1 day remaining)
At the end of the report, the AI added a line of its own: “Sensitive data redacted: the license serial has been replaced with ****.” The act of redaction is itself preserved in the record. This is exactly the state we want: masking that is explicit, visible, and auditable — not content quietly deleted.
2. The Basics of the Redaction Engine
In the product it lives under Settings → Data Redaction, positioned as a data protection engine for outbound scenarios: it steps in only before content leaves the machine — exporting an HTML replay, exporting a .zsession, or the AI writing .md documents to disk, each scenario with its own independent switch. Everyday conversation and local analysis are left untouched.
The rule layer offers four types:
- keyword — full replacement of a literal string; plain and reliable;
- regex — RE2 regular expressions, with capture groups supported in the replacement template, enabling partial masking in the “keep the head, mask the tail” style;
- ipv4_mask — structured IPv4 masking that lets you choose which octets to mask (any octet, middle ones included) instead of masking the whole address;
- domain_mask — domain masking that keeps the last few labels (by default only the TLD) and replaces the rest with mask characters.
Above the rules sits a set of built-in secrets presets covering common credential formats with high confidence and low false positives: AWS Access Key, GitHub PAT, Slack Token, Stripe Live Key, GCP API Key, OpenAI Key, and entire multi-line PEM private key blocks. These formats are unmistakable (AKIA…, ghp_…, sk_live_…), the false-positive rate is low, and enabling them costs almost nothing — even if you accidentally paste a real AWS key into a session, it will already have been replaced by the time you export.
Two design decisions are worth spelling out:
Every hit leaves a trace. Whenever an export triggers redaction, a notice tells you how many hits there were and which rules matched (listed by rule Label). Every change to the file can be accounted for — the redaction system itself must not be a black box: otherwise you can confirm neither that it masked everything it should have, nor that it masked nothing it should not have.
The state is always unambiguous. When the master switch is off, every scenario passes content through verbatim. The design looks conservative but is in fact necessary — half-on redaction is the truly dangerous state: the user believes something is masked when it is not. We would rather users know plainly that “redaction is currently off” than manufacture the illusion of “it is on, but I am not sure what went missing.” For the same reason, write_md write-to-disk redaction is off by default: it changes what gets written, so it requires explicit opt-in.
Worth adding: the principle that credentials never go out the door runs through the whole product. The read-side MCP configuration tools (mcp.describe_server, mcp.export_config) return authToken values showing only the last 4 characters, and environment variable values are masked in full — export or not, credentials already sit in a masked state in the conversation stream.
3. Two Defects, Two Fixes
A redaction mechanism is hard to get right on the first try. Our changelog records two real defect fixes:
First: IPs at the start of a line inside code blocks were not masked. A session exports as a JSON structure, and code block content carries escapes. The old implementation applied rules to the text before unescaping, so IPs at line-start positions inside code blocks failed to match. The fix was to apply rules to the decoded JSON text values — first restore the text to its final rendered form, then decide what to mask.
Second: deep fields in nested JSON were missed. Tool call arguments and results often embed another layer of JSON; the old implementation scanned only top-level text values, leaving the deeper fields as a blind spot. The fix was to recurse down through nested JSON structures, bringing tool call arguments and results fully under coverage.
Both defects point to the same lesson: a redaction engine's adversary is not “the sensitive string” but the shape of the data. Escaping, nesting, encoding — every transformation of shape is a potential channel through which sensitive data slips past the mask. Both fixes were written into the changelog and the reference manual exactly as they happened — a defect is not what should scare you; a fix that leaves no record is.
4. More Redaction Is Not Better Redaction
Finally, the fact that is easy to overlook. Inspect our published sessions closely and you will find that quite a few internal IPs remain — not an oversight, but a judgment call. Three real examples, all verifiable on the website:
In the IRC botnet traffic analysis session, the victim host 192.168.7.133 and the internal DNS 192.168.7.2 are kept as-is. They are traffic inside the sample pcap — the communication relationships between the victim host and the C2 are the case's chain of evidence itself. Mask them and the triage can no longer be reproduced, at which point publishing the replay loses its meaning. IOCs in a sample are objects of triage, not private data.
The EDR behavioral alert chain analysis session contains the string 10.0.19041.4522 — it looks like an internal IP but is in fact a Windows version number. An undiscriminating IPv4 regex would mangle it into 10.0.*.*, and the OS version information in the alert analysis would be lost with it. This is exactly why ipv4_mask is designed to let you choose which octet to mask — the mechanism should be precise; the judgment should be left to professionals.
In the Antiy IEP EPP inspection session, the license serial is masked to **** while the inspection target's SSH address 192.168.80.152 is kept — the former is a credential, and leaking it carries real cost; the latter is the inspection target in a demo environment, and masking it would leave readers unable to tell what the inspection was even being run against.
Three examples, one conclusion: what must be masked and what must be kept is a domain judgment — not a decision a regex can make for you. Mask too much and you destroy evidence; mask too little and you leak credentials. What the redaction engine provides is a mechanism for executing that judgment precisely; the judgment itself comes from an understanding of security work. Which is a view we have held all along: tools for security scenarios are best built by security teams.
5. A Closed-Source Product Needs Transparency Too
As a security agent that faces the adversary head-on, we chose not to open our detection core, carrying the obligation of transparency through “verifiable releases” instead (see Why We Chose Closed Source). Publishing our sessions is the other half of the same answer —
We do not publish our source code; we publish our behavior. How the AI analyzes a sample, how it keeps its restraint inside the read-only analysis zone, how it errs and how it recovers — the entire process can be inspected as it happened, downloaded, and imported to reproduce. Source transparency proves what the code looks like; behavioral transparency proves what it actually did. For an agent about to enter your workbench, the latter is usually closer to what users really care about.
And redaction is one of the foundations on which this behavioral transparency stands: because the exit holds, the process can be published in full. Your sessions are stored locally, and on export the same set of rules works for you — whatever we dare to hang on our website, you can just as safely send to a colleague, hand to a client, or write into a postmortem report.
Transparency and confidentiality are not natural opposites. Draw the line correctly between what to publish and what to protect, and the two stand together.
We'll be waiting for you in AVL Code — riding a donkey, every session replayable, every export redacted.
AVL Code — the AVL security engine, with intelligence at your side. From the Antiy Landi team.