Case study · Supply-Chain Verification
Verifying the Grok Build CLI Repository-Upload Claim
You are a security researcher verifying the data boundary of a closed-source AI coding tool. The allegation comes from a third-party researcher’s packet captures; all you hold is a 42.8 MB offline installer and a read-only-sample constraint. Work through this article and you will get an item-by-item comparison of 8 allegations (6 confirmed, 2 honestly flagged as boundary cases), an archivable verification report, and a "workspace → git bundle → GCS" taint-propagation path.
Target: grok-build 0.2.93 offline installer · 44,869,412 bytes (~42.8 MB, zstd)· Constraints: pure static · read-only sample sandbox · two agents in parallel· Model: Landi N2.5-Flash (provided by AVL-Zero)· Deliverables: verification comparison + analysis report md + taint analysis md· Duration: about 20 minutes
Session replays & reports are original records in Simplified Chinese · Built with AVL Code + the Landi model
Overview
- 01
Evidence first — two tracks in parallel, sample fingerprinted at 42.8 MB
You hand AVL Code the cereblab gist link and "the pkg under the samples directory" in one sentence; after a gist timeout and a missing directory, it matches the installer by version number and the static track gets to work.
- 02
Item-by-item comparison — 6 of 8 allegations confirmed, 2 honestly flagged
Unpacking, entropy analysis, string/IOC extraction and ELF parsing line up against the allegation list: bucket, endpoint, switches and credentials all present. Static analysis proves "can do"; "did do" is honestly deferred to the dynamic captures.
- 03
Model and deliver — verification report + taint path, two md files on disk
You ask for an md report to fix the evidence, then add a taint-propagation analysis, and receive the "workspace → git bundle → GCS" core path plus the --deny permission-mismatch conclusion — both files archive-ready.
Task for AVL Code
Analyze and verify whether the claim at https://gist.github.com/cereblab/dc9a40bc26120f4540e4e09b75ffb547 exists in the pkg under the samples directory. Prerequisites: An offline installer of a closed-source AI coding tool (the sample is recorded by hash only and kept in a separate read-only directory), plus a link pointing to the public claim (gist / reproduction repo / packet-capture record)
Reproduction materials at the end of this article Try it yourself ↓
Evidence First
The opening move is not analyzing the sample but turning the "claim" into a checkable entity:
Analyze and verify whether the claim at https://gist.github.com/cereblab/dc9a40bc26120f4540e4e09b75ffb547 exists in the pkg under the samples directory.Fetching the cereblab gist → times out; samples directory missing → two dead stops in a row
→ Matches the same-named offline installer grok-build 0.2.93 by version number (44,869,412 bytes, zstd)
→ The static-analysis track starts: unpacking, entropy, strings and IOCs, ELF parsingItem-by-Item Comparison
The unpack, entropy, string and ELF outputs line up against the allegation list — static evidence vs public claims, item by item:
(the same instruction still running, no new input)Unpacking verified: entropy 7.999 → 6.24, unpacked correctly
→ String/IOC extraction hits every part of the upload chain: bucket grok-code-session-traces · endpoint POST /v1/storage · GROK_TRACE_UPLOAD_* switches · devstorage.full_control credential scope
→ git bundle packaging present in the client
→ Comparison verdict: 6 of 8 allegations confirmed; "did do" is honestly deferred to cereblab’s dynamic capturesModel and Deliver
The evidence must be archivable and the path reusable — two md files fix the conclusions:
Write an analysis report in md format
Add a taint-propagation analysis and save it as mdVerification-comparison report md: the 8 allegations tabled item by item
→ Taint-propagation analysis md: the workspace → git bundle → GCS core path
→ Permission-mismatch conclusion: read-deny (--deny) does not stop files from being bundled
→ Both md files on disk, archive-readyTry it yourself
Follow the four steps below to verify any "closed-source tool secretly uploads" claim the same way: find a checkable entity first, statically prove "can do", then hand "did do" over to dynamic evidence.
Write an analysis report in md format Add a taint-propagation analysis and save it as md - Step 1: prepare two leads — the claim link and the offline installer (the sample is recorded by hash only, kept in a separate read-only directory): claim <gist-url> · sample <pkg-path>.
- Step 2: hand both leads to AVL Code together (the verification instruction follows chapter 1 verbatim); if the gist fetch times out or the directory is missing, let it inspect the directory structure first and then switch to another evidence route.
- Step 3: ask for an archivable verification report — the first instruction above.
- Step 4: append the taint-propagation analysis to fix the propagation path and the permission-mismatch conclusion — the second instruction above.
- Honest limitations: the session HTML export contains no per-call tool traces; the duo right column’s action sequence is compiled from session messages and the delivered reports, at message-level granularity. Entropy 7.999→6.24, the two-agent split, the ~20-minute duration and the read-only sample sandbox come from the case library’s existing records and were not re-verified inside the session. cereblab’s dynamic captures (5.10 GiB / 192 KB / server-side fix on 2026-07-13) are external evidence; this task did not reproduce the capture, and post-fix server behavior cannot be verified from an offline sample.
Next question: Verifying "can do" is done — the next task is detecting "doing it now": deploy a detection rule for gs:// uploads and the /v1/storage endpoint on egress traffic, or run the same static workflow against the next closed-source AI CLI.
