All cases
#013 · Supply-Chain Verification

Case study · Supply-Chain Verification

Verifying the Grok Build CLI Repository-Upload Claim

You are a security researcher verifying the data boundary of a closed-source AI coding tool. The allegation comes from a third-party researcher’s packet captures; all you hold is a 42.8 MB offline installer and a read-only-sample constraint. Work through this article and you will get an item-by-item comparison of 8 allegations (6 confirmed, 2 honestly flagged as boundary cases), an archivable verification report, and a "workspace → git bundle → GCS" taint-propagation path.

Target: grok-build 0.2.93 offline installer · 44,869,412 bytes (~42.8 MB, zstd)· Constraints: pure static · read-only sample sandbox · two agents in parallel· Model: Landi N2.5-Flash (provided by AVL-Zero)· Deliverables: verification comparison + analysis report md + taint analysis md· Duration: about 20 minutes

Session replays & reports are original records in Simplified Chinese · Built with AVL Code + the Landi model

Overview

  1. 01

    Evidence first — two tracks in parallel, sample fingerprinted at 42.8 MB

    You hand AVL Code the cereblab gist link and "the pkg under the samples directory" in one sentence; after a gist timeout and a missing directory, it matches the installer by version number and the static track gets to work.

  2. 02

    Item-by-item comparison — 6 of 8 allegations confirmed, 2 honestly flagged

    Unpacking, entropy analysis, string/IOC extraction and ELF parsing line up against the allegation list: bucket, endpoint, switches and credentials all present. Static analysis proves "can do"; "did do" is honestly deferred to the dynamic captures.

  3. 03

    Model and deliver — verification report + taint path, two md files on disk

    You ask for an md report to fix the evidence, then add a taint-propagation analysis, and receive the "workspace → git bundle → GCS" core path plus the --deny permission-mismatch conclusion — both files archive-ready.

Task for AVL Code

Analyze and verify whether the claim at https://gist.github.com/cereblab/dc9a40bc26120f4540e4e09b75ffb547 exists in the pkg under the samples directory.

Prerequisites: An offline installer of a closed-source AI coding tool (the sample is recorded by hash only and kept in a separate read-only directory), plus a link pointing to the public claim (gist / reproduction repo / packet-capture record)

Reproduction materials at the end of this article Try it yourself ↓

Evidence First

The opening move is not analyzing the sample but turning the "claim" into a checkable entity:

You type → AVL Code
Analyze and verify whether the claim at https://gist.github.com/cereblab/dc9a40bc26120f4540e4e09b75ffb547 exists in the pkg under the samples directory.
AVL Code runs → you see
Fetching the cereblab gist → times out; samples directory missing → two dead stops in a row
→ Matches the same-named offline installer grok-build 0.2.93 by version number (44,869,412 bytes, zstd)
→ The static-analysis track starts: unpacking, entropy, strings and IOCs, ELF parsing

Item-by-Item Comparison

The unpack, entropy, string and ELF outputs line up against the allegation list — static evidence vs public claims, item by item:

You type → AVL Code
(the same instruction still running, no new input)
AVL Code runs → you see
Unpacking verified: entropy 7.999 → 6.24, unpacked correctly
→ String/IOC extraction hits every part of the upload chain: bucket grok-code-session-traces · endpoint POST /v1/storage · GROK_TRACE_UPLOAD_* switches · devstorage.full_control credential scope
→ git bundle packaging present in the client
→ Comparison verdict: 6 of 8 allegations confirmed; "did do" is honestly deferred to cereblab’s dynamic captures

Model and Deliver

The evidence must be archivable and the path reusable — two md files fix the conclusions:

You type → AVL Code
Write an analysis report in md format

Add a taint-propagation analysis and save it as md
AVL Code runs → you see
Verification-comparison report md: the 8 allegations tabled item by item
→ Taint-propagation analysis md: the workspace → git bundle → GCS core path
→ Permission-mismatch conclusion: read-deny (--deny) does not stop files from being bundled
→ Both md files on disk, archive-ready

Try it yourself

Follow the four steps below to verify any "closed-source tool secretly uploads" claim the same way: find a checkable entity first, statically prove "can do", then hand "did do" over to dynamic evidence.

Write an analysis report in md format
Add a taint-propagation analysis and save it as md
  • Step 1: prepare two leads — the claim link and the offline installer (the sample is recorded by hash only, kept in a separate read-only directory): claim <gist-url> · sample <pkg-path>.
  • Step 2: hand both leads to AVL Code together (the verification instruction follows chapter 1 verbatim); if the gist fetch times out or the directory is missing, let it inspect the directory structure first and then switch to another evidence route.
  • Step 3: ask for an archivable verification report — the first instruction above.
  • Step 4: append the taint-propagation analysis to fix the propagation path and the permission-mismatch conclusion — the second instruction above.
  • Honest limitations: the session HTML export contains no per-call tool traces; the duo right column’s action sequence is compiled from session messages and the delivered reports, at message-level granularity. Entropy 7.999→6.24, the two-agent split, the ~20-minute duration and the read-only sample sandbox come from the case library’s existing records and were not re-verified inside the session. cereblab’s dynamic captures (5.10 GiB / 192 KB / server-side fix on 2026-07-13) are external evidence; this task did not reproduce the capture, and post-fix server behavior cannot be verified from an offline sample.

Next question: Verifying "can do" is done — the next task is detecting "doing it now": deploy a detection rule for gs:// uploads and the /v1/storage endpoint on egress traffic, or run the same static workflow against the next closed-source AI CLI.