The Puzzle
In February 2026, Jane Street released a puzzle: a neural network checkpoint, model_3_11.pt, with a single question - what input string makes the network output a positive value? All other inputs return −15. No architecture description was given. No training code. Just the weights.
The puzzle sits at an interesting intersection of reverse engineering and machine learning. A conventionally trained network would be opaque - activations spread diffusely across millions of parameters and there is no clean way to recover a specific input from the output. This one, we suspected, was different. The integer-valued weights and binary output were strong signals that the network was hand-crafted rather than trained.
Opening the Archive
PyTorch .pt Each tensor is stored as a flat binary blob under data/N for integer index N. We can extract the weights without any torch.load call and avoid the security risks of arbitrary pickle deserialization on an untrusted checkpoint.
5,442 tensors. Every weight we sampled was an exact integer (verified to within 1e-4). A gradient-descent-trained network of this size would have smooth, non-integer weights - the integer constraint is an immediate signal that this model was constructed programmatically.
Inferring the Architecture
PyTorch's Sequential serialiser stores weight tensors at even indices and bias tensors at odd indices. For a Linear(n_in, n_out) layer: len(weight) == n_in × n_out and len(bias) == n_out. This gives us a clean recovery condition — if len(weight) % len(bias) == 0, we have a valid layer. No pickle parsing required.
2,721 layers. The final weight pattern - sixteen +1s, sixteen −2s, sixteen +1s - and the bias of −15 are not arbitrary. They are the signature of a specific arithmetic structure, which becomes clear once we understand how the network implements equality checking.
The Equality Mechanism
The core insight is that the network tests whether each byte of MD5(input) matches a stored target byte. It does this using what we call the hat function - the second discrete difference of ReLU.
For a single byte with target value t, three neurons compute:
The biases of those neurons store −(t+1), −t, and −(t−1) respectively. The output is exactly 1 when x = t, and 0 for every other integer. This gives us an exact equality test purely from linear algebra and ReLU activations.
The final layer then aggregates: it sums all 48 hat outputs (16 bytes × 3 groups, a redundancy for robustness) with weights +1, −2, +1 per group and subtracts 15. The output is 1.0 only when all 16 target bytes match — otherwise it is ≤ −15.
Extracting the Target Hash
The penultimate layer has 48 outputs, arranged as three groups of 16. Each group encodes the same 16 target bytes with a different offset - essentially a triple-redundant storage of the 128-bit MD5 hash:
| Group | Indices | Bias value | Recovery formula |
|---|---|---|---|
| 1 | 0–15 | −(t+1) | t = −b − 1 |
| 2 | 16–31 | −t | t = −b |
| 3 | 32–47 | −(t−1) | t = −b + 1 |
Reading group 2 (the canonical group) and asserting all three agree:
Forward Pass Validation
Before inverting the hash, we validate the full forward pass with a pure NumPy implementation - ReLU on every layer except the final linear output:
Case sensitivity matters: "Bitter Lesson" with capitals is a different MD5 and returns −15. The exact string "bitter lesson" - lowercase, two words, no punctuation - is the only input that passes.
Inverting the Hash
MD5 is not reversible by design, but the search space for a two-word English phrase is small. We run a targeted vocabulary check first: phrases drawn from Rich Sutton's 2019 essay The Bitter Lesson, then fall back to a parallelised brute-force over a curated word list if needed.
The targeted search hits on the first entry in the vocabulary list. The brute-force path (parallelised over a ~80-word vocabulary, ~6,400 combinations) is included as fallback in the notebook and would complete in under a second.
Verification
What the Network Is
The model is a hand-crafted MD5 circuit implemented entirely in 2,721 stacked linear layers with ReLU activations. The bulk of those layers compute the MD5 hash of the input byte-by-byte. The final two layers perform the equality check against a target hash stored in the penultimate biases.
The network is not trained. Every weight is an integer. The 1.1 GB size is a consequence of implementing MD5's bitwise operations as matrix multiplications - a highly redundant representation, but one that a standard PyTorch inference pass handles correctly.
The payload inside the weights says "bitter lesson" - the title of Rich Sutton's essay arguing that general-purpose methods leveraging compute consistently beat methods encoding human knowledge. The puzzle is self-referential: a network that looks like it might know something deep is actually just running a hash check, and the answer it's checking for is a reminder to trust computation over human intuition.
Methodology Notes
Architecture recovery without pickle parsing
The (weight, bias) pairing and the divisibility condition len(weight) % len(bias) == 0 uniquely identifies every Linear layer without reading data.pkl. All 2,721 layers recover cleanly.
Hash extraction is exact
The three bias groups provide a built-in consistency check. All three groups agree to within floating-point precision (the biases are integer-valued, so agreement is exact). The extracted hash is deterministic and reproducible on any copy of the checkpoint.