Context
This project started from a Jane Street-style puzzle: a residual network had been broken into dozens of saved layer fragments, and the objective was to reconstruct the original model using only the pieces and a historical dataset.
The interesting part was that this was not a normal training problem. The model already existed. The challenge was to recover its architecture and exact ordering from indirect signals.
What I built
I built a full reconstruction pipeline around three main steps.
A pairing view that helped narrow down which fragments belonged together before searching for the final order.
- Classify the fragments into input layers, output layers, and the final layer using tensor shapes.
- Infer input/output pairings by scoring every candidate match with several heuristics.
- Search for the block order using both greedy placement and a faster simulated annealing variant with cached intermediate states.
The pairing stage combined several signals instead of relying on only one metric. I used norms, trace-like statistics, activation behavior, dead-neuron alignment, and residual-structure heuristics, then solved the final one-to-one assignment with the Hungarian algorithm.
Result
The final solution recovered the exact model permutation. In practice, the reconstructed network behaved like the original instead of merely approximating it.
One of the ordering diagnostics used to compare search behavior and converge on the final arrangement.
Why I like this project
I like this one because it combines model understanding, search, and practical experimentation. It was not enough to throw one optimization method at the problem. The project needed a sequence of informed heuristics, diagnostics, and iterative refinement.
Why this approach worked
What made the project work was breaking the puzzle into stages with the right feedback loop at each step. Pairing quality had to be good before ordering quality could matter, and the visual diagnostics made it much easier to tell which heuristics were genuinely useful.