Search papers, labs, and topics across Lattice.
This study critically examines the performance of a port-Hamiltonian deep equilibrium model (DEQ) with learned initialization across two reasoning tasks, revealing that the model's implicit computation often results in a no-op, where the solved equilibrium matches the initial state with high precision. Despite expectations that increased iterations would correlate with problem difficulty, the findings show no significant improvement in accuracy, with a notable lack of correlation between iteration counts and task complexity. The research highlights issues of gradient starvation and proposes a diagnostic protocol for evaluating implicit computation claims, ultimately demonstrating that the DEQ fails to outperform a simple two-layer MLP in these tasks.
Implicit computation in deep equilibrium models may be a mirage, as our findings reveal that the model often fails to improve upon its initialization, raising questions about their efficacy in complex reasoning tasks.
Deep equilibrium models promise input-adaptive implicit computation: harder problems should demand more solver iterations, and the solved equilibrium should encode the result of genuine iterative inference. We report a cautionary study of a port-Hamiltonian DEQ with a learned initialization on two reasoning tasks -- ProofWriter entailment over frozen DeBERTa embeddings and a BFS-verified graph-reachability benchmark -- in which the implicit computation is a silent no-op. Across tasks, seeds, and controlled ablation arms, the solved equilibrium equals the solver's start point to numerical precision, and bypassing the solver entirely changes test accuracy by +0.00 percentage points in 18 of 19 training runs. Controlled interventions falsify the tempting explanation: removing the anchoring term reproduces every result, and retraining with noise-decoupled starts yields a solver that converges to the noisy start while the decoder learns to ignore it. The single escaping run diverges instead ($\|h^{*}-z_0\|=171$), producing a co-adapted noise channel whose removal improves accuracy. Iteration counts are uncorrelated with ground-truth difficulty ($r=0.009$), and the full apparatus never outperforms a two-layer MLP on either task. We trace the mechanism to gradient starvation along two distinct routes, show that the standard zeroing ablation is confounded and gives wildly seed-dependent answers where the correct substitution test gives a stable zero, and distill a four-test diagnostic protocol for auditing claimed implicit computation. All experiments run on a single free Colab GPU; code, raw logs, and analysis scripts are released.
