Jasper Zhang

Fast-weight attention can enhance language modeling and improve task performance by effectively managing context and memory in recurrent networks.

Conformer ensemble statistics can reduce RMSE by over 13% for solvation properties, but surprisingly, they offer no advantage for electronic or steric tasks.

Gradient-based task analysis in multi-task learning is fundamentally flawed on standard benchmarks due to insufficient overlap in training data, explaining years of inconsistent results.

Context-aware models for molecular property prediction can unlock accurate predictions in data-scarce regimes, but beware – distribution shifts can cause them to backfire, and your benchmarks might be meaningless.