Search papers, labs, and topics across Lattice.
7
0
5
14
Unbiased sampling of source prefixes in TLMs can reduce runtime by several orders of magnitude while maintaining accuracy in estimating target prefix probabilities.
LITTLELEARNER reveals that even a well-defined knowledge scope can yield a competent language model, but it won't expand its capabilities beyond its educational boundaries.
Conventional RoPE's non-repeating rotations may hinder its ability to access distant context, unlike periodic RoPE, which excels at recognizing modular languages.
Surprisal theory's reliance on arbitrary tokenization schemes undermines its validity, but this framework offers a way to fix it.
Forget specialized prefix-parsing algorithms: a simple grammar transformation lets you use standard parsers for efficient prefix parsing and next-token prediction.
Forget simple probability averaging: a byte-level sequential Monte Carlo algorithm unlocks better language model ensembles by enabling sophisticated aggregation strategies and handling mismatched vocabularies.
Repurpose existing language models for entirely new output formats like bytes, words, or even DNA sequences, all without retraining, by wrapping them in a finite-state transducer.
