Yoav Artzi

Explicitly modeling copy operations unlocks substantial improvements in LLM efficiency and performance, even for models already adept at in-context learning.

Free pause tokens boost language model performance without increasing context length or latency, achieving significant gains with minimal training overhead.

Sentiment in social-media conversations is significantly influenced by discourse moves, with specific interventions capable of mitigating negativity or amplifying hostility.

Co-evolving knowledge bases with reasoning capabilities can dramatically enhance the accuracy and relevance of answers in knowledge-intensive tasks.

CO-LMLM achieves lower perplexity than models trained on 40 times more data, revolutionizing how we leverage knowledge bases in language generation.

Separating state storage from next-token prediction in Transformers leads to a 2-3 percentage point improvement in downstream task performance.

LLMs suffer from a severe gradient bottleneck in the output layer, suppressing 95-99% of the gradient norm and crippling training.