Steven Y. Feng

Forget what you know: RAG's marginal utility hinges on model scale, task type, and pretraining saturation, offering a quantitative guide to balancing pretraining and retrieval.

Training language models on individual children's language reveals that distributional and interactional linguistic features, not just dataset size, are key to efficient learning, mirroring factors that drive child language acquisition.

Bilingual language models can achieve performance comparable to monolingual models in both languages, challenging the assumption that bilingual input poses significant learning obstacles.