Fei-Yi Wang

This work compares open-weight and API-served general-purpose, multimodal, and astronomy-specialized models using judge-based correctness and complementary reference metrics and highlights the role of domain-specific evaluation in determining which models, capabilities, and evaluation criteria are appropriate for scientific workflows.

Training MoE models just got a whole lot faster: Piper achieves up to 3.5x higher MFU by intelligently scheduling pipeline parallelism and optimizing communication.

Turns out, federated learning with PEFT doesn't protect your LLM training data as well as you thought: FedSpy-LLM can reconstruct surprisingly long sequences from shared gradients, even across different model architectures.

Dataset distillation can be sped up by 18x on ImageNet-1K without sacrificing accuracy by focusing optimization on high-loss regions.