RLHF & Preference Learning - Weekly Roundup

ALIBI is presented, a semantic cover story attack against frontier LLM-based malware analyzers that adds a small, non-executed read-only section to a compiled binary, containing a coherent but false security product narrative, without altering imports or executable behavior.

ALIBI is presented, a semantic cover story attack against frontier LLM-based malware analyzers that adds a small, non-executed read-only section to a compiled binary, containing a coherent but false security product narrative, without altering imports or executable behavior.

CoFree (Collapse-Free Reasoning Embedding), a two-stage framework that progressively integrates LLM reasoning into query and document embedding optimization while preserving reasoning quality, is proposed.

The probabilistic Allen algebra (PAA) is developed, a generative and complete extension in which relation probabilities are derived from distributions over interval boundaries rather than assigned as scores.

Results show that clean CTF performance understates vulnerability to deceptive task evidence, and AgentLSD isolates this effect and provides a reproducible benchmark for studying it.

A three-tier observability framework comprising catalog metadata, passive compliance signals, and live vulnerability analysis is applied to empirically characterize the public MCP server ecosystem, identifying a Security-Observability Tradeoff observed in the current ecosystem.

This work proposes EvoSkill-GUI, a training-free framework in which each skill is a structured multi-file package containing retrieval metadata, executable plans, backup localization, failure-recovery rules, accessibility utilities, and failure cases.

EchoPath is introduced, a model-agnostic harness that converts artifact-validated GUI trajectories into standardized, parameter-controlled callable memories, analogous to Model Context Protocol (MCP)-style tool calls rather than unstructured experience records, which support a bounded form of enterprise GUI memory.

This paper reviews over 120 security publications from 2008 to 2025, focusing on how these isolation boundaries can be breached, and introduces a quantitative scoring framework called ADPO, which rates defenses from 0 to 3 based on their Accuracy, Deployment ease, Performance impact, and Operational overhead.

This work introduces LSREP, a Longitudinal State-Replay Evaluation Protocol combining ordered replay, explicit lifecycle schedules, repeated probes, evolving reference answers, and mechanism-fidelity checks that exposes distinct failure modes that neither architectural descriptions nor aggregate scores identify alone.

Extensive offline evaluation, human calibration, and online A/B tests show that SARA-7B generates more specific, polarity-consistent, and grounded rationales than strong MLLM baselines, while SARA-Ranker improves engagement and reduces negative feedback in production.

StackTok is introduced, a training-free selector that treats query relevance as the objective and visual coverage as budget-calibrated support in vision-language models and ranks first among training-free selectors in every tested model--budget setting.

Turn-level Multiscale Density Ratio Estimation (tlm-DRE), which assigns different weights on corresponding turns and proposes asymmetric token-level training based on the positive-negative space gaps across multiple turns of tasks, enables LLMs to perform robustly in multi-turn reasoning tasks with both in-domain and out-of-domain conditions.

An end-to-end model is proposed, which is named DiaWhisper, that fine-tunes Whisper-large-v3 with LoRA and an auxiliary frame-level role head for transcription and attribution together with DiaWhisper-DPO, a failure-mined refinement that uses genuine decoding failures as DPO rejected completions without human preference annotation.

Savor is introduced, a training framework that augments the output schema with token and answer confidence, optimises the policy with a Group Relative Policy Optimisation objective that penalises calibration error and poor abstention decisions, and uses the learned confidence at inference time to revisit visual evidence only when the model is uncertain.

ERPBench is introduced, a benchmark that evaluates screenshot-only agents on a live and reproducible ERP system and scores each task against ground-truth values in its database and presents a production-grade harness that gates agent actions behind human approval for safe deployment, which ERPBench runs autonomously.

This study reconceptualizes pairwise classification not as a technique confined to offline RL, but as a domain-agnostic structural principle for determining which of the two candidates scores higher when a context is shared.

Flow-Matched Motion Priors (FMP) is introduced, an online scalar reward learned from paths connecting current rollout histories to an expert motion bank, and Controlled reward-model experiments show substantially better generalization beyond the fitting rollout than value-only or endpoint-only fitting.

HazardAuditor is introduced, an execution-grounded framework that runs heterogeneous agents in controlled environments and normalizes their interactions into a canonical event representation for cross-framework supervision, and observes that token-level post-training objectives create a structural mismatch for generative guards, causing longer rationales to dominate gradient updates.

CRN v2, a lightweight logit-level correction module that sits atop a fully frozen Gemma 4 E2B model that sits atop a fully frozen Gemma 4 E2B model is proposed, a study of a design principle (frozen base + logit correction + KL anchoring), not a claim of architectural novelty.

StalePO is introduced, an objective derived from three requirements this regime imposes that improves the fraction of segments passing all LLM-as-judge MQM quality checks by 14.9 and 4.6 percentage points, respectively, with gains concentrated on style and fluency.

A principal hierarchy spanning human user, operator/deployer, orchestrator agent, sub-agent, and tool endpoint as an organizing framework is introduced, and seven structural requirements are proposed, derived from a four-layer reference architecture, and runtime enforcement and aggregation bounds are identified as the principal unresolved gaps.

The results support a narrow conclusion: the verifier changes what the system ships, while deterministic code supplies enforcement and auditability; they do not establish superiority to other agents or generalization beyond lab targets.

A stencil computation repeatedly updates every cell of a grid from its neighbours'values at the previous timestep, and a line of work reduces this by composing many timesteps into one linear operator and applying it with a Fast Fourier Transform.