The paper investigates how a small grounding model for a GUI agent should receive the action type. Using Qwen2‑VL‑2B fine‑tuned with LoRA on Android, the authors compare a flat baseline with five conditioning methods—auxiliary loss, hard‑routed action word, additive learned embedding, prepended learned token, and prompt word—under matched data, compute, and decoding settings. Results show that auxiliary loss, additive embedding, and prompt word each improve hit@0.10 by five to seven points over the baseline, while hard routing and prepended token perform similarly to the baseline; however, removing a problematic off‑screen touch point class lifts the baseline by nearly seven points, after which no method outperforms it. "whyItMatters":"The study identifies effective strategies for encoding action type in small grounding models, which could improve the accuracy of GUI agents in real‑world Android applications."
arXiv:2606. 11520v1 Announce Type: cross Abstract: Training capable OS agents requires data that simultaneously captures structured user intents, multi-turn task delegation, and grounded tool execution--properties absent from existing datasets.
The paper investigates whether adding a short classification instruction after a user’s prompt improves the ability of activation probes to detect malicious inputs in large language models. Across 13 safety benchmarks and three open‑weight model families, a classification suffix consistently boosts out‑of‑distribution detection (up to ~4 AUC points) compared to no suffix, and the benefit transfers to multi‑position pooling probes used in production. The improvement stems from the classification format itself rather than the specific content of the instruction, though the optimal suffix varies with the model and readout type.
arXiv:2609.13308v1 Announce Type: cross Abstract: A companion evaluation found that naming the target part in a manipulation prompt increased action accuracy by 0.32-0.63 across eight vision-language...
arXiv:2607. 01465v1 Announce Type: new Abstract: Large language models are trained to predict the next token, not to act inside a specific API.
Q-Reflector is a standalone GUI action reflector built on Qwen3‑VL‑8B that learns to generate pre‑ and post‑conditions for each executed action and then evaluates execution success and task validity. It is trained with supervised fine‑tuning, teacher‑student distillation, and reward‑based refinement, and achieves 82.16% accuracy on 1,082 AndroidWorld transitions, outperforming baseline models. In MobileWorld tasks, it improves task success rates by 17.09 and 14.53 percentage points over no external reflection, and surpasses GPT‑5.2 and Claude Haiku 4.5 by 3.42 and 5.98 percentage points respectively.
arXiv:2606. 16432v1 Announce Type: cross Abstract: User instructions are often underspecified because humans rely on implicit assumptions about the surrounding environment.
Paritok-4B is a 4‑billion‑parameter LoRA compressor designed for coding agents, which extracts and retains key spans of code rather than paraphrasing them. It is intent‑conditioned, selecting lines that are most relevant to the agent’s current task, and achieves high fidelity with 96% of identifiers, paths, and numbers preserved. Trained on 67,074 real OpenHands trajectories and fine‑tuned on Qwen3‑4B, it compresses agent context to about 25.7% of its original size while keeping 86.5% of the uncompressed solve quality across 300 SWE‑bench Lite instances.
arXiv:2605. 07284v2 Announce Type: replace Abstract: A late-layer change learned during post-training may work on the base model's earlier state, or it may depend on earlier computation learned with it.
arXiv:2604. 27720v2 Announce Type: replace Abstract: Vision-language models (VLMs) are increasingly applied to medical visual question answering (Med-VQA), yet whether they can \emph{localize} the evidence behind their answers---a prerequisite for clinical auditability---is poorly characterized.
arXiv:2609.22910v1 Announce Type: new Abstract: Vision-language agents that crop and zoom are trained with rewards that credit a successful tool call, yet a successful call does not show that the mod...
Coding agents re-send large file reads and tool outputs to a frontier LLM every turn, and this context dominates their token bill. General-purpose prompt compressors are trained on prose and suit code...
The paper introduces WebMRE, an offline benchmark comprising 541 tasks and 5,293 steps extracted from WebArena trajectories, designed to provide deterministic scoring for web agents without live environments. It enables the first systematic study of how guide sentences and grounded actions reinforce each other, showing that jointly decoding a guide improves element selection accuracy and that the guide acts as a causal instruction channel. The authors fine‑tune models that outperform leading zero‑shot baselines on all offline metrics.