arXiv Computation and Language

Selection Is Retrieval, Abstention Is Not: On-Device Tool Routing over 70 Korean-English Actions

Hugging Face Trending Papers
Jul 21

RAGAL: A Frugal, Fully Local Retrieval-Augmented Assistant for Technical Support at a Government Agency

Public institutions hold large volumes of sensitive documents and support tickets that cannot leave the premises, ruling out cloud-hosted language models entirely. We report on RAGAL, a retrieval-augmented assistant for the technical-support team of AFIR, the Romanian Agency for Financing Rural Investments, built and operated under three hard constraints: zero data egress (no external API calls, even for synthetic data), a read-only mandate (the assistant drafts, humans execute), and a single 8 GB consumer laptop as the only development and training machine.

arXiv AI
Nov 6

Label-Wording Augmentation in a Typed-Decision Encoder: Gains on Unfamiliar Words, Not on Abbreviations or Codes

The paper evaluates a 307‑million‑parameter multilingual cross‑encoder, Ru‑Jev, on Russian‑language tasks, comparing its original and an augmented release that presents routing labels in multiple surface forms. Experiments show that the augmentation improves accuracy on unfamiliar ordinary words, borrowed jargon, and office slang—up to 0.24 points—while providing no benefit for abbreviations or opaque codes. The study also highlights evaluation hazards and notes that increased accuracy often coincides with reduced risk‑coverage, especially with many candidate labels.

By Vladimir Beskorovainyi
arXiv AI
Sep 4

Learning What Not to Forget: Long-Horizon Agent Memory from a Few Kilobytes of Learning

The paper introduces LRE (Learned Relevance Eviction), a lightweight, CPU‑only, language‑model‑free scorer that learns which parts of an agent’s interaction history are task‑critical and preserves them verbatim. In experiments, LRE matches or surpasses baseline eviction policies on accuracy‑cost trade‑offs, recovers 93% of full‑history accuracy, reduces worst‑case prompt size by 52%, and outperforms dense and token‑pruning encoders in conversational memory while being 295–1569× smaller. The method also achieves superior budgeted answer quality on LoCoMo reading and can be trained annotation‑free, recovering 95% of supervised scorer performance.

By Nusrat Jahan Lia, Aritra Mazumder
arXiv AI
Oct 16

Language Is a Security Surface for Agent Guards

The study investigates whether the language of a user’s instruction affects the safety verdicts of agent guards. By translating 1,220 labeled steps from an English benchmark into German, French, Turkish, Swahili, and Amharic while keeping all other data identical, the authors find that many open, small, and purpose‑built guards alter their verdicts when the instruction is translated, with larger effects for rarer languages. The impact varies with guard size, with smaller models showing greater loss of recall and larger models (e.g., Gemini 3.8 Flash) remaining largely unchanged.

By Hasan Demirkiran, Jens Ernstberger
arXiv AI
Aug 28

MemToC: Benchmarking Memory-Tool Conflict Resolution in Large Language Models

MemToC is a controlled benchmark that tests how large language models resolve conflicts between their internal memory and tool outputs. It contains 6,504 episodes built from 542 factual questions, each paired with a model‑generated closed‑book answer and a tool return whose correctness is known, creating four distinct source‑correctness scenarios. Across five 7‑9B open‑weight models, tool responses overwhelmingly dominate closed‑book answers, and only a minority of instruction‑tuned models correctly retain a verified answer when the tool is wrong, while most follow a correct tool or repeat a wrong tool.

By Arseniy Varlamov, Rishat Zinnatullin, Elisei Rykov, Alexander Panchenko, Ilseyar Alimova
arXiv Machine Learning
Oct 23

Joint or Per-Option? Compact Encoders for Closed-Set Decisions from 17 Million to 1 Billion Parameters

The paper investigates how compact an encoder can be for closed‑set decision tasks in language‑model pipelines, comparing joint versus per‑option scoring. In a 32‑million‑parameter study, per‑option scoring outperforms joint scoring by 5.8 chance‑corrected skill points on a 37‑benchmark suite, and two external benchmarks corroborate this advantage. Across six released joint encoders ranging from 17 million to 1 billion parameters, the 68‑ and 150‑million‑parameter models surpass a 435‑million‑parameter DeBERTa‑v3 zero‑shot classifier on 209 overlapping tasks and match it on 145 unseen tasks, while the smallest model processes a median request in 14 ms versus 728 ms for the classifier.

By Tardelli Ronan Coelho Stekel
arXiv AI
Oct 2

On-Device Named-Entity Recognition: A Deployability Study of Accuracy, Cost, Reliability, and Confidence

The paper evaluates nine on‑device named‑entity recognition models ranging from classical taggers to large language models, measuring not only accuracy but also latency and output validity. Using a silver‑gold benchmark derived from an LLM judge panel and a human‑validated corpus, the study shows that encoder‑based models achieve comparable accuracy to a 4 B instruct LLM while being much smaller, faster, and producing no malformed output. Confidence calibration of GLiNER is analyzed, revealing over‑confidence but improved reliability after temperature scaling and thresholding.

By Vinay Kumar Chaganti
arXiv Computation and Language
Oct 16

Intern-Decision: Efficient Multimodal Decision Making with Autoregressive Masked Language Modeling

Intern-Decision is a family of multimodal decision models (0.8B, 2B, and 4B parameters) that output structured choices and candidate probabilities directly from text and optional images, avoiding the latency of free‑form responses. The models use an autoregressive masked language modeling objective, placing one decision placeholder per field in a complete assistant JSON skeleton, and evaluate all fields in a single causal forward pass. The 4B variant achieves 90.02 % unweighted average accuracy across seven evaluation suites, with a mean latency of 44.16 ms on an RTX 4090 for a 289‑token request, and shows improved calibration and accuracy when combined with temperature scaling and confidence‑based handoff.

By Tong Zhu, Zhe Xu, Zefeng He, Yunkai Chai, Xuyang Hu, Dehong Zheng, Yun Luo, Yafu Li, Qi Zhang, Yu Cheng
arXiv Computation and Language
Nov 2

Completeness and Cost of Retrieval Structures for Multi-Tool Requests in LLM Agents

The paper evaluates six retrieval structures for multi‑tool requests in LLM agents, measuring how many required tools appear in the top‑k results and the associated LLM calls, tokens, and latency. Across two testbeds—345 AppWorld queries and 103 MCP‑Bench tasks—none of the untrained, single‑LLM‑call structures completes more than 10% of AppWorld queries at k = 16. A trained set‑level retriever and an LLM‑navigated hierarchy achieve higher completeness (up to 58% on MCP‑Bench) but at different costs and failure modes, with complete retrieval sets leading to 100% task success in a downstream AppWorld agent test.

By Shuyang Zhang
arXiv AI
Sep 17

Decodable but Misrouted: Sparse Features Uncover a Readout Gap in Vision-Language Models for Harmful Meme Detection

The paper investigates why large vision‑language models sometimes misclassify harmful memes, attributing failures to either missing internal evidence or poor routing of evidence to the output. Using sparse autoencoders, role‑conditioned probes, and causal interventions on Gemma‑3 and Qwen3.5, the authors show that sparse readouts consistently outperform native predictions across six harmful content benchmarks, revealing a readout gap that is largely due to routing rather than representation. The study also demonstrates that calibration‑only routing recovers most of the performance gap and that the issue persists across languages and is not solely driven by OCR signals.

By Girish A. Koushik, Diptesh Kanojia, Helen Treharne