Natural language processing

Classical and neural NLP: translation, question answering, tokenization and the evaluation of language understanding.

1,293 stories · RSS feed

arXiv AI

2d ago

No Universal Signal Predicts Sample-Level LLM Regression under Version Updates

arXiv:2608. 13607v1 Announce Type: new Abstract: Frontier LLMs are updated frequently and typically outperform their predecessors in aggregate.

By Jia Sheng, Yiwei Lu

llms nlp benchmarks

arXiv AI

2d ago

BUZZY: Contrastive Scoring to Mitigate Text-Induced Bias in Multimodal Multiple-Choice QA

arXiv:2603. 28026v2 Announce Type: replace Abstract: Multimodal multiple-choice question answering (MCQA) provides a standardized and objectively measurable setting for evaluating vision-language models (VLMs).

By Taeyun Roh, Suhyeong Park, Dongyoung Lee, Eunyeong Jo, Wonjune Jang, Junha Jung, Jaewoo Kang

llms nlp efficiency multimodal benchmarks safety

arXiv AI

2d ago

A Pathway to General-Purpose Scientific AI: Multimodal Comprehension of Scientific Images

arXiv:2608. 14075v1 Announce Type: new Abstract: Scientific figures and tables encode essential experimental evidence, yet remain difficult for digital libraries and multimodal AI systems to retrieve and interpret.

By Jennifer D'Souza, Fahad Ahmed, Cecilia Andrea Bustamante Andrade, Lina Frolova, Poorani Gnanasambandan, Dilshad Hussain, Muhammad Uzair Khan, Nkembeng Kevin Nkengfoa, Paul Praveen J., Fabio Priante, Sjoerd Franciscus van der Werf, Thomas Frederik Jan van Roeden

nlp multimodal benchmarks

arXiv Machine Learning

2d ago

Semantic Differentiation for Tackling Challenges in Watermarking Low-Entropy Constrained Generation Outputs

arXiv:2601. 11629v2 Announce Type: replace-cross Abstract: We demonstrate that while the current approaches for language model watermarking are effective for open-ended generation, they are inadequate at watermarking LM outputs for constrained generation tasks with low-entropy output spaces.

By Nghia T. Le, Alan Ritter, Kartik Goyal

llms nlp

arXiv Machine Learning

2d ago

Geometric Filtering of LLM-Generated Samples for Few-Shot Text Classification

arXiv:2608. 13866v1 Announce Type: new Abstract: Large language models (LLMs) can generate synthetic training data for text classification, but the quality of generated samples is heterogeneous: some fall in correct class regions of the embedding space while others land in peripheral or cross-class zones.

By Benjam\'in Schindler, Gonzalo A. Ruz

llms rag nlp

arXiv AI

2d ago

Leveraging Few-Shot Learning and Large Language Models for Analyzing Blood Pressure Variations Across Biological Sex from Scientific Literature

arXiv:2402. 01826v2 Announce Type: replace-cross Abstract: Current blood pressure (BP) technologies and standards were established decades ago, and these standards are still used worldwide today, often without adjusting BP readings for individual demographic factors such as sex and age.

By Yuting Guo, Seyedeh Somayyeh Mousavi, Reza Sameni, Abeed Sarker

llms nlp

arXiv AI

2d ago

Adaptive Stopping for Multi-Turn LLM Reasoning

arXiv:2604. 01413v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) increasingly rely on multi-turn reasoning and interaction, such as adaptive retrieval-augmented generation (RAG) and ReAct-style agents, to answer difficult questions.

By Xiaofan Zhou, Huy Nguyen, Bo Yu, Chenxi Liu, Lu Cheng

llms rag agents nlp benchmarks

arXiv AI

2d ago

TimeSage-EV: A Live Benchmark for Agentic Time Series Analysis in Evolving Environments

arXiv:2608. 14270v1 Announce Type: new Abstract: Time series analysis in high-stakes domains relies on recurring data releases, where new observations can alter the evidence base and the validity of later conclusions.

By Qingren Yao, Yaxuan Kong, Yuqi Nie, Yichen Li, Stefan Zohren, Anna Vettoruzzo, Qingsong Wen, Ming Jin, Joaquin Vanschoren

llms agents nlp benchmarks

arXiv AI

2d ago

APTER: Adaptive Post-Training with Expert-Grounded Rubrics

arXiv:2608. 14212v1 Announce Type: new Abstract: As large language models enter professional domains, they must satisfy domain constraints, include critical evidence, and provide complete reasoning rather than merely produce fluent responses.

By Xukai Wang, Liangqi Li, Zhiyue Xu, Jingang Zhou, Xiaoyu Shi, Jiansheng Cai, Bo Zhang, Zhe Li, Xu-Yao Zhang

llms nlp reinforcement-learning fine-tuning

arXiv AI

2d ago

Measuring Fairness in Large Audio Language Models via Semantic-Aware Bias Estimation

arXiv:2608. 13624v1 Announce Type: cross Abstract: Large Audio Language Models (LALMs) have seen increasing use for audio understanding tasks such as speech recognition and audio question answering, raising concerns about fairness across demographic subgroups.

By Zhe Liu

llms rag nlp multimodal benchmarks safety

arXiv AI

2d ago

Seeing Red, Thinking Bad: Color Bias in Vision Language Models

arXiv:2608. 14286v1 Announce Type: cross Abstract: Vision language models (VLMs) are increasingly used in industrial decision-making systems, such as recruitment support and recommendation.

By Kohsuke Ide, Ryousuke Yamada, Yoshihiro Fukuhara, Hirokatsu Kataoka, Yutaka Satoh

llms nlp multimodal safety

arXiv AI

2d ago

Can Language Models Understand mmWave Data? Benchmarking Large Language Models for mmWave Radar-Based Human Understanding

arXiv:2608. 14179v1 Announce Type: new Abstract: Large language models (LLMs) have shown remarkable reasoning and generative capabilities, motivating their use as universal reasoning engines for perception.

By Jeongwan Shin, Jaehyeon Kim, Donguk Ko, Jaeho Choi

llms nlp multimodal benchmarks safety

arXiv AI

2d ago

Not All Tokens Are Equal: Inflation-Aware Routing for Agentic LLM Systems

arXiv:2608. 13571v1 Announce Type: cross Abstract: When a language model fails to answer a query on the first attempt, an agentic system retries, consuming additional tokens each time.

By Heming Fu, Shan Lin, Qianqian Xie, Guojun Xiong

llms agents nlp

arXiv AI

5d ago

Trie Automata for Constrained Decoding over Large Finite Sets

arXiv:2608. 12574v1 Announce Type: new Abstract: Large language models increasingly need to generate structured outputs that conform to predefined schemas, with one common constraint being selection from a finite set of valid strings.

By Xingzi Xu, Karim Bouyarmane

llms nlp efficiency

arXiv AI

5d ago

Polish Medical Visual Question Answering: Vision-Language Models Underutilize Visual Evidence

arXiv:2608. 12928v1 Announce Type: new Abstract: We introduce a Polish-language medical visual question answering (VQA) benchmark, built from Polish Board Certification Examination questions for licensed physicians and dentists pursuing specialist certification.

By Jakub Pokrywka, {\L}ukasz Grzybowski, Antoni Lasik, Marek Kubis, Jeremi Ignacy Kaczmarek, Wojciech Kusa

llms nlp multimodal benchmarks

arXiv AI

5d ago

SPADE: Speculative Decoding for Precise and Low Cost Distributed Edge Cloud Inference

arXiv:2608. 13076v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved remarkable success in natural language understanding and generation, but their deployment is constrained by high computational demands.

By Divya Jyoti Bajpai, Kishan Kumar Upadhyay, Manjesh Kumar Hanawal

llms nlp

arXiv AI

5d ago

Better Decomposition, Free Aggregation: A Synthesizer-Folding Framework for Multilingual Multi-Hop Question Answering

arXiv:2608. 13160v1 Announce Type: cross Abstract: Multilingual retrieval-augmented generation (mRAG) equips large language models with access to globally distributed external knowledge for complex multilingual question answering.

By Yilin Wang, Yuchun Fan, Weidong Bao, Zili Wei, Shi Feng, Tong Xiao, Zhengtao Yu, Jingbo Zhu

llms rag nlp safety

arXiv AI

5d ago

Numeracy in Large Language Models: Fundamental Limitations and Paths to Improvement

arXiv:2608. 13129v1 Announce Type: new Abstract: Large language models (LLMs) achieve strong results on mathematical reasoning benchmarks yet remain unreliable on elementary numerical tasks, including magnitude comparison, large-integer arithmetic, fractions, and scientific notation.

By Aoxin Ni

llms rag nlp fine-tuning benchmarks

arXiv AI

5d ago

DiffGRM: Diffusion-based Generative Recommendation Model

arXiv:2510. 21805v2 Announce Type: replace-cross Abstract: Generative recommendation (GR) is an emerging paradigm that represents each item via a tokenizer as an n-digit semantic ID (SID) and predicts the next item by autoregressively generating its SID conditioned on the user's history.

By Zhao Liu, Yichen Zhu, Yiqing Yang, Xiao Lv, Guoping Tang, Rui Huang, Qiang Luo, Ruiming Tang, Kun Gai, Guorui Zhou

diffusion nlp

arXiv AI

5d ago

Into the ORBIT for Time Series: Training Regimes for Foundation Models

arXiv:2608. 13262v1 Announce Type: cross Abstract: Time series foundation models (TSFMs) have advanced primarily through architectural innovation, while training regimes for large-scale heterogeneous corpora remain under-explored.

By Hongjie Xia, Yiding Liu, Yifan Hu, Peiyuan Liu, Zewei Dong

llms nlp safety