The article "The Life of a Token: from Words to Bits on the Wire" explores how large language models convert text into network traffic during training. It traces the transformation from words to tokens, then to vectors, and finally to binary streams that traverse high‑performance computing systems. Using Dante’s Divine Comedy as a case study, the tutorial examines how tokenization, embeddings, and parallelization affect the volume, structure, and timing of data exchanged across the network, and provides analytical traffic models and numerical examples to clarify the communication demands of LLM training.
By Davide Avesani (CEDRIC - ROC), Pengwenlong Gu (CEDRIC - ROC), Sotiris Skaperas (Cnam), Stefano Secci (CEDRIC - ROC)
The paper introduces a Transformer-based Quantum State Characterizer (TQSC) designed to accurately reconstruct photonic polarization states in remote state preparation (RSP) experiments subject to complex scattering and dynamic Gaussian noise. TQSC achieves over 99.999% fidelity for noisy measurements and demonstrates significant error reduction in a practical MNIST image transmission task, while its attention mechanisms offer physically grounded insights into observable correlations. The model’s robustness is further validated on Qiskit-simulated Bloch-ball states, indicating its potential for intelligent quantum information processing.
By Bo Tang, Zixuan Liao, Hao Li, Yilin Yang, Jiani Lei, Zengya Li, Jing Qiu, Zhaohui Dong, Zhengyang Mao, Yuanhua Li, Yuanlin Zheng, Xianfeng Chen
The paper introduces low‑rank orthogonalization, a technique that exploits the low‑rank nature of gradients in neural network training to perform matrix orthogonalization more efficiently. Building on this, the authors present low‑rank matrix‑signed gradient descent (MSGD) and a low‑rank variant of the Muon optimizer, showing through experiments that low‑rank Muon matches or surpasses vanilla Muon on GPT‑2 and LLaMA pretraining, especially for larger models. Theoretical analysis provides iteration‑complexity bounds for both low‑rank MSGD and low‑rank Muon under heavy‑tailed noise.
By Chuan He, Zhanwang Deng, Zhaosong Lu
The paper introduces L2C‑TFM, a reinforcement‑learning framework for cleaning tabular data before feeding it to Tabular Foundation Models (TFMs). It proposes a model‑aware reward that regularizes the Wasserstein distance between cleaned and dirty data, aiming to preserve distributional stability. Experiments on ten OpenML datasets show that while some reward designs fail, the model‑aware reward performs comparably to a random‑forest baseline and improves minority‑class macro‑F1 under class imbalance, and a policy trained on one dataset can transfer to others.
By Laure Berti-Equille
The paper introduces a new benchmark for assessing out-of-distribution robustness in graph-based Android malware classifiers, highlighting that current models drop up to 45% accuracy on unseen malware variants. It presents two scenarios—MalNet-Tiny-Common for covariate shift and MalNet-Tiny-Distinct for domain shift—and identifies a limitation in existing benchmarks that rely solely on structure-only function call graphs. To address this, the authors propose a semantic enrichment framework that augments graph topology with function-level attributes and LLM-based code embeddings, demonstrating that this data-centric approach improves robustness under distribution shift and complements model-based methods.
By Ngoc N. Tran, Anwar Said, Waseem Abbas, Tyler Derr, Xenofon D. Koutsoukos
The paper compares convolutional neural networks (CNN), Vision Transformers (ViT), and hierarchical Swin Transformers for quark‑gluon jet classification using a three‑channel jet‑image representation. CNN and Swin models outperform ViT, indicating that local jet substructure is crucial for discrimination. The study also shows that block‑wise fine‑tuning, Momentum Contrast pretraining, and a compact Swin variant can improve performance while reducing parameters.
By Daeun Kim, Jaeyoon Cho, Jiwon Lee, Wonjun Jeong, Hyeongwoo Noh, Giyeong Kim, Seunghwan Yang, MinJung Kweon
The paper introduces the Modality Discrepancy Transformer (MDT), a model designed to detect ambivalence and hesitancy in clinical videos by capturing cross‑modal disagreement across facial, vocal, and linguistic signals. MDT expands a 6‑token representation to 9 tokens that include modality embeddings, absolute‑difference features, and Hadamard‑product discrepancy features, which are processed through Transformer self‑attention with FiLM‑based text conditioning and LoRA fine‑tuning. On the BAH dataset from the 3rd ABAW Challenge, MDT achieves a Macro F1 score of 0.7408 on the labelled test split and 0.7368 on the private leaderboard, surpassing the strongest baseline by over 10 points while training in under 20 minutes on a single GPU.
By Shiyu Luo, Yu Wang, Jiawen Huang, Zhaoxiang Xiao, Chenxi Huang, Qi Zhang, Bin Liu
Neo-Classic is a new benchmark designed to evaluate linguistic‑aesthetic reasoning in Classical Chinese poetry. It uses an out‑of‑sample dataset of strictly metrical poems written by contemporary experts and a set of reverse‑understanding probes, avoiding reliance on historical corpora. Experiments with leading LLMs show a 20–50% performance drop on contemporary texts and low accuracy (0–13%) on discourse‑level ordering, indicating that current models excel at local patterns but struggle with global hierarchical planning.
By Han Zhang, Zihan Gu, Zhiyuan Wang, Tianyi Ma, Jiacheng Lu, Xinyan Zhang, Yuhao Wei, Cheng Hua
The paper introduces a frontend‑backend architecture for full‑duplex speech‑to‑speech models that enables tool calls while preserving natural conversational flow. The frontend emits a delegation token and streams ASR transcripts to a text‑based backend LLM, which performs tool calls and returns results that are re‑injected into the frontend via a lightweight prefill‑and‑repeat mechanism before streaming TTS synthesis. In single‑turn evaluations the system achieves 92‑97% tool‑call recall, 81.2% accuracy in rejecting irrelevant calls, and competitive performance on Full‑Duplex‑Bench‑V3 and EVA‑Bench when paired with a large backend model.
By Ke Hu, Slyne Deng, Chen Chen, Elena Rastorgueva, Edresson Casanova, Punit Kumar, Dharmendra Choudhary, Nikhil Srihari, Ameya Sunil Mahabaleshwarkar, Viet Anh Trinh, Slim Essid, Oluwatobi Olabiyi, Zhehuai Chen
Reflective Recovery is a self‑supervised method that turns failed reasoning attempts into training data, enabling large language models to learn how to correct mistakes during inference. By extracting initial segments of erroneous trajectories and using them as prompts, the approach teaches models to recognize and recover from errors without external critics. Experiments show significant accuracy gains on benchmarks such as AIME 2025 and Minerva, and the method overcomes the scaling collapse problem, fostering emergent self‑correction behaviors.
By Qirui Chen, Renjie Pi, Jiahui Gao, Lingpeng Kong
The study investigates whether pretrained transformer models encode functional words—such as pronouns and adverbs—in a way that mirrors human usage. By comparing embeddings of nouns with those of their functional counterparts in both isolated and parallel sentences, the authors find that functional words occupy a central yet distinct position in embedding space and that parallel lexicalized and functional sentences reside in different subspaces. Experiments show that only a mixed training set of functional and lexicalized sentences reveals shared syntactic and semantic structure, whereas training on either type alone fails to capture this parallelism.
By Giuseppe Samo, Vivi Nastase, Paola Merlo
Compass‑v3 is a 245B‑parameter Mixture‑of‑Experts language model tailored for Southeast Asian e‑commerce, featuring 71B active parameters per token and hardware‑efficient expert parallelism. It is trained on 12 trillion multilingual tokens and synthetic e‑commerce instructions, and incorporates Optimal‑Transport Direct Preference Optimization to improve instruction adherence. Benchmarks show it outperforms GPT‑4, DeepSeek‑V3.1, and Qwen3‑235B, and it is already deployed at scale on Shopee, handling over 70% of the platform’s LLM traffic.
By Sophia Maria
CORTEX is a novel framework that transforms web‑scale corpus construction from flat document filtering into structured knowledge organization using an Ontological Corpus Graph (OCG). The OCG comprises a quality‑refined content layer, a lightweight ontology layer that evolves via LLMs, and a cross‑domain alignment layer that supports arbitrary taxonomic resolution. Experiments demonstrate CORTEX’s effectiveness, and the authors release a 24.14 B‑token refined corpus, its OCG, and a cross‑domain benchmark called CortexBench for evaluating large language models.
By Chengtao Gan, Xiaoke Guo, Yushan Zhu, Zhaoyan Gong, Zhiqiang Liu, Songze Li, Huajun Chen, Wen Zhang
The paper introduces an adaptive region‑dividing strategy that projects a 3D point cloud onto a bird’s‑eye‑view plane to detect building regions, then back‑projects bounding boxes to create structure‑aligned training blocks for unified scene‑level evaluation. It also proposes a fine‑grained classification model using a point transformer classifier and a spatially‑supervised contrastive loss to improve inter‑class discriminability, addressing class imbalance with a weighted cross‑entropy. Experiments on UrbanBIS and STPLS3D datasets show the method outperforms state‑of‑the‑art approaches in both building instance segmentation and fine‑grained classification.
By Weiyuan Zhang, Qi Zhang, Hui Huang
The paper introduces a new task called visual scene negative captioning, which aims to describe what should be present in an image but is actually absent, a capability crucial for safety-critical applications. It proposes the CRCD framework, which uses counterfactual reconstruction and contrastive decoding to overcome affirmation bias, limited mental filling, and representation bias. CRCD employs a dual-branch architecture for amodal completion and functional association, along with multi-condition representation learning, to generate accurate negative captions and sets a high-performance baseline for this emerging task.
By Zhiyun Jiang, Hanyong Wang, Binbin Liang, Yu Xie, Menglong Yang, Wei Li
QCPruner is a training‑free visual token pruning method that conditions both token selection and representation on the query via bilateral utility weighting. It fuses keyword‑matched query anchors with cross‑modal cues to compute a nonnegative facility‑location objective that is monotone and submodular, guaranteeing a (1‑1/e) greedy approximation. Across multiple multimodal large language models, QCPruner consistently outperforms existing pruning methods, achieving over 96% of unpruned performance even with very few tokens retained.
By Shengli He (Guizhou University), Yongchao Liang (Guizhou University), Roumeng He (Shanghai Ocean University), Junjie Zeng (Guizhou University), Jiyuan He (Guizhou University), Can Wu (Guizhou University), Li Zheng (Guizhou University)
The paper introduces Latent-Centroid Steering (LCS), a single-pass classifier-free guidance method for vision‑language autonomous driving models. LCS replaces instance‑level residuals with class‑level latent shifts, projecting conditional representations toward precomputed command‑specific centroids to enhance command adherence. Experiments on Bench2Drive and nuScenes show that LCS cuts inference latency by about 50% while improving driving performance.
By Meibo Hu, Jiamian Wang, Pichao Wang, Zhiqiang Tao
The paper investigates whether large language models (LLMs) can internally detect harmful content, bypassing external guardrails that add latency and computational cost. By extracting activations from LLaMA‑3.1‑8B and training lightweight MLP probes, the authors achieve high F1 scores (99%, 83%, and 84%) on WildJailbreak, Beavertails, and AEGIS 2.0 benchmarks, rivaling much larger guard models while reducing overhead. This suggests that internal state monitoring can provide efficient safety checks for resource‑constrained, time‑critical deployments.
By Alizishaan Khatri, Chiquita Prabhu, Omkar Neogi
FINSKILLOPS is a multi‑agent system designed to improve SEC filing question‑answering after deployment by creating reusable skills from evidence‑grounded failure diagnoses. It manages these skills through targeted validation, regression checks, negative controls, and versioned replacement or retirement, ensuring that new patches do not introduce regressions. Across six benchmarks, a frozen skill registry outperforms other systems, and in a 12‑round operational study the system reduced the non‑correct rate from 20.0% to 12.5% while promoting only six of 33 proposed skills.
By Yanzhang Ma, Zhenghan Tai, Hanwei Wu, Sizhe Guan, Jianliang Lei, Hailin He, Chaolong Jiang, Jijun Chi, Tung Sum Thomas Kwok, Bohuai Xiao, Jingrui Tian, Xinlu Wu, Xingao Zhan, Peng Lu, Muzhi Li, Yihong Wu, Liheng Ma, Sicheng Lyu, Tianshuo Yan, Junhao Zhu, Yaqian Xu, Lei Ding, Yufei Cui, Ziquan Liu, Boyu Han, Hengli Liu, Ling Zhou, Xinyu Wang
The paper introduces the Generic Multi-Agent Trading System (GMATS), a framework for studying how large language model (LLM) based trading stacks react to black-box, input-only attacks that inject plausible social‑media content. It defines contagion metrics—belief‑shift scores at analyst and coordinator layers and attack‑clean deltas on backtest metrics—to trace the spread of adversarial signals. Experiments on a safe offline benchmark show that even simple attackers can significantly degrade risk‑return profiles, while certain multi‑agent topologies and coordinator prompts can mitigate these effects.
By Qi Rong Sua, Junhao Dong, Nguyen Duc Thai, Yuqing Wen, Cheston Tan, Yew-Soon Ong