arXiv:2610.35654v1 Announce Type: new Abstract: JEV decision models pair a language model backbone with a lightweight scoring head to select among natural-language options. Direct selection compares...
Low-rank adaptation (LoRA) has become a widely used parameter-efficient fine-tuning method for large language models. Since different modules and layers may contribute unequally to downstream adaptation, allocating rank resources under a fixed parameter budget is an important problem for balancing efficiency, expressiveness, and generalization.
arXiv:2607. 20205v1 Announce Type: cross Abstract: Low-rank adaptation (LoRA) has become a widely used parameter-efficient fine-tuning method for large language models.
arXiv:2505. 18877v4 Announce Type: replace Abstract: Low-Rank Adaptation (LoRA) lowers the computational and memory overhead of fine-tuning large models by updating a low-dimensional subspace of the pre-trained weight matrix.
ChainDoRA is a new parameter‑efficient fine‑tuning framework for large language models that replaces the dense low‑rank factorization of LoRA with a connected Tensor‑Train (TT) chain. By separating weight magnitude and direction and using a TT rank to control representation capacity, ChainDoRA achieves higher average accuracy on seven commonsense reasoning benchmarks while dramatically reducing trainable parameters—down to 5.35 M versus 56 M for LoRA and DoRA. Ablation studies show that the TT parameterization offers controllable trade‑offs between parameter cost and accuracy.
Glance is a retrieval‑augmented generation approach that eliminates the need for separate reranker models by adding a lightweight decision adapter and pointer head to a single 9B backbone. It scores candidate passages directly from a shared prefill, requiring only about 0.1 GiB of extra memory per decision skill. In fine‑tuning on MS MARCO, Glance outperforms the bge‑reranker‑v2‑m3 on SciFact and NFCorpus, and improves answer F1 on HotpotQA by 0.045, while scaling to 200 candidates it surpasses cross‑encoder reranking by 0.058 nDCG@10.
arXiv:2608. 11947v1 Announce Type: cross Abstract: Multiple-choice benchmarks are widely used to evaluate large language models, but MCQ scores conflate knowledge with sensitivity to option order, which makes them unreliable measures of model knowledge.
The study evaluates large language models (LLMs) as rerankers in conversational movie recommendation, comparing proprietary, open-weight, and fine-tuned LLMs against collaborative-filtering and sequential baselines on the ReDial benchmark. Results show that the best proprietary LLM achieves an NDCG@10 of 0.1497 with a shared semantic candidate pool, outperforming non-LLM baselines, while open-weight LLMs do not surpass a tuned shallow autoencoder under the same protocol. The analysis also highlights that reranker performance is highly sensitive to candidate generation, pool size, scoring policy, and decoding temperature, suggesting these factors should be reported as standard evaluation fields.
arXiv:2602. 09492v2 Announce Type: replace-cross Abstract: Low-rank adaptation (LoRA) is a standard approach for fine-tuning large language models, yet its many variants report conflicting empirical gains, often on the same benchmarks.
LOCUS is a post‑training technique that selects a task‑aware low‑rank adaptation subspace to reduce token‑cost while preserving utility. By updating only a tiny fraction of parameters (0.24–0.28 %) on 3 B‑parameter backbones, LOCUS cuts continuation length by up to 39.84 % on Pythia‑2.8B and 14.87–17.58 % on Qwen2.5‑3B, without materially affecting the internal preference diagnostic.
The paper investigates whether System‑1‑style decision models can replace dedicated rerankers in retrieval‑augmented generation (RAG) systems. After screening 21 open‑weight models on 11 benchmarks, the authors compare a promising decision model with a dedicated reranker on identical candidate lists. While limiting candidate text length improves reliability and makes some configurations competitive, a tuned reranker remains faster and uses less GPU memory, indicating that replacing a reranker requires balancing quality, latency, and memory.
arXiv:2606. 07610v1 Announce Type: cross Abstract: State-of-the-art GRPO-style methods for speech-aware large language model post-training suffer from coarse credit assignment, broadcasting the same terminal-reward advantage to every token in a response.