arXiv:2609.30316v1 Announce Type: cross Abstract: Language models used in financial backtests suffer from look-ahead bias, as a model trained on text published after the study period has already obse...
ASDA (Automated Skill Distillation and Adaptation) is a framework that improves large language models on financial reasoning tasks without fine‑tuning. It works by having a teacher model analyze a student’s failures, cluster errors, and generate structured skill artifacts—reasoning procedures, code templates, and worked examples—that are injected during inference. On the FAMMA benchmark, ASDA boosts arithmetic reasoning by up to 17.33% and non‑arithmetic reasoning by 5.95%, outperforming existing training‑free methods.
arXiv:2610.00767v1 Announce Type: cross Abstract: Pre-training interventions are critical to alignment research, since beliefs formed during pre-training shape how a model generalizes from later trai...
The paper introduces a data‑centric pipeline for post‑training language models on financial reasoning tasks. It mines open‑source reasoning traces, distills financial instruction data, and generates knowledge‑graph‑guided question‑answer pairs, then filters examples with lightweight classifiers and applies reinforcement learning with rule‑based verifiers. Experiments on FINESSE‑Bench show that retention‑aware adaptation—self‑distilled fine‑tuning and model merging—outperforms ordinary supervised fine‑tuning, improving accuracy by up to 3.0 points and avoiding regressions.
arXiv:2605.10194v2 Announce Type: replace Abstract: On-policy self-distillation (OPSD) uses a model as its own teacher under privileged context, providing token-level supervision on the model's own r...
arXiv:2605. 15532v3 Announce Type: replace-cross Abstract: Distillation enables compact Vision-Language Models (VLMs) to obtain strong reasoning capabilities, yet the prompts driving this process are typically chosen via simple heuristics or aggregated from off-the-shelf datasets.
arXiv:2606. 24747v1 Announce Type: new Abstract: Large Language Models (LLMs) achieve strong performance across a growing range of domains, yet their scale poses deployment challenges in applications where latency and cost constraints are critical.
arXiv:2607. 20433v1 Announce Type: cross Abstract: While language models remain frozen at their training state, the world evolves continuously.
arXiv:2608. 04374v1 Announce Type: cross Abstract: Large language models can produce fluent financial analysis, but fluency alone does not establish whether a report is suitable for institutional delivery.
Pre-training interventions are critical to alignment research, since beliefs formed during pre-training shape how a model generalizes from later training. One recently popular technique for such inter...
FrontierMat is a new benchmark comprising 330 real-world problems from cutting-edge materials research in areas such as aerospace, semiconductors, embodied intelligence, and nuclear fusion. Each problem is validated by experts and requires a language model to infer a withheld outcome or draw a conclusion from experimental evidence, with 130 problems demanding cross‑scale reasoning. The strongest general‑purpose model scores only 53.7%, and domain‑adaptation techniques can raise performance to 55.4%, matching the best proprietary models while improving on public materials benchmarks.
The paper introduces a continual learning approach for large language models called Multi‑Teacher On‑Policy Distillation (MOPD) that avoids storing intermediate expert snapshots by replaying prompts from earlier domains using the previous student as a teacher. Two replay strategies are examined: a shared replay budget across domains and a fixed budget per domain, with replay steps that either distill token distributions alone or also align hidden states, attention scores, and value relations. Experiments on a five‑domain benchmark (logic, math, code, safety, instruction following) demonstrate that replay alone closes most of the performance gap between sequential distillation and joint MOPD, achieving comparable results to recovery from all stored snapshots when using a fixed replay share per domain.