arXiv AI
Jul 1
Large Databases Need Small, Open-Weight Language Models
arXiv:2606. 31808v1 Announce Type: new Abstract: Language model systems built around proprietary APIs often operate on a token-based cost model.
Apertus LLM Family Expansion via Distillation and Quantization
arXiv:2605. 29128v2 Announce Type: replace Abstract: The wide adoption of LLMs has led to their use in great variety of applications and scenarios, such as chatbot assistants and data annotation, creating the need for the models to satisfy certain budget and hardware constraints.
Related stories
arXiv AI
Jun 16
LLM4RTL: Tool-Assisted LLM for RTL Generation
arXiv:2606. 15500v1 Announce Type: cross Abstract: Large language models (LLMs) have facilitated impressive progress in software engineering, code generation, tooling, and systems.
arXiv AI
Jun 26
CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs
arXiv:2606. 26650v1 Announce Type: cross Abstract: In this paper, we present CAT-Q, Cost-efficient and Accurate Ternary Quantization, for compressing and accelerating LLMs.
arXiv AI
Aug 5
TQLite: Multi-LLM Jury Guided Distillation for Real-time MQM Translation Quality Evaluation
arXiv:2608. 02975v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated impressive performance in MQM-based translation quality (TQ) evaluation, and recent advances in large reasoning models (LRMs) promise even greater improvements.
Hugging Face Trending Papers
Jun 25
CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs
In this paper, we present CAT-Q, Cost-efficient and Accurate Ternary Quantization, for compressing and accelerating LLMs. Unlike existing state-of-the-art ternary quantization methods that rely on data-intensive and costly quantization-aware training to mitigate severe performance degradation, CAT-Q is a simple yet effective post-training quantization scheme that is readily applicable to LLMs with diverse architectures and model sizes.
arXiv AI
22h ago
Beyond FLOPs: Energy-Aware Knowledge Distillation for Sustainable LLMs on Code-Related Task
arXiv:2608. 17515v1 Announce Type: cross Abstract: Background: Large Language Models (LLMs) are increasingly being applied to Software Engineering (SE) tasks, achieving high accuracy across problems such as clone detection, vulnerability prediction, and code summarization.
arXiv Machine Learning
Jul 8
FourTune: Towards Fully 4-Bit Efficient Post-Training for Diffusion Models
arXiv:2607. 05711v1 Announce Type: new Abstract: Diffusion models have become a dominant paradigm for high-quality generative modeling, while post-training is essential for adapting them to diverse downstream applications.
arXiv AI
Jul 7
Nemotron-Labs-3-Puzzle-75B-A9B: Compressing Hybrid MoE LLMs
arXiv:2607. 04371v1 Announce Type: new Abstract: We present Nemotron-Labs-3-Puzzle-75B-A9B, a compressed variant of Nemotron-3-Super optimized for interactive deployment.
OpenAI Blog
Oct 1, 2024
Model Distillation in the API
Fine-tune a cost-efficient model with the outputs of a large frontier model–all on the OpenAI platform
arXiv Machine Learning
Jul 28
Benchmarking LLMs for Verilog Design Flows
arXiv:2607. 22759v1 Announce Type: cross Abstract: Large language models (LLMs) show promise in code generation, but their capabilities to produce correct, synthesizable hardware description language (HDL) code still remain to be properly benchmarked.
arXiv AI
Jun 30
HARD-KV: Head-Adaptive Regularization for Decoding-time KV Compression
arXiv:2606. 28831v1 Announce Type: cross Abstract: Long-context LLM inference faces a fundamental conflict: head-adaptive compression algorithms (e.
arXiv AI
Jun 4
Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data
arXiv:2606. 04238v1 Announce Type: cross Abstract: Aggressive weight quantization to 2-bit precision offers substantial throughput and memory gains for large language model (LLM) inference, but typically incurs severe accuracy degradation.