← Back to all news
Hugging Face Trending Papers

Benchmarking Trustworthiness of SLMs: Pre-trained vs. Compressed

Read the original on Hugging Face Trending Papers →

Small Language Models (SLMs) have emerged as a more efficient alternative to traditional Large Language Models (LLMs), offering promising potential in resource-constrained scenarios. Existing approaches to building SLMs typically follow two paths: training compact models from scratch, or compressing larger pre-trained models using methods such as pruning, quantization, or distillation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

Related stories

arXiv AI
Aug 19

Benchmarking the Benchmarks: Evaluating Automated Safety Benchmarks for Small Language Models

The paper investigates whether existing AI safety benchmarks, designed for large language models, are suitable for evaluating small language models (SLMs). By testing five benchmark suites on 26 open‑source SLMs with a unified scoring rubric, the authors find that ambiguous judgments dominate, especially for complex prompts and certain architectures. This ambiguity, linked to factors like lexical density and output perplexity, undermines the reliability of aggregate leaderboards and reveals a confound between model capability and perceived safety.

By Nyamtulla Shaik, Fengjun Li, Bo Luo
llmsbenchmarkssafety
More like this →
arXiv AI
Sep 4

VoxPrivacy: A Benchmark for Evaluating Interactional Privacy of Speech Language Models

The paper introduces VoxPrivacy, a benchmark for assessing interactional privacy in Speech Language Models (SLMs). It evaluates models on a 32‑hour bilingual dataset across three difficulty tiers, revealing that most open‑source SLMs perform near random on conditional privacy decisions and even strong closed‑source systems struggle with proactive privacy inference. The authors also validate these findings on a real‑speech subset and show that fine‑tuning on a 4,000‑hour training set can improve privacy‑preserving capabilities while maintaining robustness.

By Yuxiang Wang, Hongyu Liu, Dekun Chen, Xueyao Zhang, Zhizheng Wu
llmsfine-tuningbenchmarkssafety
More like this →