← Back to all news
arXiv Computation and Language By Joshua Wong, Chris Tanner

Analyzing Traditional and Neural Approaches to Multilingual Readability Assessment

Read the original on arXiv Computation and Language →

The paper investigates how transformer-based models and traditional feature-based models capture readability signals across five languages using the ReadMe++ dataset. By applying SHAP to identify key features for traditional classifiers and then probing XLM‑R and language‑specific encoders with TCAV, the authors find that transformers recover surface‑length, syntactic, and lexical‑diversity cues and reflect the CEFR ordinal structure, though alignment varies by model family, language, and layer. The study highlights that high linear separability does not guarantee directional influence, cautioning against overreliance on linear probing for readability features.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Computation and Language
Sep 7

A Systematic Comparison of Multilingual Interpretability Methods Reveals Anisotropy-Driven Failures

The paper evaluates four metrics—CKA, ANC, GMM dominance per token, and ILO—used to measure cross‑lingual representation sharing in multilingual language models. Across 21 models ranging from 125 M to 14 B parameters, the metrics disagree, and the authors attribute this to anisotropy, where representations cluster in a narrow embedding cone. Only ILO shows a strong, robust correlation with cross‑lingual transfer performance (Spearman’s ρ = 0.90) after controlling for model size, family, and task variation, leading the authors to recommend ILO as the primary metric alongside anisotropy diagnostics.

By Oskar Holmstr\"om, Marcel Bollmann, Marco Kuhlmann
llmsragsafety
More like this →
arXiv Computation and Language
Sep 4

Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probe

The paper introduces an Encoding Probe that reconstructs language model representations using interpretable features, addressing limitations of traditional decoding probes such as incomparable feature contributions and correlation effects. It evaluates this approach on text and speech transformer models, examining features from acoustics, phonetics, syntax, lexicon, and speaker identity. Findings reveal that speaker-related effects vary with training objectives and datasets, while syntactic and lexical features independently contribute to reconstruction, offering a complementary perspective on model interpretation.

By Gaofei Shen, Martijn Bentum, Tomas O. Lentz, Afra Alishahi, Grzegorz Chrupa{\l}a
llms
More like this →
arXiv AI
Jun 6

From Scoring to Explanations: Evaluating SHAP and LLM Rationales for Rubric-based Teaching Quality Assessment

arXiv:2606. 05180v1 Announce Type: cross Abstract: Automated scoring models are increasingly used to assign rubric-based quality ratings to complex language performances, including classroom transcripts, yet they typically provide little insight into why a particular score is produced.

By Ivo Bueno, Babette B\"uhler, Philipp Stark, Tim F\"utterer, Ulrich Trautwein, Dorottya Demszky, Heather Hill, Enkelejda Kasneci
llmsfine-tuningsafety
More like this →