The paper presents the AI Appeals Processor, a deep‑learning system deployed in a CPU‑only government environment to classify and route citizen appeals. On a dataset of 10,000 Russian‑language appeals, multilingual BERT achieved 82% accuracy on a three‑way appeal‑type task, outperforming other models such as fastText, Word2Vec+LSTM, and SVM. After deployment, the system reduced end‑to‑end handling time by 53‑56% and, following several retraining cycles, achieved over 95% correct classifications in production, though the high accuracy is not directly comparable to the test‑set result due to taxonomy issues.
arXiv:2605. 29738v2 Announce Type: replace-cross Abstract: Legal NLP benchmarks overwhelmingly evaluate a single language or aggregate tasks that differ fundamentally across jurisdictions, making cross-lingual comparison impossible.
The paper evaluates legal text classification models for Korean sexual offense cases, comparing traditional machine learning, large language models, and fine‑tuned domain models. Fine‑tuned KLUE‑BERT achieved the highest accuracy of 99.3%, outperforming GPT‑3.5, GPT‑4.0, and other traditional approaches. Explainable AI techniques were used to analyze predictions, revealing linguistic features that influence decisions and highlighting limitations in capturing subtle textual cues, especially in real‑world KICS data.
The paper introduces a lifecycle framework for LLM-as-a-Judge systems used to evaluate recommendation explanations at Netflix. It outlines four phases—Birth, Training, Deployment, and Monitoring—detailing how each stage addresses specific technical and operational challenges. The authors report that after five weeks of A/B testing, judge-aligned explanations increased novel content viewing and successful browse-to-play sessions without quality takedowns.
arXiv:2601.14172v4 Announce Type: replace-cross Abstract: We study neural multi-label classification under severe label imbalance through sentence-level detection of the 19 refined Schwartz human val...
arXiv:2012. 02110v2 Announce Type: replace-cross Abstract: Pre-trained language models have significantly advanced natural language processing (NLP), especially with the introduction of BERT and its optimized version, RoBERTa.
arXiv:2509.00141v2 Announce Type: replace-cross Abstract: Statutory corpora and judicial decisions are growing faster than legal professionals can read them, while individual judgments often exceed t...
The paper introduces a method for determining whether retrieval-augmented generation (RAG) systems have sufficient, insufficient, or conflicting evidence to answer a question. By training a lightweight linear classifier on hidden activations and attention-derived features from 16 language models, the authors demonstrate that these internal signals reliably predict the adequacy of retrieved documents, outperforming prompting-based baselines and specialized RAG models. Analysis shows that middle-layer hidden states carry the most informative signals for this triage task.
arXiv:2608. 10939v1 Announce Type: cross Abstract: Multilingual short-text classification supports operational systems such as content moderation, customer support routing, and intent recognition, yet aggregate evaluation often hides large differences between high-resource and low-resource languages.
arXiv:2607. 18785v2 Announce Type: replace Abstract: As large language model agents gain access to increasingly large skill libraries, retrieving the right skill becomes critical to reliable capability selection and execution.
The paper demonstrates that large language model (LLM) evaluators, whether reward‑model based or prompted LLM‑as‑a‑Judge, exhibit significant language bias in multilingual settings. Experiments with semantically identical instruction‑response pairs across 23 languages reveal that lower‑resource languages receive higher scores, a bias that persists across eight open‑weight evaluators and is not detectable by standard pairwise accuracy metrics. The authors link the bias to model uncertainty and language identity, showing it cannot be explained by content difficulty alone.
DeepOpen is a decision‑oriented intent‑classification framework tailored for the BANKING77 and CLINC150 datasets. On BANKING77, a single English classifier trained with sharpness‑aware minimization (SAM) achieved a test accuracy of 94.1234% (2,899/3,080), the highest among three fixed SAM seeds. For CLINC150, a probability fusion of one Laya specialist and three DeBERTa‑v3‑large classifiers reached 98.0222% accuracy (4,411/4,500). The study reports strong task‑specific performance, notes selection effects, and highlights unresolved historical execution‑path differences rather than a new algorithmic advantage.
arXiv:2607. 25891v1 Announce Type: new Abstract: Evaluating AI agents in interactive environments is hindered by fragmented tasks, scaffolds, verifiers, and scoring rules.