← Back to all news
arXiv Computation and Language

Effects of Transcript Compression on LLM-based Medical Misinformation Detection in Japanese YouTube Videos

The study investigates how different transcript compression methods affect large language model (LLM) detection of medical misinformation in Japanese YouTube videos. Four input designs were compared: full transcripts, LLM-generated summaries, RAPTOR-based retrieval‑augmented generation (RAG), and a Screening approach that extracts candidate medical sentences. Results show that full transcripts yield the best classification accuracy, while all compressed inputs increase false negatives, with summaries causing the largest performance drop and Screening performing best among compressed methods yet still missing many relevant sentences. Linguistic analysis indicates that compression reduces affective, social, temporal, cognitive, and conversational cues, and increases the prominence of institutional and technical terms, thereby making fake videos appear more coherent and authoritative.

Related stories

arXiv Computation and Language
Sep 24

Lost in Speech: Trilingual Spoken Hallucination Detection Across Audio and Transcripts

The paper introduces a trilingual spoken hallucination detection benchmark covering English, Russian, and Kazakh news, with 12,013 samples that include synthetic alterations and severity levels, as well as 290 fact‑checked misinformation items. Detectors are evaluated in a reference‑free setting on text, ASR transcripts, and audio, revealing that most models underperform baseline classifiers, except Gemma‑3n on transcripts. Synthetic‑trained detectors achieve high macro‑F1 scores on real‑world misinformation, but Russian provenance analysis highlights model‑dependent signals that confound synthetic benchmarks.

By Meruyert Aristombayeva, Jason S. Lucas, Chaewan Chun, Dongwon Lee
fine-tuningmultimodalbenchmarkssafety
More like this →
arXiv Computation and Language
Sep 18

FakeSpotter: A content and strategy agnostic Viral Misinformation Detection Tool

FakeSpotter is a new tool that estimates the viral misinformation risk of textual content by measuring structural fingerprints of misinformation instead of directly judging truthfulness. It operates across linguistic, narrative, logical, and critical‑thinking dimensions, using repeated large language model assessments and domain‑specific logistic regression classifiers for both short and long texts. In a labeled corpus of 764 texts, FakeSpotter achieved macro F1 scores of 0.788 for short texts and 0.793 for long texts, and its interpretive layer offers explainable outputs such as feature‑based scores, signal agreement, and a caution index for social listening.

By Giovanni Spitale, Federico Germani
llms
More like this →
arXiv Computation and Language
Sep 11

Whisper-Based Speech Transcription from Videos Across Multiple Languages for Cross-Cultural Understanding

The paper introduces methods to improve speech recognition for multilingual video transcription using Whisper-based tools, targeting cross‑cultural understanding. It reports an average transcription error rate of 30% across seven languages, which can be lowered to 20% with modest fine‑tuning. The authors also release associated speech and metadata to aid community refinement of these techniques.

By Michael Picheny
llmsfine-tuningmultimodal
More like this →
arXiv AI
Jul 21

Sparse Evidence Can Suffice: Agentic Evidence Seeking for Multimodal Video Misinformation Detection

arXiv:2607. 18080v1 Announce Type: cross Abstract: Multimodal video misinformation detection is commonly formulated as a holistic video-understanding task, where the entire video and its associated content are processed and judged in a single pass.

By Haochen Zhao, Yongxiu Xu, Xinkui Lin, Dong Xie, Jiarui Lu, Yuqi Qian, Yubin Wang, Hongbo Xu, Gaopeng Gou
agentsreinforcement-learningmultimodalbenchmarks
More like this →
arXiv AI
Sep 21

VidOmni-Bench: A Benchmark for Fine-Grained Video Understanding via Spatio-Temporal Event Verification across Complexity and Duration

VidOmni-Bench is a new benchmark for fine‑grained video understanding that asks models to verify whether each event in dense video captions is supported by the video. It contains 500 videos covering five complexity types and durations from 4 seconds to 90 minutes, and uses human‑verified sentence‑level labels to create hard negatives. Experiments show that Video‑LLMs often hallucinate events, struggle to detect incorrect descriptions, and exhibit varying weaknesses depending on video complexity and duration.

By Changbeen Kim, Junwon Chang, Kipyo Kim, Risa Shinoda, Kuniaki Saito, Donghyun Kim
llmsnlpbenchmarks
More like this →
arXiv AI
Sep 25

To Trust or Not to Trust: Retrieval-Augmented Fact Checking in Speech

The paper introduces VeriSpeak, a benchmark of 3,879 spoken claims for evaluating fact verification in Large Audio Language Models (LALMs). It shows a clear modality gap: models that verify written claims well often fail on spoken versions, and retrieval alone offers limited improvement. Combining retrieval with explicit reasoning yields the best performance, reaching 86.1% accuracy and demonstrating the need for grounded reasoning over retrieved evidence in speech misinformation detection.

By Debajyoti Mazumder, Mamta, Abhirama Subramanyam Penamakuri
llmsragbenchmarks
More like this →