← Back to all news
arXiv Machine Learning

LLM4Trust: Exploring the Capabilities of Large Language Models for Trust Evaluation

The paper introduces LLM4Trust, a benchmark framework that evaluates large language models (LLMs) for trust assessment in cybersecurity. It constructs trust graphs to represent five fundamental trust properties, designs property‑understanding tasks, and tests eight LLMs across nine prompting strategies. The study identifies effective LLM‑prompt combinations, applies them to real datasets, proposes methods to handle context limits, and demonstrates that LLMs can understand trust properties while highlighting vulnerabilities and high inference costs, for which defense mechanisms and batch inference are suggested.

Related stories

arXiv AI
5d ago

LTBD: Learnable Trust-Boundary Delimiters for Prompt Injection Defense

The paper introduces Learnable Trust‑Boundary Delimiters (LTBD), a lightweight defense that embeds learnable delimiters into LLM inputs to explicitly mark trusted user instructions versus untrusted external data. LTBD keeps the model’s parameters unchanged, achieving superior performance over inference‑time defenses and matching training‑based methods while adding negligible inference overhead. Experiments show near‑zero attack success rates on AlpacaFarm and TaskTracker, and the method remains robust even against adaptive attackers who know the defense.

By Luman Zhao, Minghui Xu, Yue Zhang, Yijun Yang
llmsfine-tuningmultimodal
More like this →
arXiv AI
Jun 29

DMind Benchmark: Toward a Holistic Assessment of LLM Capabilities across the Web3 Domain

arXiv:2504. 16116v4 Announce Type: replace-cross Abstract: The Web3 ecosystem, underpinned by cryptographic primitives and decentralized consensus, represents a high-stakes environment where software vulnerabilities and incentive misalignments translate directly into financial loss.

By Enhao Huang, Pengyu Sun, Shuxun Wang, Zixin Lin, Alex Chen, Kaichun Hu, Joey Ouyang, Frank Li, Zhiyu Zhang, Haobo Wang, Yiming Li, Zhan Qin, James Yi, Gang Zhao, Ziang Ling, Lowes Yang
llmsbenchmarkssafety
More like this →
arXiv AI
Aug 24

Trustworthy RAG: An Evaluation Agent for Detecting Misinformation and Knowledge Poisoning in Generative AI Systems

The paper introduces Trustworthy RAG, an evaluation agent designed to detect misinformation and knowledge poisoning in Retrieval-Augmented Generation systems. It combines natural language inference verification, a five-signal poison detector, and a weighted Trust Index to assess the reliability of retrieved content. Experiments on multiple LLMs show high accuracy and precision, with the agent effectively blocking unsafe advice in a secure-coding assistant scenario.

By Balkrishna Giri, Md Toufique Hasan, Jussi Rasku, Muhammad Waseem, Pekka Abrahamsson
llmsragagents
More like this →
Hugging Face Trending Papers
Aug 12

Benchmarking Trustworthiness of SLMs: Pre-trained vs. Compressed

Small Language Models (SLMs) have emerged as a more efficient alternative to traditional Large Language Models (LLMs), offering promising potential in resource-constrained scenarios. Existing approaches to building SLMs typically follow two paths: training compact models from scratch, or compressing larger pre-trained models using methods such as pruning, quantization, or distillation.

llmsefficiencybenchmarkssafety
More like this →
arXiv Machine Learning
Aug 27

Are LLM-Enhanced GNNs Privacy-Safe?

The paper evaluates privacy risks in graph neural networks enhanced by large language models (LLMs). Using a five‑stage framework, the authors test six real‑world text‑attributed graph datasets with 42 model configurations and six privacy attack methods across link, label, and membership inference threats. Results show that LLM‑enhanced GNNs are more vulnerable than shallow baselines, with semantic enrichment amplifying exploitable signals, and that differential privacy can reduce risk but at a significant cost to utility.

By Longzhu He, Zelang Wen, Chaozhuo Li, Sen Su
llmsragsafety
More like this →