arXiv AI

CSI-TextBench: A Dataset and Benchmark for Language-Grounded Ambient Sensing Perception

CSI-TextBench is a large‑scale dataset and benchmark that aligns WiFi Channel State Information (CSI) with natural language, combining 285,790 samples from three sensing benchmarks into 2.0 million CSI‑text training tuples. It pairs these samples with 2,291 hierarchically structured, label‑derived descriptions across seven tasks, enabling many‑to‑many alignment. Benchmarking five alignment methods shows that contrastive CSI‑text models can match supervised classifiers, generalize to unseen classes with few samples, and answer compositional queries, achieving 86.9% top‑1 accuracy on 140 composite scenes.

arXiv AI
Jul 14

The Universal Language of CSI:Unifying Wireless Sensing Across Devices and Environments

arXiv:2607. 09727v1 Announce Type: cross Abstract: WiFi sensing based on Channel State Information (CSI) promises ubiquitous, device-free perception, yet current research remains trapped in a Tower of Babel - fragmented into isolated silos where models are tailored to specific hardware dialects, fixed environments, and narrow tasks.

By Jiayi Chen, Weiting Ou, Guangxu Zhu
arXiv Machine Learning
Oct 28

AmbientFM: A Foundation Model for Ambient Sensing

AmbientFM is a foundation model designed for ambient sensing using ubiquitous WiFi signals. It is pre‑trained on 9.2 million unlabeled Channel State Information samples collected over 439 days from 20 commercial devices in real homes, making it the largest real‑world WiFi CSI pre‑training corpus reported to date. The model learns transferable wireless representations through contrastive learning, masked reconstruction, and physics‑informed objectives, and with lightweight adaptation it can transfer to nine downstream tasks, achieving above 0.90 AUROC on all classification benchmarks and supporting dense spatial reconstruction.

By Guozhen Zhu, Yuqian Hu, Sakila Jayaweera, Weihang Gao, Wei-Hsiang Wang, Jiaxuan Zhang, Beibei Wang, Chenshu Wu, K. J. Ray Liu
Hugging Face Trending Papers
Jun 9

Closing the Modality Gap in Zero-Shot HAR: Contrastive Training and Separability-Optimized Prototypes on IMU Data

Zero-shot learning (ZSL) for inertial measurement unit (IMU)-based human activity recognition (HAR) faces a central challenge: bridging the gap between sensor embeddings and semantic class representations. We systematically evaluate seven configurations combining three inference methods with two training pipelines on the PAMAP2 dataset, using 14 seen and 4 unseen activity classes with subjects 108 and 109 held out for testing.

arXiv Machine Learning
Aug 28

HALO: A Heterogeneity-Aware Language-Aligned IMU Foundation Model for Open-Set Human Activity Recognition

HALO is a heterogeneity‑aware, language‑aligned foundation model for inertial measurement unit (IMU) based human activity recognition. It uses a two‑stage training process: first, a self‑supervised encoder learns to handle diverse sensor configurations and natural‑language sensor descriptions; second, the encoder is aligned with text embeddings through synonym‑aware contrastive learning, enabling open‑set recognition via cosine similarity. Trained on ten public HAR datasets, HALO outperforms five state‑of‑the‑art baselines across eight metrics while using only ~35 M parameters, and improves zero‑shot open‑set accuracy by 13.7 percentage points over 87 training labels.

By Zihan Ding, Liyu Zhang, Xiaomin Ouyang
arXiv Machine Learning
Jul 20

More with Less: a Large Scale Remote Sensing VLM with a Simple Recipe

arXiv:2607. 15942v1 Announce Type: cross Abstract: Remote sensing vision-language models are increasingly expected to support open-ended reasoning over Earth Observation data and a variety of tasks.

By Stefan Maria Ailuro (INSAIT, Sofia University "St. Kliment Ohridski"), Mario Markov (INSAIT, Sofia University "St. Kliment Ohridski"), Mohammad Mahdi (INSAIT, Sofia University "St. Kliment Ohridski"), Luc Van Gool (INSAIT, Sofia University "St. Kliment Ohridski"), Danda Pani Paudel (INSAIT, Sofia University "St. Kliment Ohridski")
arXiv Computer Vision
Sep 22

DeCo: Efficient Decouple-to-Couple Learning for Multi-Task Visual Grounding

DeCo introduces an efficient decouple-to-couple learning framework for multi-task visual grounding, addressing conflicts between localization and segmentation tasks. It first applies Task-aware Semantic Decoupling (TSD) to separate shared visual cues into task-specific features guided by salient words, then uses Hybrid Prior Coupling (HPC) to merge sentence-level semantic priors with mask-derived spatial priors for improved grounding. Experiments across multiple natural and remote sensing datasets show that DeCo achieves state‑of‑the‑art performance while requiring only lightweight trainable parameters on a frozen multimodal encoder.

By Xiaoqiang Lu, Licheng Jiao, Long Sun, Yuting Yang, Xu Liu, Lingling Li, Wenping Ma, Fang Liu
arXiv Computer Vision
Oct 16

Grounded Human-Attributed Description and Activity Recognition in Videos

The paper introduces Grounded Human-Attributed Description and Activity Recognition (GHADAR), a task that requires describing the characteristics and activities of every person in a video given their locations. It presents a new dataset derived from AVA-Actions with open‑set captions, a method that constrains cross‑attention masks in vision‑language models to better use grounding information, and a holistic VLM‑based evaluation schema that compares concepts directly between video and predictions. Experiments show the proposed method surpasses state‑of‑the‑art VLMs and highlight limitations of existing evaluation metrics.

By Jay N. Paranjape, Jurijs Nazarovs, Rohan Sarkar, Yue Guo, Lipeng Ke, Sankar Venkataraman, Vishal M. Patel, Nataraj Jammalamadaka
arXiv AI
Jul 7

STELLA: Efficient Sensor-to-LLM Translation for On-Device Human Activity Recognition

arXiv:2607. 03089v1 Announce Type: cross Abstract: HAR is increasingly expected to run continuously on edge devices, yet recent LLM-based methods remain hard to deploy: raw sensor prompts are long, cloud inference adds latency and privacy risk, and fine-tuned LLM pipelines turn general-purpose models into task-specific classifiers.

By Nirhoshan Sivaroopan, Albert Zomaya, Kanchana Thilakarathna