Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.
arXiv:2601.21670v4 Announce Type: replace-cross Abstract: Multimodal fusion is often treated as an optimization-balancing problem, where training signals are adjusted to prevent one modality from dom...
arXiv:2608.29278v1 Announce Type: new Abstract: Omni-modal large language models are increasingly evaluated on clean text--vision--audio inputs, where every channel is present, synchronized, and read...
arXiv:2608.29897v1 Announce Type: new Abstract: Long-horizon agents need a context manager to compress growing interaction histories into a bounded working context, via passive strategies or active s...
arXiv:2608.30712v1 Announce Type: new Abstract: Large multimodal models follow instructions about what to generate, but not necessarily about what evidence to rely on. Hence, models may continue to d...
arXiv:2608.31035v1 Announce Type: new Abstract: Codec-based text-to-speech (TTS) models make language-model post-training applicable to speech generation, but it remains unclear when learned perceptu...
arXiv:2606.01914v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) remain unreliable on spatial multiple-choice questions, and their failures are often attributed to poorly...
arXiv:2606.15872v2 Announce Type: replace Abstract: Frontier scientific reasoning remains a major challenge for large language models (LLMs), where even the strongest commercial systems fall short of...
arXiv:2601.03549v3 Announce Type: replace-cross Abstract: Sign Language Translation (SLT) is a challenging cross-modal task requiring joint modeling of manual articulations and non-manual signals. Ex...
arXiv:2608.29121v1 Announce Type: new Abstract: Open-Vocabulary Audio-Visual Semantic Segmentation (OV-AVSS) aims to perform pixel-level segmentation of sound-emitting objects from an open set of cat...
arXiv:2608.29268v1 Announce Type: new Abstract: PCB engineering drawings mix sparse graphics, dense tables, and text whose meaning depends on page position. Localizing the regions and sending crops t...
arXiv:2608.29663v1 Announce Type: new Abstract: Remote photoplethysmography (rPPG) enables contactless physiological measurement from facial videos, yet its subtle pulse-related variations are easily...
arXiv:2608.30247v1 Announce Type: new Abstract: Recent unified open-vocabulary detection (OVD) supports heterogeneous prompts, including text queries, visual exemplars, and their combinations, but of...
arXiv:2608.30657v1 Announce Type: new Abstract: Fixed-viewpoint infrastructure sensors repeatedly observe the same traffic space, making roadside 3D occupancy structurally different from ego-vehicle...
arXiv:2608.30824v1 Announce Type: new Abstract: Combining whole-body magnetic resonance imaging (WB-MRI) with clinical variables has the potential to improve systemic disease diagnosis by leveraging...
This study evaluates machine learning and deep‑learning models for classifying balanced versus imbalanced postural states in immersive virtual reality using a multimodal dataset of kinematic, EMG, and EDA signals. The Mamba‑inspired CNN (MI‑CNN) achieved the highest accuracy (96.76%) and, through SHapley Additive exPlanations (SHAP), identified kinematic features as the most influential for detecting imbalance. Even after reducing input dimensionality by 33% based on SHAP importance, the model maintained near‑optimal performance (0.957 accuracy and F1‑score).
WeAgent-MMSearch introduces a multimodal search agent that preserves retrieved images as persistent references, enabling the model to inspect, process, and cite them throughout a search trajectory. The system includes a harness (WeAgent-Harness), a post‑training method (FA‑GSPO) that recovers salvageable rollouts, and a new benchmark (VisTarget‑Bench) to evaluate image‑retrieval versus visual‑perception failures. Evaluation shows that agentic post‑training boosts performance by 19.22 points, allowing the model to outperform similarly sized open‑source models and compete with much larger ones.
RLG-TPV introduces a multimodal Tri-Perspective View framework that fuses camera, radar, and training‑time LiDAR data for 3D object detection. It uses radar and LiDAR to guide a ray‑deformable attention lift, refining depth distributions and providing geometric supervision for side and front planes, while radar cross‑section awareness spreads evidence spatially. On nuScenes, the method attains 0.4981 mAP and 0.5959 NDS, improving orientation and velocity accuracy by about 32 % and 31 % over the CRN baseline.
The paper introduces NavMCP, a scaffolding framework that couples vision‑language models (VLMs) with navigation foundation models (NFMs) to enable long‑horizon physical‑world agents. NavMCP orchestrates three communication channels—intent, observation, and memory—to allow the VLM to decide what evidence to seek and the NFM to ground semantic sub‑goals into closed‑loop navigation, without retraining either model. The approach achieves state‑of‑the‑art results on several embodied question‑answering benchmarks and significantly outperforms episodic interfaces on the Unitree Go2 robot as task horizons lengthen.
GarmentWeaver is a new framework for multimodal sewing pattern generation that uses a schema‑aware approach to construct compact hierarchical targets. By activating garment‑relevant structural branches and building on a pretrained vision‑language model, it predicts executable sewing patterns in a structured manner. Experiments show that GarmentWeaver produces more accurate, executable patterns and yields better simulation results than strong baselines.
arXiv:2608.29374v1 Announce Type: new Abstract: Tool-augmented multimodal reasoning integrates external tools (e.g., object detection, depth estimation) into multimodal large language models (MLLMs)...