Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.
Cross-view object geo-localization (CVOGL) locates a target in satellite imagery using drone or street-view queries. Existing methods train separate detectors for each viewpoint, leading to parameter...
The integration of large-scale foundation models with physical embodiments has led to significant advancements in robotics known as Generative Physical Artificial Intelligence (GPAI). These agentic AI...
The paper introduces PosterVisor, a framework that replaces transient prompts with persistent control for scientific poster generation. It uses an Orchestrator to compile rubrics into Semantic‑Geometric Contracts that bind claims, sources, visuals, budgets, and spatial commitments, and employs Recursive Contract Enforcement to dynamically trigger checks and prevent silent regressions. Implementations in HTML/CSS and PPTX show improved QA accuracy and higher human preference on benchmark datasets.
The paper presents a multimodal machine learning framework that classifies Emirati residential architectural styles by combining visual features from images and textual descriptions using OpenAI's CLIP model. The unified 512‑dimensional embeddings are reduced with UMAP, clustered with K‑Means, and then used to train an SVM classifier, achieving a 98% accuracy across eight style clusters. This approach outperforms previous studies and demonstrates the value of integrating visual and textual data for cultural heritage analysis.
SPEAR NeXT is a compact, pixel‑wise multimodal spectral‑temporal foundation model that learns temporal self‑supervision by predicting future latent Earth states from past observations. It encodes instantaneous states from optical, radar, and environmental data into 32‑dimensional embeddings, then models their evolution with a causally masked transformer that forecasts multiple future horizons. The model uses Rotary Position Embeddings to capture relative temporal order and month/year embeddings to encode seasonal and interannual context.
EMODY Flow is a lightweight flow‑matching framework that generates synchronized full‑body motion and facial expressions conditioned on speech and emotion. It attaches to a frozen Qwen‑3 Omni model, reusing its audio codecs to drive two parallel DiT generators for SMPL‑X body pose and FLAME facial expressions. An auxiliary emotion classifier at training time restores emotion sensitivity, enabling EMODY Flow to achieve state‑of‑the‑art gesture quality on BEAT2 and zero‑shot facial animation on TFHP, with significant improvements in FGD, Beat Correlation, and Diversity metrics.
SyncVoice is a new automatic video dubbing framework that adds a lightweight Text‑Visual Fusion Module to a pretrained TTS system, aligning visual features with linguistic representations to produce temporally synchronized speech. The approach avoids complex architectural changes and achieves state‑of‑the‑art performance on the LRS3 dataset in zero‑shot dubbing. When further trained on a large bilingual audio‑visual corpus, SyncVoice improves vocal fidelity while maintaining synchronization, enabling a single model to dub both Chinese and English videos.
The paper investigates intersectional biases in multimodal clinical predictions using Electronic Healthcare Records (EHR). It introduces datasets MIMIC-Eye1 and MIMIC-IV ED, applies unified text representations from pre‑trained clinical language models, and benchmarks bias mitigation at the intersectional subgroup level. Results show that subgroup‑specific mitigation is robust across datasets, subgroups, and embeddings, effectively addressing intersectional biases in multimodal settings.
FLAT (Flexible‑Length Aligned Transmodal representations) is a joint multimodal pre‑training framework that learns a shared encoder for images and text, producing 1‑D continuous embeddings that can be directly used by downstream generative decoders. By combining contrastive alignment with bidirectional cross‑modal generative objectives, FLAT yields representations that are both discriminative and generative, enabling cross‑modal retrieval and generation with a single pre‑training stage. The model achieves strong performance on T2I generation (GenEval 71.1), image captioning (BLEU‑4 40.5, CIDEr 138.6), and retrieval tasks (Recall@5 86.8/75.8 on MS‑COCO, 98.3/93.6 on Flickr30K), and supports linear interpolation, latent space arithmetic, and zero‑shot composed retrieval.
TecoPrompt introduces a closed‑loop robust prompt‑learning framework that mitigates label noise in vision‑language models. It uses optimal transport in the CLIP semantic space to generate globally consistent pseudo‑labels, then verifies their reliability by checking trajectory stability over a K‑epoch window and an EMA‑based confidence gate. The verified labels are incorporated into prompt training via a tri‑group objective, yielding significant accuracy gains across multiple noisy datasets, such as a 0.843 accuracy on OxfordPets with 50% asymmetric noise.
TARC (Time‑Adaptive Robotic Control) is a reinforcement‑learning framework that lets a policy predict both a control action and how long it should be applied, thereby learning temporally extended actions. By optimizing task performance under constraints on the number of control switches, TARC can adapt its control rate online, using high‑frequency feedback only when necessary. Experiments on a high‑speed RC car, a Unitree Go1 quadruped, and a vision‑language action model show that TARC matches the performance of high‑frequency discrete‑time controllers while operating at less than half their control frequency.
The paper introduces PolyLatentFlow, a continuous‑time flow‑matching framework for polymer generation, and LlamaUni, a multimodal representation that fuses polymer sequences with 3D structural data. In unconditional generation, the combination yields the highest number of valid, novel candidates while preserving diversity, and in conditional settings it systematically shifts property distributions across a 200 °C target range. Across multi‑property tasks, the representation choice affects validity, training‑set replay, and structural proximity, with PolyLatentFlow + LlamaUni achieving the best balance of high validity, low replay, and high target hit yield.
The paper introduces Relation‑Orbit, a symmetry‑aware method for aggregating likelihoods from frozen vision‑language models to verify fine‑grained left‑right claims. It uses a closed‑form contrast that assigns eight normalized likelihoods based on reflection, inverse relation, and entity exchange, and asserts a claim only when the signed contrast exceeds a threshold chosen via Clopper‑Pearson bounds. Experiments on VSR, GQA, and LLaVA‑1.5/COCO show that Relation‑Orbit achieves higher mean test coverage at a 10% selective‑risk calibration target compared to an all‑eight Orbit‑Max baseline across multiple dataset‑backbone settings.
The survey "World-Action Models for Robot Learning and Control" reviews recent advances in coupling future world prediction with executable action generation for robots in open environments. It clarifies the scope of World-Action Models (WAMs) relative to conventional world models, model-based RL, and Vision‑Language‑Action policies, and organizes existing methods through a unified taxonomy covering representations, transition modeling, action interfaces, architectures, training pipelines, data modalities, and scaling strategies. The paper also surveys applications in manipulation, navigation, and autonomous driving, summarizes datasets, benchmarks, and metrics, and discusses key challenges such as action alignment, spatial consistency, long‑horizon memory, and efficient inference.
DGSG-Mind introduces a hybrid instance-aware 3D Gaussian dynamic scene graph system that integrates open‑vocabulary semantic information into dynamic 3D scene representations. By coupling a probabilistic voxel grid with explicit 3D Gaussians, it achieves robust cross‑modal instance fusion, incremental semantic mapping, and dynamic change handling through Gaussian‑based relocalization and masked refinement. The system builds a hierarchical scene graph and a 3D Gaussian Mind for multimodal reasoning, achieving state‑of‑the‑art zero‑shot 3D visual grounding and strong performance in open‑vocabulary semantic segmentation and scene reconstruction, and is demonstrated on real‑world robots.
OmniHarness is a framework that enables generalizable visual generation by learning symbolic policies from verified executions. It abstracts shared procedures and applicability conditions, allowing these policies to be instantiated, adapted, and composed for new tasks while keeping model parameters fixed. The system uses intermediate verification for refinement, self-directed inquiry to generate practice tasks, and continuous feedback to expand capabilities, achieving strong results on multiple benchmarks and outperforming baselines on Creative tasks.
MUUNRiver-Bench is a diagnostic benchmark for music retrieval that uses natural‑language instructions to define relevance for reference‑audio queries. It contains 3,440 tracks across 13 genres and 116 sub‑genres and covers seven tasks such as similar‑music, style‑preserving lyric‑rewriting, cover, and segment retrieval. Experiments with six models in eight configurations show that acoustic encoders favor local identity while text‑aligned encoders favor semantic relations, and that instruction‑aware and audio‑text fusion systems do not consistently outperform their backbones.
The paper explores whether CLIP embeddings can detect AI-generated images by using a frozen CLIP model to extract visual embeddings and training lightweight classifiers on top. On the CIFAKE benchmark, the approach achieves 95% accuracy without language reasoning, and 85% accuracy after few-shot adaptation with 20% of the data. Certain image types, such as wide-angle photographs and oil paintings, remain challenging, highlighting unexplored difficulties in AI-generated image classification.
arXiv:2609.17509v1 Announce Type: cross Abstract: Neural audio codecs are a key component in speech language modeling. However, their high frame rates lead to long sequence lengths, increasing comput...
arXiv:2609.16347v1 Announce Type: new Abstract: Sea-ice type prediction is important for climate monitoring, maritime navigation, and decision-making in polar regions. The main source of label data f...