arXiv AI
Jul 7
K9-Bench: Evaluating Multimodal LLMs on Canine-Centric Videos
arXiv:2607. 02680v1 Announce Type: cross Abstract: MLLMs have shown strong zero-shot capabilities across diverse inputs such as across images, video, audio, and text.
InterPet4D: A Multimodal 4D Human-Pet Interaction Dataset for Pet Motion Generation
Human-pet interaction estimation and generation remain underexplored due to the absence of a high-quality large-scale dataset. We present InterPet4D, the first multimodal dataset capturing natural interactions between humans and dogs.
Related stories
arXiv Computer Vision
Sep 3
Kirin: Animal Motion Generation from In-the-Wild Video
Kirin is a new framework that reconstructs 3D animal motion from in‑the‑wild videos, learns motion priors at scale, and generates realistic motion conditioned on text and image. It introduces AiM3D, the first large‑scale dataset of aligned video‑text‑motion tuples for quadruped animals, and uses an off‑the‑shelf image‑to‑3D model to automatically rig and animate 3D meshes with the generated motion. The framework and dataset provide a foundation for large‑scale, text and image‑conditioned animal motion generation and animation.
arXiv Computer Vision
Sep 14
UniMo: Unifying Human and Animal Motion Generation
UniMo introduces a unified point‑cloud based framework for generating 3D motion that works for both humans and animals, overcoming challenges posed by diverse skeletal topologies and limited animal datasets. It converts parametric skeletons into unparametric representations and uses dynamic sampling to focus on active joints. The authors also release UniML3D, a large motion‑language dataset with 145,907 sequences and 433,388 captions, and demonstrate state‑of‑the‑art performance on multiple benchmarks.
Hugging Face Trending Papers
Sep 28
VastMAT: A Large-Scale Multi-Category Benchmark for Multi-Animal Tracking
VastMAT is a large‑scale multi‑animal tracking benchmark featuring 2,947 videos, 337 animal categories, and over 3.6 million bounding boxes with 22,883 identity trajectories. It emphasizes high‑quality, expert‑reviewed annotations and introduces Seen‑category and Unseen‑category evaluation protocols, revealing significant challenges in tracking unseen animals. The authors also propose a lightweight Center‑Distance‑Augmented Association module that boosts HOTA scores for existing MOT methods without extra training.
arXiv Computer Vision
Aug 21
Inter-X++: A Comprehensive Benchmark for Multimodal Human-Human Interaction Analysis
arXiv:2608. 20312v1 Announce Type: new Abstract: The capability to perceive and synthesize human-human interactions is fundamental to developing intelligent digital human systems.
arXiv AI
Aug 18
HiPHI: A Large-Scale Benchmark for High-Precision Human Motion and Object-Interaction
arXiv:2608. 16222v1 Announce Type: cross Abstract: Humanoid intelligence requires learning over an extremely diverse space of whole-body motions and physically grounded interactions.
arXiv AI
Oct 9
SAM 3D Animal: Promptable Animal 3D Reconstruction from Images in the Wild
SAM 3D Animal is a promptable framework that performs multi‑animal 3D reconstruction from a single image, using the SMAL+ parametric animal model. It jointly reconstructs multiple instances and allows flexible prompts such as keypoints and masks to better handle crowded and occluded scenes. The authors also introduce Herd3D, a new dataset of over 5,000 images that increases species, interaction, and occlusion diversity, and demonstrate state‑of‑the‑art performance on several benchmark datasets.
arXiv AI
Sep 30
WeLike2Party! In-Context Motion Transfer for Multi-Human Image Animation
arXiv:2609.36937v1 Announce Type: cross Abstract: Human image animation aims to transfer motion from a driving video to subjects in a reference image. Despite remarkable progress in video generation,...
arXiv Machine Learning
Jul 28
MIME: Multimodal Interactive Motion Encoder
arXiv:2607. 22702v1 Announce Type: cross Abstract: Text-motion representation learning has advanced rapidly, with growing interest in multi person interactions for animation, AR/VR, and embodied AI.
Hugging Face Trending Papers
Aug 5
Promptable Animal Pose Tracking Across Species
Animal pose estimation and tracking is important for wildlife monitoring and conservation research, and with limited expert time for labelling automated approaches are imperative. While human pose estimation and tracking has seen rapid progress thanks to large annotated datasets, animal pose remain challenging, due to large morphological and behavioural differences between species and limited annotated data.
arXiv Computer Vision
Sep 25
Ego-Exo4D Human Meshes Dataset: 4D Human Motion Reconstruction for Ego-Exo Captures
Ego-Exo4D is a large-scale dataset that offers synchronized egocentric and multi-view exocentric video for applications such as skill learning, procedural activity understanding, and embodied AI. The original dataset only includes sparse 3D human pose annotations, making dense motion reconstruction challenging. To address this, the authors introduce Ego-Exo4D-HM, a new dataset containing 4D human motion reconstructions for the Ego-Exo4D captures, along with a reconstruction pipeline and accompanying code and documentation.
arXiv Computer Vision
Sep 14
Uni-HOI:A Unified framework for Learning the Joint distribution of Text and Human-Object Interaction
Uni-HOI is a unified framework that learns the joint distribution among text, human motion, and object motion for 4D human‑object interaction (HOI). It uses large language models and two motion‑specific VQ‑VAEs to convert heterogeneous motion data into token sequences, enabling seamless integration of all three modalities. A two‑stage training strategy first captures correlations on a large‑scale HOI dataset and then fine‑tunes for specific tasks, achieving strong performance on text‑driven HOI generation, object‑motion‑driven human motion generation, and human‑motion‑driven object motion prediction.