arXiv AI
BGM2Pose: Active 3D Human Pose Estimation with Non-Stationary Sounds
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 7
Sound-based Multi-Person 3D Pose Estimation
The paper introduces SoundMHPE, an encoder‑decoder framework that estimates 3D poses of multiple people using only acoustic signals. It addresses challenges such as overlapping acoustic signatures and inter‑person reflections by employing a multi‑scale acoustic encoder and a temporal pose decoder with attention. The authors created the 6‑hour Acoustic Multi‑person Pose (AMP) dataset and show that SoundMHPE outperforms baseline models.
Hugging Face Trending Papers
Sep 4
Sound-based Multi-Person 3D Pose Estimation
The paper introduces SoundMHPE, the first system to estimate multi‑person 3D poses using only acoustic signals. It tackles challenges such as overlapping acoustic signatures and inter‑person reflections by employing an Acoustic Multi‑scale Encoder and a Temporal Pose Decoder with attention. The authors built a 6‑hour Acoustic Multi‑person Pose dataset and show that SoundMHPE outperforms baseline models.
arXiv Computer Vision
4d ago
BeatDance: Generating Beat-Consistent 3D Dance with Hierarchical Spatial-Temporal Modeling
arXiv:2609.37400v1 Announce Type: new Abstract: Generating realistic 3D dance from music is a challenging task that requires accurate synchronization with musical rhythms while capturing the spatial...
arXiv AI
Jun 3
AVTrack: Audio-Visual Tracking in Human-centric Complex Scenes
arXiv:2606. 02724v1 Announce Type: cross Abstract: Audio-visual speaker tracking aims to localize and track active speakers by leveraging auditory and visual cues, enabling fine-grained, human-centric scene understanding.
arXiv AI
2d ago
Supervising Sound Localization by In-the-wild Egomotion
The paper introduces a method for learning binaural sound localization by using egomotion as a supervisory signal. By tracking how a camera’s direction changes relative to a sound source during a video, the authors train an audio model to predict sound directions that align with visual estimates of camera motion derived from multi‑view geometry. They evaluate this approach on a newly proposed dataset of real‑world audio‑visual videos with egomotion, demonstrating that the model can learn from real data and perform well on sound localization tasks.
arXiv AI
Sep 24
SsgCaps: A controlled dataset for the evaluation of sound scene generation algorithms
SsgCaps is a publicly available dataset of human-engineered sound scenes, each paired with a precisely structured prompt that guides the sampling process. The prompts are drawn from a predefined action-based typology, enabling extensive yet plausible sampling. A comparative quantitative analysis shows only small differences between the open and private versions, supporting the recommendation of the open version for benchmarking sound scene generation algorithms.