arXiv AI
Jul 29
MusiChat: Vibe Composing for Music Creation
arXiv:2607. 24873v1 Announce Type: new Abstract: Recent advances in AI music generation have enabled users to create complete musical pieces from natural-language prompts.
MusicTwins: Training Audio-Language Models to Listen like Their Persona
MusicTwins is a framework that trains audio‑language models to emulate individual personas by combining persona summaries, personal memory, and post‑training. It builds persona representations from human writings that capture background, views, personality, writing style, and a musical profile, and then fine‑tunes Audio Flamingo Next using supervised learning and reinforcement learning. Evaluation on ten held‑out personas shows significant gains in persona fidelity, including personality, writing style, and music‑related responses.
Related stories
Hugging Face Trending Papers
Jul 27
MusiChat: Vibe Composing for Music Creation
Recent advances in AI music generation have enabled users to create complete musical pieces from natural-language prompts. However, most existing systems follow a prompt-and-regenerate paradigm, making iterative refinement difficult because users must repeatedly recreate compositions instead of directly evolving existing musical ideas.
arXiv AI
Jun 3
TalkPlayData 2: An Agentic Synthetic Data Pipeline for Multimodal Conversational Music Recommendation
arXiv:2509. 09685v5 Announce Type: replace-cross Abstract: We present TalkPlayData 2, a synthetic dataset for multimodal conversational music recommendation generated by an agentic data pipeline.
arXiv AI
Jun 12
CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction
arXiv:2603. 00610v3 Announce Type: replace-cross Abstract: While music generation models have evolved to handle complex multimodal inputs mixing text, lyrics, and reference audio, evaluation mechanisms have lagged behind.
arXiv AI
Jun 18
Continuous Audio Thinking for Large Audio Language Models
arXiv:2606. 18273v1 Announce Type: cross Abstract: Large audio language models (LALMs) have shown impressive capabilities on diverse audio understanding tasks, ranging from speech transcription to music analysis.
arXiv AI
Jun 30
LeVo 2: Stable and Melodious Song Generation via Hierarchical Representation Modeling and Progressive Post-Training
arXiv:2606. 30642v1 Announce Type: cross Abstract: Full-length song generation must preserve coherence and musicality, render detailed vocal and accompaniment acoustics, and follow lyrics and prompts.
arXiv AI
Oct 31
A Tutorial on Generative Models for Music Audio
A tutorial monograph on generative models for music audio covers a wide range of applications—from instrumental clips and full songs with vocals to edits of existing recordings and real‑time accompaniment systems. It explains neural audio codecs, autoencoders, autoregressive and masked token models, diffusion and flow matching, sampling acceleration, and preference optimization, linking training objectives to published systems through worked examples. The book also discusses musical requirements for different tasks, compares representations and methods, and addresses evaluation, data provenance, memorization, attribution, and legal implications.
arXiv Computation and Language
Sep 22
Pretrained Persona Mixture Models and Tandem Models for Human Simulation
arXiv:2609.22607v1 Announce Type: new Abstract: We argue here that the current dominant practice in LLM human simulation: prompting instruction-tuned assistant language models to role-play personas,...
arXiv AI
Sep 21
Do Personality-Tuned LLMs Make Better Social Agents?
The paper examines whether fine‑tuning large language models (LLMs) with personality‑labelled data improves their ability to act as socially interactive agents. Two small open‑weight LLMs were fine‑tuned on a corpus of personality‑labelled social media posts and dialogues, and the resulting models were evaluated in various social interaction scenarios by independent LLM judges. The findings show that the fine‑tuned models do not outperform their baseline counterparts in role‑playing personalities, though they offer comparable text quality and increased linguistic diversity for the Qwen models; low inter‑rater agreement limits confidence in the results, suggesting future work should focus on training data quality and domain alignment.
arXiv Machine Learning
Oct 5
Learning Jazz Pianist Style with Cross-Attention Conditioning
The paper demonstrates that a pretrained symbolic music transformer already encodes jazz pianist identity sufficiently for accurate classification across two benchmarks. By adding cross‑attention over learned pianist embeddings, the model can generate music conditioned on a specific artist’s style, and evaluation protocols confirm that the generated continuations are correctly attributed to the intended pianist. Additionally, the classifier is repurposed to identify the most characteristic moments in a performance, revealing the musical gestures that distinguish each pianist’s voice.
arXiv AI
Sep 21
MENASpeechBank: A Reference Voice Bank with Persona-Conditioned Multi-Turn Conversations for AudioLLMs
MENASpeechBank is a new reference voice bank that provides about 18,000 high‑quality utterances from 124 speakers across multiple MENA countries, covering English, Modern Standard Arabic, and regional Arabic varieties. The dataset is built through a controllable pipeline that creates persona profiles inspired by the World Values Survey, defines a taxonomy of roughly 5,000 conversational scenarios, matches personas to scenarios via semantic similarity, and generates around 417,000 role‑play conversations using an LLM. Synthetic speaker‑conditioned user‑turn audio is produced from reference recordings to maintain speaker diversity, and both synthetic and human‑recorded conversations are evaluated and analyzed for quality.
arXiv Computation and Language
Oct 23
Predicting Music Valence and Arousal via Textual and Multimodal In-Context Learning in Large Language Models
The paper explores using in‑context learning (ICL) with large language models to predict continuous music valence and arousal. By providing contextual demonstrations from an auxiliary dataset, the authors refine predictions from a small base model, testing both text‑only (lyrics and audio captions) and multimodal (raw audio) ICL variants. Experiments on the MER60 dataset show significant gains, especially for valence, with the text‑based ICL raising R² from 0.110 to 0.427 using 80 shots and the multimodal ICL achieving 0.346 with just 10 shots.