VoiceDesigner: Text-to-Voice Generation and Editing via Unified Diffusion Modeling and Data Augmentation
arXiv, 2026
A unified diffusion framework for diverse, controllable text-to-voice generation and voice editing.
Jiarui Hai
Making machines understand and generate sound
Education
Johns Hopkins University
Tsinghua University
Tsinghua University
Experience
Apple
Adobe Research
Tencent AI Lab
Kuaishou AI Platform
About Me
I am a final-year Ph.D. student at Johns Hopkins University, fortunate to be advised by Prof. Mounya Elhilali.
My Research focuses on AI for understanding and generating sound. My work spans topics such as controllable generation, voice design, source separation, and detection. I enjoy bringing research to life through interactive demos, open-source tools, and datasets for the broader audio and machine learning community.
Outside of academia, I make Music as a producer, mostly hip-hop and hyperpop. This experience shapes how I approach audio research, giving me a creator's perspective on what makes audio tools expressive, controllable, and genuinely useful in the creative process.
At Tsinghua, I studied civil engineering and information management, taking courses such as fluid mechanics and econometrics while exploring data-driven research in business analytics and medical informatics. This quantitative foundation later came together with my long-standing interest in music, leading me to audio research.
Seeking Full-Time Positions
I am open to full-time Research Scientist or
Research Engineer positions, with a focus on
audio, speech, or music technology, starting in early 2027.
Please feel free to contact me at jhai2@jhu.edu if there is a potential fit.
News
Oct 2025
EzAudio was accepted as an oral presentation at Interspeech.
Jun 2025
We built OpenSound to share speech and audio models with live demos.
We built OpenSound to share speech and audio models with live demos.
Sep 2024
Sep 2024
Gave an oral presentation of my first first-author paper, Diff-Pitcher, at WASPAA.
Oct 2023
Joined LCAP @ Johns Hopkins University as a PhD student.
Aug 2022
* Equal contribution
† Research mentorship
† Research mentorship
Research Highlights
CapSpeech: Enabling Downstream Applications in Style-Captioned Text-to-Speech
TASLP, 2026
[TL;DR] [Paper] [Homepage] [Live Demo]
A style-captioned TTS dataset and framework enabling controllable, style-aware text-to-speech for downstream applications.
FlexSED: Towards Open-Vocabulary Sound Event Detection
WASPAA, 2025 Spotlight
[TL;DR] [Paper] [Code] [Live Demo]
An open-vocabulary sound event detection approach that generalizes to unseen classes via flexible text-conditioned modeling.
EzAudio: Enhancing Text-to-Audio Generation with Efficient Diffusion Transformer
Interspeech, 2025 Oral
[TL;DR] [Paper] [Homepage] [Live Demo]
An efficient diffusion transformer that improves text-to-audio generation quality while reducing compute.
More Projects
Education
Johns Hopkins University
Tsinghua University
Tsinghua University
Experience
Apple
Adobe Research
Tencent AI Lab
Kuaishou AI Platform