Open-source Resources
Service and Teaching
Conference Reviewer
ICASSP, Interspeech, ASRU, SLT, WASPAA
Journal Reviewer
TASLP, IJCV
Teaching Assistant
Audio Signal Processing
EN.520.445/645
Fall 2025
Audio Signal Processing
EN.520.445/645
Fall 2024
* Equal contribution,
† Research mentorship
Selected Publications
Domain
All
Audio
Voice
Music
Paradigm
All
Generation
Understanding
@misc{hai2026voicedesigner, title = {VoiceDesigner: Text-to-Voice Generation and Editing via Unified Diffusion Modeling and Data Augmentation}, author = {Hai, Jiarui and Thakkar, Karan and Chen, Ke and Wang, Yunyun and Su, Jiaqi and Kumar, Rithesh and Elhilali, Mounya and Jin, Zeyu}, year = {2026}, eprint = {2608.13613}, archivePrefix = {arXiv}, primaryClass = {eess.AS}, url = {https://arxiv.org/abs/2608.13613} }
VoiceDesigner: Text-to-Voice Generation and Editing via Unified Diffusion Modeling and Data Augmentation

Jiarui Hai, Karan Thakkar, Ke Chen, Yunyun Wang, Jiaqi Su, Rithesh Kumar, Mounya Elhilali, Zeyu Jin

arXiv, 2026

A unified diffusion framework for diverse, controllable text-to-voice generation and voice editing.

VoiceDesigner: Text-to-Voice Generation and Editing via Unified Diffusion Modeling and Data Augmentation

Jiarui Hai, Karan Thakkar, Ke Chen, Yunyun Wang, Jiaqi Su, Rithesh Kumar, Mounya Elhilali, Zeyu Jin

arXiv, 2026

A unified diffusion framework for diverse, controllable text-to-voice generation and voice editing.

@misc{yee2026voicetrace, title = {VoiceTrace: A Benchmark and Retrieval Framework for Who-Said-What Speech Retrieval}, author = {Yee, Aaron and Lu, Fengjie and Hai, Jiarui and Jiang, Chenang and Wang, Helin and Tu, Siwei and You, Weitao and Sun, Lingyun}, year = {2026}, eprint = {2609.18521}, archivePrefix = {arXiv}, primaryClass = {cs.SD}, url = {https://arxiv.org/abs/2609.18521} }
VoiceTrace: A Benchmark and Retrieval Framework for Who-Said-What Speech Retrieval
VoiceTrace: A Benchmark and Retrieval Framework for Who-Said-What Speech Retrieval

Aaron Yee, Fengjie Lu, Jiarui Hai†, Chenang Jiang, Helin Wang, Siwei Tu, Weitao You, Lingyun Sun

arXiv, 2026

A benchmark and two-stage retrieval framework that jointly retrieves spoken content by what was said and who said it.

VoiceTrace: A Benchmark and Retrieval Framework for Who-Said-What Speech Retrieval

Aaron Yee, Fengjie Lu, Jiarui Hai†, Chenang Jiang, Helin Wang, Siwei Tu, Weitao You, Lingyun Sun

arXiv, 2026

A benchmark and two-stage retrieval framework that jointly retrieves spoken content by what was said and who said it.

@misc{lu2026alm2veclearningaudioembeddings, title = {ALM2Vec: Learning Audio Embeddings for Universal Audio Retrieval with Large Audio-Language Models}, author = {Lu, Fengjie and Jiang, Chenang and Hai, Jiarui and Wang, Helin and Yee, Aaron}, year = {2026}, eprint = {2606.30682}, archivePrefix = {arXiv}, primaryClass = {cs.SD}, url = {https://arxiv.org/abs/2606.30682} }
ALM2Vec: Learning Audio Embeddings for Universal Audio Retrieval with Large Audio-Language Models
ALM2Vec: Learning Audio Embeddings for Universal Audio Retrieval with Large Audio-Language Models

Fengjie Lu, Chenang Jiang, Jiarui Hai†, Helin Wang, Aaron Yee

arXiv, 2026

A universal audio embedding framework that transfers large audio–language models into an instruction-aware retrieval space across domains and tasks.

ALM2Vec: Learning Audio Embeddings for Universal Audio Retrieval with Large Audio-Language Models

Fengjie Lu, Chenang Jiang, Jiarui Hai†, Helin Wang, Aaron Yee

arXiv, 2026

A universal audio embedding framework that transfers large audio–language models into an instruction-aware retrieval space across domains and tasks.

@article{wang2025capspeech, title = {Capspeech: Enabling downstream applications in style-captioned text-to-speech}, author = {Wang, Helin and Hai, Jiarui and Chong, Dading and Thakkar, Karan and Feng, Tiantian and Yang, Dongchao and Lee, Junhyeok and Thebaud, Thomas and Velazquez, Laureano Moro and Villalba, Jesus and Qin, Zengyi and Narayanan, Shrikanth and Elhilali, Mounya and Dehak, Najim}, journal = {IEEE Transactions on Audio, Speech and Language Processing}, year = {2026} }
CapSpeech: Enabling Downstream Applications in Style-Captioned Text-to-Speech

Helin Wang*, Jiarui Hai*, Dading Chong, Karan Thakkar, Tiantian Feng, Dongchao Yang, Junhyeok Lee, Thomas Thebaud, Laureano Moro Velazquez, Jesús Villalba, Zengyi Qin, Shrikanth Narayanan, Mounya Elhilali, Najim Dehak

TASLP, 2026

A style-captioned TTS dataset and framework enabling controllable, style-aware text-to-speech for downstream applications.

CapSpeech: Enabling Downstream Applications in Style-Captioned Text-to-Speech

Helin Wang*, Jiarui Hai*, Dading Chong, Karan Thakkar, Tiantian Feng, Dongchao Yang, Junhyeok Lee, Thomas Thebaud, Laureano Moro Velazquez, Jesús Villalba, Zengyi Qin, Shrikanth Narayanan, Mounya Elhilali, Najim Dehak

TASLP, 2026

A style-captioned TTS dataset and framework enabling controllable, style-aware text-to-speech for downstream applications.

@article{zang2026summary, title = {Summary of The Inaugural Music Source Restoration Challenge}, author = {Zang, Yongyi and Hai, Jiarui and Ge, Wanying and Kong, Qiuqiang and Dai, Zheqi and Wang, Helin and Mitsufuji, Yuki and Plumbley, Mark D}, journal = {arXiv preprint arXiv:2601.04343}, year = {2026} }
Summary of The Inaugural Music Source Restoration Challenge
Summary of The Inaugural Music Source Restoration Challenge

Yongyi Zang, Jiarui Hai, Wanying Ge, Qiuqiang Kong, Zheqi Dai, Helin Wang, Yuki Mitsufuji, Mark D. Plumbley

ICASSP, 2026 Challenge

A summary of the inaugural Music Source Restoration Challenge, covering tasks, data, baselines, and key findings.

Summary of The Inaugural Music Source Restoration Challenge

Yongyi Zang, Jiarui Hai, Wanying Ge, Qiuqiang Kong, Zheqi Dai, Helin Wang, Yuki Mitsufuji, Mark D. Plumbley

ICASSP, 2026 Challenge

A summary of the inaugural Music Source Restoration Challenge, covering tasks, data, baselines, and key findings.

@article{wang2026solospeech, title = {Solospeech: Enhancing intelligibility and quality in target speech extraction through a cascaded generative pipeline}, author = {Wang, Helin and Hai, Jiarui and Yang, Dongchao and Chen, Chen and Li, Kai and Peng, Junyi and Thebaud, Thomas and Moro-Vel{\'a}zquez, Laureano and Villalba, Jes{\'u}s and Dehak, Najim}, journal = {IEEE Transactions on Audio, Speech and Language Processing}, year = {2026} }
SoloSpeech: Enhancing Intelligibility and Quality in Target Speech Extraction Through a Cascaded Generative Pipeline

Helin Wang, Jiarui Hai, Dongchao Yang, Chen Chen, Kai Li, Junyi Peng, Thomas Thebaud, Laureano Moro Velazquez, Jesús Villalba, Najim Dehak

TASLP, 2026

A cascaded generative pipeline for target speech extraction that improves intelligibility and perceptual quality.

SoloSpeech: Enhancing Intelligibility and Quality in Target Speech Extraction Through a Cascaded Generative Pipeline

Helin Wang, Jiarui Hai, Dongchao Yang, Chen Chen, Kai Li, Junyi Peng, Thomas Thebaud, Laureano Moro Velazquez, Jesús Villalba, Najim Dehak

TASLP, 2026

A cascaded generative pipeline for target speech extraction that improves intelligibility and perceptual quality.

@inproceedings{hai2025flexsed, title = {Flexsed: Towards open-vocabulary sound event detection}, author = {Hai, Jiarui and Wang, Helin and Guo, Weizhe and Elhilali, Mounya}, booktitle = {2025 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)}, year = {2025} }
FlexSED: Towards Open-Vocabulary Sound Event Detection

Jiarui Hai, Helin Wang, Weizhe Guo, Mounya Elhilali

WASPAA, 2025 Spotlight

An open-vocabulary sound event detection approach that generalizes to unseen classes via flexible text-conditioned modeling.

FlexSED: Towards Open-Vocabulary Sound Event Detection

Jiarui Hai, Helin Wang, Weizhe Guo, Mounya Elhilali

WASPAA, 2025 Spotlight

An open-vocabulary sound event detection approach that generalizes to unseen classes via flexible text-conditioned modeling.

@inproceedings{hai2025ezaudio, title = {EzAudio: Enhancing Text-to-Audio Generation with Efficient Diffusion Transformer}, author = {Hai, Jiarui and Xu, Yong and Zhang, Hao and Li, Chenxing and Wang, Helin and Elhilali, Mounya and Yu, Dong}, booktitle = {Interspeech 2025}, year = {2025} }
EzAudio: Enhancing Text-to-Audio Generation with Efficient Diffusion Transformer

Jiarui Hai, Yong Xu, Hao Zhang, Chenxing Li, Helin Wang, Mounya Elhilali, Dong Yu

Interspeech, 2025 Oral

An efficient diffusion transformer that improves text-to-audio generation quality while reducing compute.

EzAudio: Enhancing Text-to-Audio Generation with Efficient Diffusion Transformer

Jiarui Hai, Yong Xu, Hao Zhang, Chenxing Li, Helin Wang, Mounya Elhilali, Dong Yu

Interspeech, 2025 Oral

An efficient diffusion transformer that improves text-to-audio generation quality while reducing compute.

@inproceedings{hai2025synsonic, title = {SynSonic: Augmenting Sound Event Detection through Text-to-Audio Diffusion ControlNet and Effective Sample Filtering}, author = {Hai, Jiarui and Elhilali, Mounya}, booktitle = {2025 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)}, year = {2025} }
SynSonic: Augmenting Sound Event Detection through Text-to-Audio Diffusion ControlNet and Effective Sample Filtering
SynSonic: Augmenting Sound Event Detection through Text-to-Audio Diffusion ControlNet and Effective Sample Filtering

Jiarui Hai, Mounya Elhilali

WASPAA, 2025

A text-to-audio diffusion ControlNet augmentation pipeline with sample filtering to improve sound event detection.

SynSonic: Augmenting Sound Event Detection through Text-to-Audio Diffusion ControlNet and Effective Sample Filtering

Jiarui Hai, Mounya Elhilali

WASPAA, 2025

A text-to-audio diffusion ControlNet augmentation pipeline with sample filtering to improve sound event detection.

@inproceedings{wang2025soloaudio, title = {Soloaudio: Target sound extraction with language-oriented audio diffusion transformer}, author = {Wang, Helin and Hai, Jiarui and Lu, Yen-Ju and Thakkar, Karan and Elhilali, Mounya and Dehak, Najim}, booktitle = {ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, year = {2025} }
SoloAudio: Target Sound Extraction with Language-Oriented Audio Diffusion Transformer

Helin Wang*, Jiarui Hai*, Yen-Ju Lu, Karan Thakkar, Mounya Elhilali, Najim Dehak

ICASSP, 2025

A language-conditioned audio diffusion transformer for target sound extraction from mixtures.

SoloAudio: Target Sound Extraction with Language-Oriented Audio Diffusion Transformer

Helin Wang*, Jiarui Hai*, Yen-Ju Lu, Karan Thakkar, Mounya Elhilali, Najim Dehak

ICASSP, 2025

A language-conditioned audio diffusion transformer for target sound extraction from mixtures.

DreamVoice: Text-Guided Voice Conversion
DreamVoice: Text-Guided Voice Conversion

Jiarui Hai*, Karan Thakkar*, Helin Wang, Zengyi Qin, Mounya Elhilali

Interspeech, 2024

Text-guided voice conversion that follows natural-language prompts to control voice attributes and speaking style.

DreamVoice: Text-Guided Voice Conversion

Jiarui Hai*, Karan Thakkar*, Helin Wang, Zengyi Qin, Mounya Elhilali

Interspeech, 2024

Text-guided voice conversion that follows natural-language prompts to control voice attributes and speaking style.

DPM-TSE: A Diffusion Probabilistic Model for Target Sound Extraction
DPM-TSE: A Diffusion Probabilistic Model for Target Sound Extraction

Jiarui Hai*, Helin Wang*, Dongchao Yang, Karan Thakkar, Najim Dehak, Mounya Elhilali

ICASSP, 2024

A diffusion probabilistic model for target sound extraction that separates a desired source from audio mixtures.

DPM-TSE: A Diffusion Probabilistic Model for Target Sound Extraction

Jiarui Hai*, Helin Wang*, Dongchao Yang, Karan Thakkar, Najim Dehak, Mounya Elhilali

ICASSP, 2024

A diffusion probabilistic model for target sound extraction that separates a desired source from audio mixtures.

Diff-Pitcher: Diffusion-based Singing Voice Pitch Correction
Diff-Pitcher: Diffusion-based Singing Voice Pitch Correction

Jiarui Hai, Mounya Elhilali

WASPAA, 2023 Oral

A diffusion-based method for singing voice pitch correction that adjusts pitch while preserving timbre and expression.

Diff-Pitcher: Diffusion-based Singing Voice Pitch Correction

Jiarui Hai, Mounya Elhilali

WASPAA, 2023 Oral

A diffusion-based method for singing voice pitch correction that adjusts pitch while preserving timbre and expression.

Boosting Modality Representation with Pre-trained Models and Multi-task Training for Multimodal Sentiment Analysis
Boosting Modality Representation with Pre-trained Models and Multi-task Training for Multimodal Sentiment Analysis

Jiarui Hai*, Yu-Jeh Liu*, Mounya Elhilali

ASRU, 2023

Multi-task training with pre-trained modality encoders to learn stronger multimodal representations for sentiment analysis.

Boosting Modality Representation with Pre-trained Models and Multi-task Training for Multimodal Sentiment Analysis

Jiarui Hai*, Yu-Jeh Liu*, Mounya Elhilali

ASRU, 2023

Multi-task training with pre-trained modality encoders to learn stronger multimodal representations for sentiment analysis.

Open-source Resources
Service and Teaching
Conference Reviewer
ICASSP, Interspeech, ASRU, SLT, WASPAA
Journal Reviewer
TASLP, IJCV
Teaching Assistant
Audio Signal Processing
EN.520.445/645
Fall 2025
Audio Signal Processing
EN.520.445/645
Fall 2024