Related papers: Quantifying and Correlating Rhythm Formants in Spe…
Large-scale latent diffusion models (LDMs) excel in content generation across various modalities, but their reliance on phonemes and durations in text-to-speech (TTS) limits scalability and access from other fields. While recent studies…
Typical high quality text-to-speech (TTS) systems today use a two-stage architecture, with a spectrum model stage that generates spectral frames and a vocoder stage that generates the actual audio. High-quality spectrum models usually…
We consider a round turbulent jet grazing a rectangular plate angled at $45^\circ$. Through sound pressure measurements, the tonal dynamics associated with jet-edge interaction are explored in a parameter space comprising jet Mach number,…
During voiced speech, the human vocal folds interact with the vocal tract acoustics. The resulting glottal source-resonator coupling has been observed using mathematical and physical models as well as in in vivo phonation. We propose a…
Far-field speech recognition in noisy and reverberant conditions remains a challenging problem despite recent deep learning breakthroughs. This problem is commonly addressed by acquiring a speech signal from multiple microphones and…
Inspired by the recent experimental signatures of relativistic hydrodynamics in graphene, we investigate theoretically the behavior of hydrodynamic sound modes in such quasi-relativistic fluids near charge neutrality, within linear…
Large Language Models (LLMs) show promise in lyric-to-melody generation, but models trained with Supervised Fine-Tuning (SFT) often produce musically implausible melodies with issues like poor rhythm and unsuitable vocal ranges, a…
Large language models (LLMs) and multimodal models have become powerful general-purpose reasoning systems. However, radio-frequency (RF) signals, which underpin wireless systems, are still not natively supported by these models. Existing…
This study experimentally validates a numerical model of electromagnetic propagation through the human head during the pronunciation of different vowels, with the goal of improving our understanding of the underlying physical phenomena. A…
Formant synthesis aims to generate speech with controllable formant structures, enabling precise control of vocal resonance and phonetic features. However, while existing formant synthesis approaches enable precise formant manipulation,…
Multi-channel speech enhancement extracts speech using multiple microphones that capture spatial cues. Effectively utilizing directional information is key for multi-channel enhancement. Deep learning shows great potential on multi-channel…
Mechanical resonators that possess coupled modes with harmonic frequency relations have recently sparked interest due to their suitability for controllable energy transfer and non-Hermitian dynamics. Here, we show coupling between high…
LLM-based automatic speech recognition models demonstrate strong performance by connecting audio encoders and LLMs. However, data scarcity of paired speech and transcription often hinders their adaptation to new domains, making text-only…
A new mechanism of low frequency (1/f-like) noise generation is described and analyzed. It is attributed to higher frequency asymmetric resonance modes, which are stimulated by a random factor, e.g. due to thermal excitation.…
Flow-Matching (FM)-based zero-shot text-to-speech (TTS) systems exhibit high-quality speech synthesis and robust generalization capabilities. However, the speaker representation ability of such systems remains underexplored, primarily due…
Various resonant and near-resonant patterns form in a light-sensitive Belousov-Zhabotinsky (BZ) reaction in response to a spatially-homogeneous time-periodic perturbation with light. The regions (tongues) in the forcing frequency and…
Most speech enhancement algorithms make use of the short-time Fourier transform (STFT), which is a simple and flexible time-frequency decomposition that estimates the short-time spectrum of a signal. However, the duration of short STFT…
This paper proposes a novel time-frequency warped waveform for short symbols, massive machine-type communication (mMTC), and internet of things (IoT) applications. The waveform is composed of asymmetric raised cosine (RC) pulses to increase…
Low frequency oscillator (LFO) driven audio effects such as phaser, flanger, and chorus, modify an input signal using time-varying filters and delays, resulting in characteristic sweeping or widening effects. It has been shown that these…
Recent advances in large language models (LLMs) and development of audio codecs greatly propel the zero-shot TTS. They can synthesize personalized speech with only a 3-second speech of an unseen speaker as acoustic prompt. However, they…