Related papers: Spectrogram features for audio and speech analysis

A spectrogram of a ship wake is a heat map that visualises the time-dependent frequency spectrum of surface height measurements taken at a single point as the ship travels by. Spectrograms are easy to compute and, if properly interpreted,…

Spectral imaging is an umbrella term for energy-resolved x-ray imaging in medicine. The technique makes use of the energy dependence of x-ray attenuation to either increase the contrast-to-noise ratio, or to provide quantitative image data…

The tongue's intricate 3D structure, comprising localized functional units, plays a crucial role in the production of speech. When measured using tagged MRI, these functional units exhibit cohesive displacements and derived quantities that…

This paper considers a representation learning strategy to model speech signals from patients with Parkinson's disease and cleft lip and palate. In particular, it compares different parametrized representation types such as wideband and…

In recent years, the spectral analysis of appropriately defined kernel matrices has emerged as a principled way to extract the low-dimensional structure often prevalent in high-dimensional data. Here we provide an introduction to spectral…

One of the major problems in modeling natural signals is that signals with very similar structure may locally have completely different measurements, e.g., images taken under different illumination conditions, or the speech signal captured…

Network representations are useful for describing the structure of a large variety of complex systems. Although most studies of real-world networks suppose that nodes are connected by only a single type of edge, most natural and engineered…

While deep learning has reduced the prevalence of manual feature extraction, transformation of data via feature engineering remains essential for improving model performance, particularly for underwater acoustic signals. The methods by…

Large-scale multimodal contrastive learning has recently achieved impressive success in learning rich and transferable representations, yet it remains fundamentally limited by the uniform treatment of feature dimensions and the neglect of…

Convolutional Neural Networks are widely used in various machine learning domains. In image processing, the features can be obtained by applying 2D convolution to all spatial dimensions of the input. However, in the audio case, frequency…

Linguistic information is encoded at varying timescales (subwords, phrases, etc.) and communicative levels, such as syntax and semantics. Contextualized embeddings have analogously been found to capture these phenomena at distinctive layers…

The push to train ever larger neural networks has motivated the study of initialization and training at large network width. A key challenge is to scale training so that a network's internal representations evolve nontrivially at all…

There is a large ongoing scientific effort in mechanistic interpretability to map embeddings and internal representations of AI systems into human-understandable concepts. A key element of this effort is the linear representation…

Machine learning in atomistic materials science has grown to become a powerful tool, with most approaches focusing on atomic arrangements, typically decomposed into local atomic environments. This approach, while well-suited for…

With the advent of modern AI architectures, a shift has happened towards end-to-end architectures. This pivot has led to neural architectures being trained without domain-specific biases/knowledge, optimized according to the task. We in…

Two-dimensional (2D) materials have attracted extensive attention due to their unique characteristics and application potentials. Raman spectroscopy, as a rapid and non-destructive probe, exhibits distinct features and holds notable…

Speaker embeddings are widely used in speaker verification systems and other applications where it is useful to characterise the voice of a speaker with a fixed-length vector. These embeddings tend to be treated as "black box" encodings,…

Self-supervised models, namely, wav2vec and its variants, have shown promising results in various downstream tasks in the speech domain. However, their inner workings are poorly understood, calling for in-depth analyses on what the model…

Second-order variational properties have been shown to play important theoretical and numerical roles for different classes of optimization problems. Among such properties, twice epi-differentiability has a special place because of its…

Despite their ability to represent highly expressive functions, deep learning models seem to find simple solutions that generalize surprisingly well. Spectral bias -- the tendency of neural networks to prioritize learning low frequency…