Related papers: Log Complex Color for Visual Pattern Recognition o…
Optical vibration sensing enables recovering the scene sound directly from the surface vibration of nearby objects, turning everyday objects into ``visual microphones''. However, most prior methods had focused on capturing the vibrations of…
The amplitude and phase of a material's nonlinear optical response provide insight into the underlying electronic dynamics that determine its optical properties. Phase-sensitive nonlinear spectroscopy techniques are widely implemented to…
High harmonic spectroscopy has the potential to combine attosecond temporal with sub-Angstrom spatial resolution of the early nuclear and multielectron dynamics in molecules. It involves strong field ionization of the molecule by the IR…
Light echoes give us a unique perspective on the nature of supernovae and non-terminal stellar explosions. Spectroscopy of light echoes can reveal details on the kinematics of the ejecta, probe asymmetry, and reveal details on its…
The mixed quantum/classical approach of Skinner and co-workers has been widely used to calculate the line shapes of the infrared spectra of water (H2O), but less attention has been paid to the use of this approach in quantitatively…
The audio spectrogram is a time-frequency representation that has been widely used for audio classification. One of the key attributes of the audio spectrogram is the temporal resolution, which depends on the hop size used in the Short-Time…
Modern day audio signal classification techniques lack the ability to classify low feature audio signals in the form of spectrographic temporal frequency data representations. Additionally, currently utilized techniques rely on full diverse…
Modern approaches to sound synthesis using deep neural networks are hard to control, especially when fine-grained conditioning information is not available, hindering their adoption by musicians. In this paper, we cast the generation of…
These notes offer a unified introduction to spectral methods for the study of complex systems. They are intended as an operative manual rather than a theorem-proof textbook: the emphasis is on tools, identities, and perspectives that can be…
Historically, most speech models in machine-learning have used the mel-spectrogram as a speech representation. Recently, discrete audio tokens produced by neural audio codecs have become a popular alternate speech representation for speech…
Multi-modality perception is essential to develop interactive intelligence. In this work, we consider a new task of visual information-infused audio inpainting, \ie synthesizing missing audio segments that correspond to their accompanying…
Visual sound source separation aims at identifying sound components from a given sound mixture with the presence of visual cues. Prior works have demonstrated impressive results, but with the expense of large multi-stage architectures and…
Sound event detection systems typically consist of two stages: extracting hand-crafted features from the raw audio waveform, and learning a mapping between these features and the target sound events using a classifier. Recently, the focus…
The wavelength calibration of spectrographs is an essential but challenging task in many disciplines. Calibration is traditionally accomplished by imaging the spectrum of a light source containing features that are known to appear at…
Consumer-grade music recordings such as those captured by mobile devices typically contain distortions in the form of background noise, reverb, and microphone-induced EQ. This paper presents a deep learning approach to enhance low-quality…
Quantum spectroscopy seeks to probe chemical systems using nonclassical light, which has properties that are qualitatively and quantitatively different than conventional light sources. One promising technique uses intensity-correlated twin…
Spectral envelope is one of the most important features that characterize the timbre of an instrument sound. However, it is difficult to use spectral information in the framework of conventional spectrogram decomposition methods. We…
Light-matter interaction is exploited in spectroscopic techniques to access information about molecular, atomic or nuclear constituents of the sample of interest. While scattered light carries both amplitude and phase information of the…
Singing voice separation aims to separate music into vocals and accompaniment components. One of the major constraints for the task is the limited amount of training data with separated vocals. Data augmentation techniques such as random…
We present a new method for real- and complex-valued image reconstruction from two intensity measurements made in the Fourier plane: the Fourier magnitude of the unknown image, and the intensity of the interference pattern arising from…