Related papers: Photonic Modes Prediction via Multi-Modal Diffusio…

Structural coloration is commonly modeled using wave optics for reliable and photorealistic rendering of natural, quasi-periodic and complex nanostructures. Such models often rely on dense, preliminary or preprocessed data to accurately…

We demonstrate that embedding physics-driven constraints into machine learning process can dramatically improve accuracy and generalizability of the resulting model. Physics-informed learning is illustrated on the example of analysis of…

Transformer models trained on massive text corpora have become the de facto models for a wide range of natural language processing tasks. However, learning effective word representations for function words remains challenging. Multimodal…

Models for inferring monocular shape of surfaces with diffuse reflection -- shape from shading -- ought to produce distributions of outputs, because there are fundamental mathematical ambiguities of both continuous (e.g., bas-relief) and…

Vision-language pretrained models have seen remarkable success, but their application to safety-critical settings is limited by their lack of interpretability. To improve the interpretability of vision-language models such as CLIP, we…

Diffusion models for image generation function by progressively adding noise to an image set and training a model to separate out the signal from the noise. The noise profile used by these models is white noise -- that is, noise based on…

We present a novel learning-based method to build a differentiable computational model of a real fluorescence microscope. Our model can be used to calibrate a real optical setup directly from data samples and to engineer point spread…

Text-to-image diffusion models are now capable of generating images that are often indistinguishable from real images. To generate such images, these models must understand the semantics of the objects they are asked to generate. In this…

Understanding the structure of real data is paramount in advancing modern deep-learning methodologies. Natural data such as images are believed to be composed of features organized in a hierarchical and combinatorial manner, which neural…

Load forecasting plays a pivotal role in the safe and stable operation of power systems. Conventional deep learning methods often struggle to adapt to few-shot scenarios frequently encountered in industrial applications. Existing…

In this article we address the theoretical study of a multiscale drift-diffusion (DD) model for the description of photoconversion mechanisms in organic solar cells. The multiscale nature of the formulation is based on the co-presence of…

We address the reconstruction of the full photon distribution of multimode fields generated by seeded parametric down-conversion (PDC). Our scheme is based on on/off avalanche photodetection assisted by maximum-likelihood (MaxLik)…

State-of-the-art learned reconstruction methods often rely on black-box modules that, despite their strong performance, raise questions about their interpretability and robustness. Here, we build on a recently proposed image reconstruction…

Multimodal semantic understanding often has to deal with uncertainty, which means the obtained messages tend to refer to multiple targets. Such uncertainty is problematic for our interpretation, including inter- and intra-modal uncertainty.…

Edge intelligence is constrained by the energy and latency costs of shuttling data through electronic memory hierarchies. Optical systems offer a fundamentally different computational regime: once an input wavefront is launched into a…

Pre-trained diffusion models have demonstrated remarkable proficiency in synthesizing images across a wide range of scenarios with customizable prompts, indicating their effective capacity to capture universal features. Motivated by this,…

When multimode optical fibers are perturbed, the data that is transmitted through them is scrambled. This presents a major difficulty for many possible applications, such as multimode fiber-based telecommunication and endoscopy. To overcome…

Advancements in generative models have sparked significant interest in generating images while adhering to specific structural guidelines. Scene graph to image generation is one such task of generating images which are consistent with the…

The dislocation created in the topological material lays the foundation of many significant findings to control light but requires delicate fabrication of the material. To extend its flexibility and reconfigurability, we propose the…

Professional-grade software applications are powerful but complicated$-$expert users can achieve impressive results, but novices often struggle to complete even basic tasks. Photo editing is a prime example: after loading a photo, the user…