相关论文: OptoGPT: A Foundation Model for Inverse Design in …

Reconstructing objects from posed images is a crucial and complex task in computer graphics and computer vision. While NeRF-based neural reconstruction methods have exhibited impressive reconstruction ability, they tend to be…

An optical translational projector (OTP) designed by transformation optics is applied to improve the energy transfer efficiency in a wireless energy transfer (WET) system. Numerical simulation results show our OTP can greatly enhance the…

Inverse rendering aims to reconstruct geometry and reflectance from captured images. Display-camera imaging systems offer unique advantages for this task: each pixel can easily function as a programmable point light source, and the…

We consider solving ill-posed imaging inverse problems without access to an explicit image prior or ground-truth examples. An overarching challenge in inverse problems is that there are many undesired images that fit to the observed…

Inverse design has revolutionized the field of photonics, enabling automated development of complex structures and geometries with unique functionalities unmatched by classical design. However, the use of inverse design in nonlinear…

We present Lumina-mGPT, a family of multimodal autoregressive models capable of various vision and language tasks, particularly excelling in generating flexible photorealistic images from text descriptions. By initializing from multimodal…

Sum frequency generation (SFG) has multiple applications, from optical sources to imaging, where efficient conversion requires either long interaction distances or large field concentrations in a quadratic nonlinear material. Metasurfaces…

Designing free-form photonic devices is fundamentally challenging due to the vast number of possible geometries and the complex requirements of fabrication constraints. Traditional inverse-design approaches--whether driven by human…

Gradient-based inverse design in photonics has already achieved remarkable results in designing small-footprint, high-performance optical devices. The adjoint variable method, which allows for the efficient computation of gradients, has…

On-chip optical neural networks (ONNs) have recently emerged as an attractive hardware accelerator for deep learning applications, characterized by high computing density, low latency, and compact size. As these networks rely heavily on…

The reconstruction of X-rays CT images from sparse or limited-angle geometries is a highly challenging task. The lack of data typically results in artifacts in the reconstructed image and may even lead to object distortions. For this…

Diffractive optical information processors have demonstrated significant promise in delivering high-speed, parallel, and energy efficient inference for scaling machine learning tasks. Training, however, remains a major computational…

Metaoptics are thin, planar surfaces consisting of many subwavelength optical resonators that can be designed to simultaneously control the amplitude, phase, and polarization to arbitrarily shape an optical wavefront much in the same manner…

Transformers are a popular choice for classification tasks and as backbones for object detection tasks. However, their high latency brings challenges in their adaptation to lightweight object detection systems. We present an approximation…

Despite remarkable progress in image translation, the complex scene with multiple discrepant objects remains a challenging problem. The translated images have low fidelity and tiny objects in fewer details causing unsatisfactory performance…

The design of the optimal inverse discrete cosine transform (IDCT) to compensate the quantization error is proposed for effective lossy image compression in this work. The forward and inverse DCTs are designed in pair in current image/video…

Adaptive optics in combination with multi-photon techniques is a powerful approach to image deep into a specimen. Remarkably, virtually all adaptive optics schemes today rely on wavefront modulators which are reflective, diffractive, or…

A new approach to the design of graded Photonic Crystals (GPCs) devices is proposed by exploiting the inverse scattering framework as a synthesis tool. The introduced general methodology can be applied to arbitrary far-field specifications,…

Multi-task dense scene understanding is a thriving research domain that requires simultaneous perception and reasoning on a series of correlated tasks with pixel-wise prediction. Most existing works encounter a severe limitation of modeling…

Inverse design represents a paradigm shift in the development of nanophotonic devices, where optimal geometries and materials are discovered by an algorithm rather than symmetry considerations or intuition. Here we present a very general…