Learning and predicting photonic responses of plasmonic nanoparticle assemblies via dual variational autoencoders

We demonstrate the application of machine learning for rapid and accurate extraction of plasmonic particles cluster geometries from hyperspectral image data via a dual variational autoencoder (dual-VAE). In this approach, the information is shared between the latent spaces of two VAEs acting on the particle shape data and spectral data, respectively, but enforcing a common encoding on the shape-spectra pairs. We show that this approach can establish the relationship between the geometric characteristics of nanoparticles and their far-field photonic responses, demonstrating that we can use hyperspectral darkfield microscopy to accurately predict the geometry (number of particles, arrangement) of a multiparticle assemblies below the diffraction limit in an automated fashion with high fidelity (for monomers (0.96), dimers (0.86), and trimers (0.58). This approach of building structure-property relationships via shared encoding is universal and should have applications to a broader range of materials science and physics problems in imaging of both molecular and nanomaterial systems.