Although LEGO sets have entertained generations of children and adults, the challenge of designing customized builds matching the complexity of real-world or imagined scenes remains too great for the average enthusiast. In order to make this feat possible, we implement a system that generates a LEGO brick model from 2D images. We design a novel solution to this problem that uses an octree-structured autoencoder trained on 3D voxelized models to obtain a feasible latent representation for model reconstruction, and a separate network trained to predict this latent representation from 2D images. LEGO models are obtained by algorithmic conversion of the 3D voxelized model to bricks. We demonstrate first-of-its-kind conversion of photographs to 3D LEGO models. An octree architecture enables the flexibility to produce multiple resolutions to best fit a user's creative vision or design needs. In order to demonstrate the broad applicability of our system, we generate step-by-step building instructions and animations for LEGO models of objects and human faces. Finally, we test these automatically generated LEGO sets by constructing physical builds using real LEGO bricks.
Cite
@article{arxiv.2108.08477,
title = {Image2Lego: Customized LEGO Set Generation from Images},
author = {Kyle Lennon and Katharina Fransen and Alexander O'Brien and Yumeng Cao and Matthew Beveridge and Yamin Arefeen and Nikhil Singh and Iddo Drori},
journal= {arXiv preprint arXiv:2108.08477},
year = {2021}
}