Nail Polish Try-On: Realtime Semantic Segmentation of Small Objects for Native and Browser Smartphone AR Applications

Brendan Duke; Abdalla Ahmed; Edmund Phung; Irina Kezele; Parham Aarabi

Nail Polish Try-On: Realtime Semantic Segmentation of Small Objects for Native and Browser Smartphone AR Applications

Computer Vision and Pattern Recognition 2019-06-11 v2 Machine Learning

Authors: Brendan Duke , Abdalla Ahmed , Edmund Phung , Irina Kezele , Parham Aarabi

Abstract

We provide a system for semantic segmentation of small objects that enables nail polish try-on AR applications to run client-side in realtime in native and web mobile applications. By adjusting input resolution and neural network depth, our model design enables a smooth trade-off of performance and runtime, with the highest performance setting achieving~\num{94.5} mIoU at 29.8ms runtime in native applications on an iPad Pro. We also provide a postprocessing and rendering algorithm for nail polish try-on, which integrates with our semantic segmentation and fingernail base-tip direction predictions.

Keywords

semantic segmentation small object augmented reality nail polish try-on real-time mobile application neural network

Cite

@article{arxiv.1906.02222,
  title  = {Nail Polish Try-On: Realtime Semantic Segmentation of Small Objects for Native and Browser Smartphone AR Applications},
  author = {Brendan Duke and Abdalla Ahmed and Edmund Phung and Irina Kezele and Parham Aarabi},
  journal= {arXiv preprint arXiv:1906.02222},
  year   = {2019}
}

Comments

4 pages, 3 figures. CVPRW 2019: Third Workshop on Computer Vision for AR/VR

Nail Polish Try-On: Realtime Semantic Segmentation of Small Objects for Native and Browser Smartphone AR Applications

Abstract

Keywords

Cite

Comments

Related papers