Existing techniques to encode spatial invariance within deep convolutional neural networks only model 2D transformation fields. This does not account for the fact that objects in a 2D space are a projection of 3D ones, and thus they have limited ability to severe object viewpoint changes. To overcome this limitation, we introduce a learnable module, cylindrical convolutional networks (CCNs), that exploit cylindrical representation of a convolutional kernel defined in the 3D space. CCNs extract a view-specific feature through a view-specific convolutional kernel to predict object category scores at each viewpoint. With the view-specific feature, we simultaneously determine objective category and viewpoints using the proposed sinusoidal soft-argmax module. Our experiments demonstrate the effectiveness of the cylindrical convolutional networks on joint object detection and viewpoint estimation.
@article{arxiv.2003.11303,
title = {Cylindrical Convolutional Networks for Joint Object Detection and Viewpoint Estimation},
author = {Sunghun Joung and Seungryong Kim and Hanjae Kim and Minsu Kim and Ig-Jae Kim and Junghyun Cho and Kwanghoon Sohn},
journal= {arXiv preprint arXiv:2003.11303},
year = {2020}
}