Endomorphisms of the symmetric 2-rig of finite sets

Let $\widehat{\mathbb{F}\mathbb{S}et}$ be the groupoid of finite sets and bijections between them equipped with the canonical symmetric rig category structure given by the disjoint union and the cartesian product of finite sets. We prove that the category (in fact, groupoid) of endomorphisms of $\widehat{\mathbb{F}\mathbb{S}et}$ is equivalent to the terminal category, thus providing some evidence that $\widehat{\mathbb{F}\mathbb{S}et}$ is the right categorical analog of the commutative rig $\mathbb{N}$ of nonnegative integers. This is shown using a particular semistrict skeletal version of $\widehat{\mathbb{F}\mathbb{S}et}$ for which the endomorphisms can be described very explicitly.