Originally selected as a neutron star (NS) candidate in the ROSAT All-Sky Survey, 1RXS J141256.0+792204 ("Calvera") was discovered to be a 59 ms X-ray pulsar in a pair of XMM-Newton observations (Zane et al. 2011). Surprisingly, their claimed detection of this pulsar in Fermi gamma-ray data requires no period derivative, severely restricting its dipole magnetic field strength, spin-down luminosity, and distance to small values. This implies that the cooling age of Calvera is much younger than its characteristic spin-down age. If so, it could be a mildly recycled pulsar, or the first "orphaned" central compact object (CCO). Here we show that the published Fermi ephemeris fails to align the pulse phases of the two X-ray observations with each other, which indicates that the Fermi detection is almost certainly spurious. Analysis of additional Fermi data also does not confirm the gamma-ray detection. This leaves the spin-down rate of Calvera less constrained, and its place among the families of NSs uncertain. It could still be either a normal pulsar, a mildly recycled pulsar, or an orphaned CCO.