High-dimensional photonic systems access large Hilbert spaces for quantum information processing. They offer proven advantages in quantum computation, communication, and sensing. However, implementing scalable, low-loss unitary gates across many modes remains a central challenge. Here we propose a deterministic, universal, and fully programmable high-dimensional quantum gate based on a cavity-assisted sum-frequency-generation process, achieving near-unity fidelity. The device implements an M-by-N truncated unitary transformation (with 1 <= M < N), or a full unitary when M = N, on frequency-bin modes. With current technology, the attainable dimensionality reaches M-by-N on the order of ten to the power of four, with N up to about one thousand, and can be further increased using multiple pulse shapers. Combined with compatible SPDC sources, high-efficiency detection, and fast feed-forward, this approach provides a scalable, fiber-compatible platform for high-dimensional frequency-bin quantum processing.