We present a theory of radio-frequency spectroscopy of impurities interacting with a quantum gas at finite temperature. By working in the canonical ensemble of a single impurity, we show that the impurity spectral response is directly connected to the finite-temperature equation of state (free energy) of the impurity. We consider two different response protocols: Injection, where the impurity is introduced into the medium from an initially noninteracting state, and ejection, where the impurity is ejected from an initially interacting state with the medium. We show that there is a simple mapping between injection and ejection spectra, which is connected to the detailed balance condition in thermal equilibrium. To illustrate the power of our approach, we specialize to the case of the Fermi polaron, corresponding to an impurity atom that is immersed in a noninteracting Fermi gas. For a mobile impurity with a mass equal to the fermion mass, we employ a finite-temperature variational approach to obtain the impurity spectral response. We find a striking nonmonotonic dependence on temperature in the impurity free energy, the contact, and the radio-frequency spectra. For the case of an infinitely heavy Fermi polaron, we derive exact results for the finite-temperature free energy, thus generalizing Fumi's theorem to arbitrary temperature. We also determine the exact dynamics of the contact after a quench of the impurity-fermion interactions. Finally, we show that the injection and ejection spectra obtained from the variational approach compare well with the exact spectra, thus demonstrating the accuracy of our approximation method.