We identify several beneficial characteristics of polarization spectroscopy as an absolute atomic reference for frequency stabilization of lasers, and demonstrate sub-kilohertz laser spectral linewidth narrowing using polarization spectroscopy with high-bandwidth feedback. Polarization spectroscopy provides a highly dispersive velocity-selective absolute atomic reference based on frequency-dependent birefringence in an optically pumped atomic gas. The pumping process leads to dominance of the primary closed transition, suppressing closely-spaced subsidiary resonances which reduce the effective capture range for conventional atomic references. The locking signal is based on subtraction of two orthogonal polarization signals, reducing the effect of laser intensity noise to the shot noise limit. We measure noise-limited servo bandwidth comparable to that of a high-finesse optical cavity without the frequency limit or complexity imposed by optical modulation normally associated with high bandwidth laser frequency stabilization. We demonstrate narrowing to 600±100 Hz laser linewidth using the beatnote between two similarly locked external cavity diode lasers.