Diet affects the risk and progression of prostate cancer, but the interplay between diet and genetic alterations in this disease is not understood. Here we present genetic evidence in the mouse showing that prostate cancer progression driven by loss of the tumor suppressor Pten is mainly unresponsive to a high-fat diet (HFD), but that coordinate loss of the protein tyrosine phosphatase Ptpn1 (encoding PTP1B) enables a highly invasive disease. Prostate cancer in Pten−/−Ptpn1−/− mice was characterized by increased cell proliferation and Akt activation, interpreted to reflect a heightened sensitivity to IGF-1 stimulation upon HFD feeding. Prostate-specific overexpression of PTP1B was not sufficient to initiate prostate cancer, arguing that it acted as a diet-dependent modifier of prostate cancer development in Pten−/− mice. Our findings offer a preclinical rationale to investigate the anticancer effects of PTP1B inhibitors currently being studied clinically for diabetes treatment as a new modality for management of prostate cancer.