This study assessed inter- and intra-observer variation in the derivation of setup positioning using four-dimensional (4D) transperineal ultrasound (TPUS). Seven radiation therapists with 4–16 months of TPUS clinical experience independently derived the setup position on a single fraction for 10 patients undergoing prostate radiotherapy. For each patient, the reference TPUS prostate contour was co-registered to a TPUS image acquired during treatment and inter-observer variation assessed. Intra-observer variation was assessed after two weeks. Inter-observer setup variation from the group median was ≤2 mm in 93.8% of cases (maximum deviation 5.4 mm). Spearman's rho correlation test resulted in a moderate to high positive correlation for inter-observer variation. The median (range) Spearman's rho correlation coefficients were 0.675 (0.439–0.939), 0.912 (0.669–1.000) and 0.964 (0.8674–1.000) for the x, y and z directions respectively. The median (range) intra-observer variation was ≤2 mm in 93.3% (60%–100%) of cases (maximum deviation 4.9 mm). Wilcoxon matched-pairs signed ranks tests yielded no statistically significant difference for intra-observer TPUS-derived setup shifts (p-value range = 0.151–0.803). The magnitude of observer variation appeared to be influenced by training and/or the length of user experience. 4D TPUS is a promising non-invasive ultrasound-based image-guided radiation therapy solution for daily treatment setup with minimal inter- and intra-observer variation.