The power of student's t test: can a non-similar test do better?

Lehmann & Stein (1948) proved the existence of non‐similar tests which can be more powerful than best similar tests. They used Student's problem of testing for a non‐zero mean given a random sample from the normal distribution with unknown variance as an example. This raises the question: should we use a non‐similar test instead of Student's t test? Questions like this can be answered by comparing the power of the test with the power envelope. This paper discusses the difficulties involved in computing power envelopes. It reports an empirical comparison of the power of the t test and the power envelope and finds that the two are almost identical especially for sample sizes greater than 20. These findings suggest that, as well as being uniformly most powerful (UMP) within the class of similar tests, Student's t test is approximately UMP within the class of all tests. For practical purposes it might also be regarded as UMP when moderate or large sample sizes are involved.