A complex network of gene regulation and interaction drives male sex determination and differentiation. While many important protein-coding genes have been identified that are necessary for proper male development, many disorders of sex development in human are still unexplained at the molecular level. This suggests that key factors and regulatory mechanisms are still unknown. In recent years, extensive data have shown that different classes of non-coding RNA play a role in almost all developmental and physiological pathways. Here we review what is known about their role in male sex determination and differentiation not only in human and other mammals, but also other species. While for some processes a key role for non-coding RNA has been identified, we are still far from having a complete picture.