Compared to its female counterpart, the microbiota of the male genital tract has not been studied extensively. With this study, we aimed to evaluate the bacterial composition of seminal fluid and its impact on sperm parameters. We hypothesized that a dysbiotic microbiota composition may have an influence on sperm quality. Semen samples of 26 men with normal spermiogram and 68 men with at least one abnormal spermiogram parameter were included in the study. Samples were stratified based on total sperm count, spermatozoa concentration, progressive motility, total motility and spermatozoa morphology. Microbiota profiling was performed using 16S rRNA gene amplicons sequencing and total bacterial load was determined using a panbacterial quantitative PCR. Semen samples broadly clustered into three microbiota profiles: Prevotella-enriched, Lactobacillus-enriched, and polymicrobial. Prevotella-enriched samples had the highest bacterial load (p < 0.05). Network analysis identified three main co-occurrence modules, among which two contained bacteria commonly found in the vaginal flora. Genera from the same module displayed similar oxygen requirements, arguing for the presence of different ecological niches for bacteria that colonize semen through the passage. Contrary to our hypothesis, shifts in overall microbiota composition (beta-diversity) did not correlate with spermiogram parameters. Similarly, we did not find any difference in microbial richness or diversity (alpha-diversity). Differential abundance testing, however, revealed three specific genera that were significantly enriched or depleted in some of the sperm quality groups (p < 0.05). Prevotella relative abundance was increased in samples with defective sperm motility while Staphylococcus was increased in the corresponding control group. In addition, we observed an increased relative abundance of Lactobacillus in samples with normal sperm morphology. Our study indicates that overall bacterial content of sperm might not play a major role in male infertility. Although no major shifts in microbiota composition or diversity were found, the differential abundance of specific bacterial genera in the sperm suggests that a small subset of microbes might impact the spermatozoal physiology during sperm transition, more specifically motility and morphology. Further studies are required to challenge this finding and develop potential strategies to induce the formation of a healthy seminal microbiota.