MSFM-UNET: enhancing medical image segmentation with multi-scale and multi-view frequency fusion

Medical image segmentation benefits greatly from accurate and efficient models. Although CNNs and Transformer-based models are widely regarded as foundational methods in the realm of medical image segmentation, each has inherent drawbacks: Convolutional Neural Networks (CNNs) frequently face challenges when it comes to accurately capturing long-range relationships because of their limited receptive fields. Conversely, Transformers excel at capturing long-range relationships but come with a high computational cost. To address these challenges, State Space Models (SSMs) like Mamba have emerged as a promising alternative, providing an effective method to represent long-range interactions while maintaining a linear complexity. In this study, we present the Multi-Scale and Multi-View Frequency Mamba UNet (MSFM-UNet), a model specifically designed to leverage Mamba’s unique strengths for improving medical image segmentation. Additionally, the Multi-Scale Feature Aggregation (MSFA) effectively merges the feature outputs generated by each encoder block with those from the decoder. Furthermore, the Multi-View Frequency Enhancement (MVFA) is employed to simultaneously capture global and local perspectives, combining frequency domain attributes to improve the representation of features across multiple scales. We performed a comprehensive evaluation of MSFM-UNet on four widely recognized public datasets: ISIC17, ISIC18, Synapse, and ACDC. The experimental results clearly demonstrate that MSFM-UNet outperforms the current leading models in medical image segmentation. The code is made publicly available at https://github.com/qczggaoqiang/MSFM-UNet.