Cervical cancer ranks first in incidence among malignant tumors of the female reproductive system, and 80% of women who die from cervical cancer worldwide are from developing countries. VIA screening based on artificial intelligence-assisted diagnosis can provide a cheap and rapid screening method. This will attract more low-income women to volunteer for regular cervical cancer screening. However, current AI-based VIA screening studies either have low accuracy or require expensive equipment assistance. In this paper, we propose the HMCFormer (Hierarchical Multi-Scale Convolutional Transformer) network, which combines the hierarchical feature extraction capability of CNNs and the global dependency modeling capability of Transformers to address the challenges of realizing intelligent VIA screening. HMCFormer can be divided into a Transformer branch and a CNN branch. The Transformer branch receives unenhanced lesion sample images, and the CNN branch receives lesion sample images enhanced by the proposed dual-color space-based image enhancement algorithm. The authors design a Hierarchical Multi-Scale Pixel Excitation (HMSPE) module for adaptive multi-scale and multi-level local feature extraction. The authors apply the structure of the Swin Transformer network with minor modifications in the global perception modeling process. In addition, the authors propose two feature fusion concepts: adaptive preprocessing and superiority-inferiority fusion, and design a feature fusion module based on these concepts, which significantly improves the collaborative ability of the Transformer branch and the CNN branch.