The defect inspection of the steel surface is crucial to modern manufacturing and highly depends on inefficient manual work. The emergence of deep learning has prompted the development of automated defect detection methods, but the current methods perform badly in the detection of the crazing and rolled-in scale-two types of defects on steel surfaces. The difficulty in the detection of crazing and rolled-in scale is mainly due to the similarity between object regions and background regions. Based on this, the authors propose a supervised spatial-attention module (SSAM). It introduces a priori knowledge compared to the traditional spatial attention mechanism, which can enhance the supervision of relevant parameters in the attention mechanism module during network training. Finally, they introduced the SSAM to the YOLOv5 and got the SSAM-YOLO. The test result on the NEU-DET dataset shows that the proposed method has better detection accuracy, achieving improvements of 7.3% and 3.02% on the AP@0.5 for the crazing and rolled-in scale. The method also outperforms the comparative main stream algorithms for steel surface defect detection, verifying the effectiveness of our algorithm.