Extracting building information from a large number of complex remote sensing images has always been an important research content of remote sensing intelligent applications.Aiming at the problems of inaccurate building segmentation in remote sensing images in complex environments and easy neglect of small building segmentation, this paper proposes a semantic segmentation algorithm for remote sensing images based on attention mechanism and Deeplabv3+ network-SC-deep. The network adopts the encoding-decoding structure, extracts deep features and shallow features by using the backbone residual attention network, aggregates the spatial and channel information weights of remote sensing images through the hollow space pyramid pooling module and channel spatial attention module, and effectively uses the multi-scale information of buildings in remote sensing images, thereby reducing the loss of image details in training and improving the extraction accuracy of the model. Experimental results show that the segmentation results of the proposed method on the Aerial imagery dataset are better than other mainstream segmentation networks, and can effectively identify and extract complex building edges and small buildings, showing better building extraction performance.