Abstract: With the increasing of remote sensing information sources, a variety of remote sensing data are used to interpret the seismic damage of buildings in seismic damage assessment. In order to accurately interpret the seismic damage grade and establish the automatic identification model, this paper focuses on the analyses of the characteristics of all kinds of buildings in the optical images, the imaging characteristics of synthetic aperture radar (SAR) images, and the three-dimensional characteristics of the light detection and ranging (LiDAR) images by means of visual interpretation, image processing, statistical analyses after collecting the remote sensing images of the M_S8.0 Wenchuan earthquake in 2008 and M_S7.1 Yushu earthquake in 2010. Furthermore, several simplified models of the building are constructed, and the profile features of the images involving many kinds destroyed building are analyzed in combination with the images from optical and radar sensors. The related study results show that the color information from optical remote sensing image is in accordance with the human eye color principle, which is propitious to obtain better interpretation of results; the SAR image can reflect the features of side/surface roughness and angle reflection; LiDAR image can obtain the three-dimensional information of the building. Therefore, the multi-source remote sensing data should be integrated effectively so as to achieve the best effect in seismic damage assessment.