The first of the automatic reconstruction of buildings from aerial images is to find where the buildings are. Many methods can be used for this work, but if the aerial photo is color, it is a very useful information for finding the houses because usually the color of one single roof is uniform and is different from the color of adjacent objects likes grass,roads,etc. Two aspects are researched here to study the possible usage of color information. First, we try to use color information to enhance linear feature extraction. Second, we try to omit the step of linear feature extraction and use the color information to extract building regions directly. In order to test the feasibility of the thought, we choose three different types of building images for experiment. The first type includes isolated buildings, the second type includes aggregation of buildings, and in the third type there are high apartment buildings. The results of the experiment show that in one hand color information do help to extract more complete linear features, and in the other hand,color information can very effectively be used to extract building regions, presumed the color of each individual roof is uniform. It proves that color information is indeed helpful for feature extraction.