Due to the huge demand and the importance of 3-D city models, an effective solution for geometrical building modeling is indispensable. From the application point of view, an interactive system for geometrical building modeling based on a semi-automatic approach is presented. A true-orthophoto rectification scheme based on the generated building models is also proposed. The data used in the proposed system are 3-D visible roof-edges that have been manually measured from a stereo-model, or the 3-D building outlines from a digital topographic map. The core technology is called the "Split-Merge-Shape" (SMS) method. The SPLIT and MERGE steps are the two main procedures for topology reconstruction from non-related roof-edges. The SHAPE step uses the available roof-edge height information to define an appropriate rooftop. With the SMS method, the topographic mapping of buildings, and geometric building modeling, can be seamlessly integrated into a unified procedure. In a conventional ortho-rectification procedure using a digital terrain model, the correction of relief displacement due to buildings is not considered. In order to restore the most complete information and to correct the relief displacement of buildings, a rigorous true-orthophoto rectification scheme is~proposed. In the proposed scheme, 3-D building models, multi-view aerial images, digital terrain models, and the sun's orientation are all utilized. Terrain and building relief displacement corrections as well as hidden area compensation, seamless mosaicking, and shadowed area enhancement are integrated into one procedure. In this paper, the SMS method is presented, including an evaluation of its feasibility, robustness, efficiency and accuracy. Finally, the generated building models are applied to true-orthophoto generation and 3-D GIS querying.