Digital scanning devices such as LiDAR have recently become affordable and available. They are capable of acquiring high-accuracy and high-resolution point clouds. Thus, the techniques for point cloud modeling have received increasingly attentions in the last decade. As the approaches reconstruct the point clouds, they face a common problem: how to handle point clouds with inherent noises. Moreover, it will be especially challenge in handing point clouds that contains sharp features, e.g., city buildings. In the paper, a novel template-based modeling approach for 3D point clouds sampled from unknown city buildings is introduced. A hierarchy algebraic template, comprising of three types of primitive geometries (that is, plane, sphere, and cylinder), is used to fit point clouds. The algebraic template is organized in a hierarchical manner. The first-level, i.e., the lowest-level, consists of the primitive geometries which are represented in algebra form. These primitive geometries are merged into 3D objects with simple shapes in the next level. These 3D objects are further joined to form the final template model in the last level. After the point clouds are partitioned into several geometric sets, the constructed template model is used to fit them. The point cloud fitting is archived by solving a least-square linear system instead of solving a non-linear one, making the approach efficient and robust in the modeling. The experimental results show that the approach is better, in terms of sharp feature fitting and noise withstanding, than the approaches based on implicit surfaces. In addition, comparing to the general least-square fitting approaches, the template-based fitting with geometry constraints improves modeling quality with respect to human visual system