Multi-lens multispectral camera system can be mounted on an Unmanned Aerial System (UAS) for high resolution remote sensing data acquisition. The adopted multi-lens structure allows it to acquire multi-band images with each independent camera and specific filter. However, this lead to significant band misregistration effects as the perspective centers, viewing angles, and lens distortion effects differ to each other. In this study, a Robust and Adaptive Band-to-Band Image Transform (RABBIT) is proposed to solve the band misregistration issues of multi-lens multispectral camera. The RABBIT utilizes a modified projective transform for transferring the multi-sensor geometry into one sensor geometry, in which all the necessary coefficients are derived from camera system calibration. Considering the coefficients uncertainty during the calibration, a simulation procedure is conducted to understand the systematic effects, and a robust and adaptive correction is thus developed to correct the various systematic errors. Several datasets were collected from three state-of-the-art multi-lens multispectral camera system for the accuracy analysis and reliability evaluation. The experiments show that RABBIT can achieve 0.2-0.4 pixels band co-registration accuracy, better than the other proposed method, and is robust to different cameras, different camera calibration approaches, and different flight missions.