As adaptive neuro-fuzzy inference system (ANFIS) has shown a high capability in solving various complicated problems, its usage has been increased so far. LiDAR is an active sensor which operates based on measuring distance by sending pulses to the ground and receiving the backscatters. This technology gives the 3D position of a point directly. Because of using millimeter level laser ranging accuracy, LiDAR is highly accurate. Dense point clouds of LiDAR can be directly used in simple applications, but the full manipulation of the LiDAR potentials and capabilities needs new methods and researches that differ from those in traditional Photogrammetry. The main output data of LiDAR are point clouds. Each point has two range and two intensity values for both the first and the last pulses. In some areas where there are some trees, the values for the first and the last pulses may differ, in which the first pulse data includes upper surfaces of trees, whereas the last pulse data includes lower surfaces, mainly ground. ANFIS is able to deal with large amounts of data with linear or nonlinear relations. In our study, the combination of digital aerial images and LiDAR data were used for the first time to probe the capabilities of the ANFIS as a classifier. The fact of non-linearity and ambiguity of this combination makes this challenge so hard. The main goal of this research is to detect buildings in city scenes from digital aerial images and LiDAR data using the ANFIS. In this regard, a genetic algorithm is run for feature selection. Four features were selected by genetic algorithm. These features were generated as ANFIS inputs including Green band, normalized difference vegetation index (NDVI), and normalized digital surface model (nDSM) using two different algorithms via morphological operations. The proposed ANFIS used three different algorithms to build its fuzzy inference system structure including grid partition, subtractive clustering, and fuzzy c-means clustering. Also, as there are many methods in building and tree detection as mentioned before, the main question is which of them is better among the others? This is not an easy question to answer because these methods are not evaluated over a unique data-set. To overcome this problem, fourth working group of third commission (WG III/4) in the international society of photogrammetry and remote sensing (ISPRS) has provided some benchmark data-sets, and has encouraged all researchers around the world to evaluate their methods on these data-sets. The results were evaluated on three different test areas, known as Areas 1, 2, and 3. The achieved results were compared with each other, as well as with ISPRS WG III/4 participants’ results, by considering Completeness, Correctness, Quality, and RMS indices per-area and per-object levels. The achieved results demonstrated the capability of the proposed ANFIS in detecting buildings in complex city scenes in comparison with other methods. Although there are some typical errors among participants’ results, most of these errors have resolved in ANFIS-base approaches. Proposed ANFIS-based methods achieved Completeness of 100% in all three test areas for buildings larger than 50 m2.