Point of interest (POI) is one of the important applications of location-based social networks (LBSNs) for users and business managers. LBSNs include various complex relations (i.e., POI-POI, user-user, user-POI, and so on), that more accurate modeling of them can lead to making a better recommendation. Since some relations are much more sophisticated than pairwise relations, and thus cannot be simply modeled by a graph. This study proposes a model for calculating the similarity of POIs and users based on hypergraph structure and by integrating that into the collaborative filtering (CF) method it can improve the recommendation performance. The results obtained from the real data set, Foursquare, show that the proposed model performs better than state-of-the-art methods in terms of accuracy. Taking high-order relations between POIs and users into account can improve recommendation performance by 2.7% in terms of accuracy. By integrating the proposed similarity learning into the collaborative filtering (CF) method, our method obtained approximately 33% improvements in accuracy compared to the traditional similarity learning methods.