This article presents an online light detection and ranging (LiDAR)-only person-following (LoPF) framework, which can be applied in both indoor and outdoor environments with only sparse LiDAR point cloud data as input. The framework consists of two cascade backbone procedures. The former is pedestrian detection, which is developed for classifying a set of human clusters from noisy and sparse LiDAR point clouds. An originally designed voxel feature is integrated for depicting the 3-D information of each potential cluster. A U-shaped convolutional neural network (CNN) is proposed to achieve human cluster detection. Then the latter procedure, pedestrian tracking, is developed to follow the target. Specifically, to achieve long-term person-following, an unscented Kalman filter (UKF), a position filter, and a target reidentification module are combined. The reidentification module contains a specifically designed support vector machine (SVM) target classifier. The developed framework is verified by extensive real-world experiments in outdoor environments using mobile robots. Compared with the state of the art, our method is better in target identification and tracking, realizing better performance in the person-following.