Abstract: To address the stray light issue in traditional coaxial lidar systems caused by the reflection and refraction of high-energy emitted laser light within the cavity, a compact coaxial lidar optical system is designed. The coaxial LiDAR system consists of a transmitting subsystem and a receiving subsystem. The transmitting subsystem includes a fiber laser, a collimating lens group, and a beam-expanding lens group. The receiving subsystem comprises a beam-reducing lens group, a focusing lens group, and an APD single-photon detector. The transmitting and receiving subsystems share a single lens.Methods for eliminating ghost images in the lidar system are described, and the influence of ghost images is mitigated by optimizing and improving the optomechanical structure. The design results show that the system has an angular resolution of 0.01°, a field of view of ±3°, and the light blocking rate of the coaxial lens group on the receiving subsystem is 12%. The designed LiDAR system features high-resolution and refined sensing capabilities, and its coaxial architecture ensures the advantage of miniaturization, providing an optimization method for solving ghost image problems caused by the transmitting subsystem.