For calibrating the laser plane to implement 3D shape measurement, an algorithm for extracting the laser stripe with sub-pixel accuracy is proposed. The proposed algorithm mainly consists of two stages: two-side edge detection and center line extraction. First, the two-side edge of laser stripe is detected using the principal component angle-based progressive probabilistic Hough transform and its width is calculated through the distance between these two edges. Secondly, the center line of laser strip is extracted with 2D Taylor expansion at a sub-pixel level and the laser plane is calibrated with the 3D reconstructed coordinates from the extracted 2D sub-pixel ones. Experimental results demonstrate that the proposed method can not only extract the laser stripe at a high speed, nearly average 78 ms/frame, but also calibrate the coplanar laser stripes at a low error, limited to 0.3 mm. The proposed algorithm can satisfy the system requirement of two-side edge detection and center line extraction, and rapid speed, high precision, as well as strong anti-jamming.