This work proposes a novel approach to separate books which are placed disorderly on a bookshelf. This work applies grayscale morphological reconstruction and Lo gradient minimization to erase texts on books while preserving book boundaries. Afterwards, Hough transform and the proposed SVM-based recovery scheme are applied to find book boundaries accurately. The proposed SVM-based recovery scheme is able to recover the book boundaries missed by Hough transform without incurring noticeable false alarm detections. Experiments demonstrate that the proposed method is able to segment books which are placed in various orientations.