Background: Computer-aided diagnosis (CADx) software that provides a second opinion has been widely used to assist physicians with various tasks. In dermatology, however, CADx has been mostly limited to melanoma or melanocytic skin cancer diagnosis. The frequency of non-melanocytic skin cancers and the accessibility of regular digital macrographs have raised interest in developing CADx for broader applications. Objectives: To investigate the feasibility of using CADx to diagnose both melanocytic and non-melanocytic skin lesions based on conventional digital photographic images. Methods: This study was approved by an institutional review board, and the requirement to obtain informed consent was waived. In total, 769 conventional photographs of melanocytic and non-melanocytic skin lesions were retrospectively reviewed and used to develop a CADx system. Conventional and new color-related image features were developed to classify the lesions as benign or malignant using support vector machines (SVMs). The performance of CADx was compared with that of dermatologists. Results: The clinicians' overall sensitivity, specificity, and accuracy were 83.33%, 85.88%, and 85.31%, respectively. New color correlation and principal component analysis (PCA) features improved the classification ability of the baseline CADx (p = 0.001). The estimated area under the receiver operating characteristic (ROC) curve (Az) of the proposed CADx system was 0.949, with a sensitivity and specificity of 85.63% and 87.65%, respectively, and a maximum accuracy of 90.64%. Conclusions: We have developed an effective CADx system to classify both melanocytic and non-melanocytic skin lesions using conventional digital macrographs. The system's performance was similar to that of dermatologists at our institute. Through improved feature extraction and SVM analysis, we found that conventional digital macrographs were feasible for providing useful information for CADx applications. The new color-related features significantly improved CADx applications for skin cancer.