Millimeter wave (mmWave) communication and multiple-input multiple-output (MIMO) are two important technologies for future communication systems. Hybrid precoding and combining have been used to reduce the cost and the power consumption of traditional full digital precoding and combining for mmWave MIMO systems. In this paper, we consider a fully-connected hybrid precoding architecture with finite resolution phase shifters in the design. Basically, we first obtain the phases of optimal precoder and combiner, which are obtained through the singular value decomposition (SVD) operation of channel vectors. Then, these phases are used to construct the RF precoder and combiner. After the RF precoder and combiner are achieved, the digital precoder and combiner can be calculated by the least squares solutions. The simulation results show that the proposed algorithms improve the capacity and reduce the complexity compared with the existing orthogonal matching pursuit (OMP)-based algorithm. Besides, an adaptive solution is further proposed to save computational time with competitive performance. With the adaptive solution, the full SVD process is not required for the weighting coefficient updating at every time slot.