This study proposed a novel methodology for reducing the vibration of a high-speed spur gear pair (up to 30,000 rpm) by performing multi-objective optimization of the peak-to-peak loaded transmission error (PPLTE) under three assembly conditions. The optimum tip relief parameters for the high-speed spur gear set were studied by considering the practical manufacturing tolerances on the gear shaft and bearing bores. First, the PPLTE under three assembly conditions was determined by loaded tooth contact analysis. Multi-objective optimization with a genetic algorithm was used to determine the optimum linear tip relief parameters, minimizing the PPLTE under three assembly conditions by 70%. Moreover, the dynamic characteristics of the original and optimum designs were simulated and compared under ideal conditions as well as axial misalignments. The optimum design exhibited a >50% reduction in the peak RMS of acceleration at the natural frequency. Finally, dynamic experiments were performed and the RMS values of the acceleration at various speeds were computed for comparison. The results from both dynamic simulation and experiment indicated that the optimum design exhibits superior dynamic characteristics to the original design.