Two recent and fully open source COSMO-SAC models are assessed for the first time on the basis of very large experimental data sets. The model performance of COSMO-SAC 2010 and COSMO-SAC-dsp (2013) is studied for vapor-liquid equilibrium (VLE) and infinite dilution activity coefficient (γi∞) predictions, and it is benchmarked with respect to the group contribution models UNIFAC and mod. UNIFAC(DO). For this purpose, binary mixture combinations of 2295 components are investigated. This leads to 10897 γi∞ and 6940 VLE mixtures, which correspond to 29173 γi∞ and 139921 VLE data points. The model performance is analyzed in terms of chemical families. A MATLAB program is provided for the interested reader to study the models in detail. The comprehensive assessment shows that there is a clear improvement from COSMO-SAC 2010 to COSMO-SAC-dsp and from UNIFAC to mod. UNIFAC(DO). The mean absolute deviation of γi∞ predictions is reduced from 95% to 86% (COSMO-SAC 2010 to COSMO-SAC-dsp) and from 73% to 58% (UNIFAC to mod. UNIFAC(DO)). A combined mean absolute deviation is introduced to study the temperature, pressure, and vapor mole fraction errors of VLE predictions, and it is reduced from 4.77% to 4.63% (COSMO-SAC 2010 to COSMO-SAC-dsp) and from 4.47% to 3.51% (UNIFAC to mod. UNIFAC(DO)). Detailed error analyses show that the accuracy of COSMO-SAC models mainly depends on chemical family types, but not on the molecular size asymmetry or polarity. The present results may serve as a reference for the reliability of predictions with COSMO-SAC methods and provide direction for future developments.