每年專案

## 摘要

This paper studies overparameterization on the matrix factorization (MF) model. We confirm that overparameterization can significantly accelerate the optimization of MF with no change in the expressiveness of the learning model. Consequently, modern applications on recommendations based on MF or its variants can largely benefit from our discovery. Specifically, we theoretically derive that applying the vanilla stochastic gradient descent (SGD) on the overparameterized MF model is equivalent to employing gradient descent with momentum and adaptive learning rate on the standard MF model. We empirically compare the overparameterized MF model with the standard MF model based on various optimizers, including vanilla SGD, AdaGrad, Adadelta, RMSprop, and Adam, using several public datasets. The experimental results comply with our analysis - overparameterization converges faster. The overparameterization technique can be applied to various learning-based recommendation models, including deep learning-based recommendation models, e.g., SVD++, nonnegative matrix factorization (NMF), factorization machine (FM), NeuralCF, Wide&Deep, and DeepFM. Therefore, we suggest utilizing the overparameterization technique to accelerate the training speed for the learning-based recommendation models whenever possible, especially when the size of the training dataset is large.

原文 | ???core.languages.en_GB??? |
---|---|

主出版物標題 | DeLTA 2020 - Proceedings of the 1st International Conference on Deep Learning Theory and Applications |

編輯 | Ana Fred, Kurosh Madani |

發行者 | SciTePress |

頁面 | 89-97 |

頁數 | 9 |

ISBN（電子） | 9789897584411 |

出版狀態 | 已出版 - 2020 |

事件 | 1st International Conference on Deep Learning Theory and Applications, DeLTA 2020 - Virtual, Online 持續時間: 8 7月 2020 → 10 7月 2020 |

### 出版系列

名字 | DeLTA 2020 - Proceedings of the 1st International Conference on Deep Learning Theory and Applications |
---|

### ???event.eventtypes.event.conference???

???event.eventtypes.event.conference??? | 1st International Conference on Deep Learning Theory and Applications, DeLTA 2020 |
---|---|

城市 | Virtual, Online |

期間 | 8/07/20 → 10/07/20 |

## 指紋

深入研究「Accelerating matrix factorization by overparameterization」主題。共同形成了獨特的指紋。## 專案

- 1 已完成