IGRP: Iterative Gradient Rank Pruning for Finding Graph Lottery Ticket

Po Wei Harn; Sai Deepthi Yeddula; Bo Hui; Jie Zhang; Libo Sun; Min Te Sun; Wei Shinn Ku

doi:10.1109/BigData55660.2022.10020964

IGRP: Iterative Gradient Rank Pruning for Finding Graph Lottery Ticket

Po Wei Harn, Sai Deepthi Yeddula, Bo Hui, Jie Zhang, Libo Sun, Min Te Sun, Wei Shinn Ku

Department of Computer Science and Information Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Graph Neural Networks (GNNs) have shown promising performance in many applications, yet remain extremely difficult to train over large-scale graph datasets. Existing weight pruning techniques can prune out the layer weights; however, they cannot fully address the high computation complexity of GNN inference, caused by large graph size and complicated node connections. In this paper, we propose an Iterative Gradient Rank Pruning (IGRP) algorithm to find graph lottery tickets (GLT) of GNNs where each GLT includes a pruned adjacency matrix and a sub-network. Our IGRP can avoid layer collapse and the winning ticket achieves Maximal critical compression. We evaluate the proposed method on small-scale (Cora and Citeseer), medium-scale (PubMed and Wiki-CS), and large-scale (Ogbn-ArXiv and Ogbn-Products) graph datasets. We demonstrate that both Single-shot and Multi-shot of IGRP outperform the state-of-the-art unified GNN sparsification (UGS) framework on node classification. The source code can be found in https://github.com/poweiharn/IGRP_GNN.

Original language	English
Title of host publication	Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022
Editors	Shusaku Tsumoto, Yukio Ohsawa, Lei Chen, Dirk Van den Poel, Xiaohua Hu, Yoichi Motomura, Takuya Takagi, Lingfei Wu, Ying Xie, Akihiro Abe, Vijay Raghavan
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	931-941
Number of pages	11
ISBN (Electronic)	9781665480451
DOIs	https://doi.org/10.1109/BigData55660.2022.10020964
State	Published - 2022
Event	2022 IEEE International Conference on Big Data, Big Data 2022 - Osaka, Japan Duration: 17 Dec 2022 → 20 Dec 2022

Publication series

Name	Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022

Conference

Conference	2022 IEEE International Conference on Big Data, Big Data 2022
Country/Territory	Japan
City	Osaka
Period	17/12/22 → 20/12/22

Keywords

Graph Neural Networks
Lottery Ticket Hypothesis
Network Pruning

Access to Document

10.1109/BigData55660.2022.10020964

Cite this

Harn, P. W., Yeddula, S. D., Hui, B., Zhang, J., Sun, L., Sun, M. T., & Ku, W. S. (2022). IGRP: Iterative Gradient Rank Pruning for Finding Graph Lottery Ticket. In S. Tsumoto, Y. Ohsawa, L. Chen, D. Van den Poel, X. Hu, Y. Motomura, T. Takagi, L. Wu, Y. Xie, A. Abe, & V. Raghavan (Eds.), Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022 (pp. 931-941). (Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData55660.2022.10020964

Harn, Po Wei ; Yeddula, Sai Deepthi ; Hui, Bo et al. / IGRP : Iterative Gradient Rank Pruning for Finding Graph Lottery Ticket. Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022. editor / Shusaku Tsumoto ; Yukio Ohsawa ; Lei Chen ; Dirk Van den Poel ; Xiaohua Hu ; Yoichi Motomura ; Takuya Takagi ; Lingfei Wu ; Ying Xie ; Akihiro Abe ; Vijay Raghavan. Institute of Electrical and Electronics Engineers Inc., 2022. pp. 931-941 (Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022).

@inproceedings{a7207dd6a3b04f7da22eb0fd291e86aa,

title = "IGRP: Iterative Gradient Rank Pruning for Finding Graph Lottery Ticket",

abstract = "Graph Neural Networks (GNNs) have shown promising performance in many applications, yet remain extremely difficult to train over large-scale graph datasets. Existing weight pruning techniques can prune out the layer weights; however, they cannot fully address the high computation complexity of GNN inference, caused by large graph size and complicated node connections. In this paper, we propose an Iterative Gradient Rank Pruning (IGRP) algorithm to find graph lottery tickets (GLT) of GNNs where each GLT includes a pruned adjacency matrix and a sub-network. Our IGRP can avoid layer collapse and the winning ticket achieves Maximal critical compression. We evaluate the proposed method on small-scale (Cora and Citeseer), medium-scale (PubMed and Wiki-CS), and large-scale (Ogbn-ArXiv and Ogbn-Products) graph datasets. We demonstrate that both Single-shot and Multi-shot of IGRP outperform the state-of-the-art unified GNN sparsification (UGS) framework on node classification. The source code can be found in https://github.com/poweiharn/IGRP_GNN.",

keywords = "Graph Neural Networks, Lottery Ticket Hypothesis, Network Pruning",

author = "Harn, {Po Wei} and Yeddula, {Sai Deepthi} and Bo Hui and Jie Zhang and Libo Sun and Sun, {Min Te} and Ku, {Wei Shinn}",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 IEEE International Conference on Big Data, Big Data 2022 ; Conference date: 17-12-2022 Through 20-12-2022",

year = "2022",

doi = "10.1109/BigData55660.2022.10020964",

language = "???core.languages.en_GB???",

series = "Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "931--941",

editor = "Shusaku Tsumoto and Yukio Ohsawa and Lei Chen and {Van den Poel}, Dirk and Xiaohua Hu and Yoichi Motomura and Takuya Takagi and Lingfei Wu and Ying Xie and Akihiro Abe and Vijay Raghavan",

booktitle = "Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022",

}

Harn, PW, Yeddula, SD, Hui, B, Zhang, J, Sun, L, Sun, MT & Ku, WS 2022, IGRP: Iterative Gradient Rank Pruning for Finding Graph Lottery Ticket. in S Tsumoto, Y Ohsawa, L Chen, D Van den Poel, X Hu, Y Motomura, T Takagi, L Wu, Y Xie, A Abe & V Raghavan (eds), Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022. Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022, Institute of Electrical and Electronics Engineers Inc., pp. 931-941, 2022 IEEE International Conference on Big Data, Big Data 2022, Osaka, Japan, 17/12/22. https://doi.org/10.1109/BigData55660.2022.10020964

IGRP: Iterative Gradient Rank Pruning for Finding Graph Lottery Ticket. / Harn, Po Wei; Yeddula, Sai Deepthi; Hui, Bo et al.
Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022. ed. / Shusaku Tsumoto; Yukio Ohsawa; Lei Chen; Dirk Van den Poel; Xiaohua Hu; Yoichi Motomura; Takuya Takagi; Lingfei Wu; Ying Xie; Akihiro Abe; Vijay Raghavan. Institute of Electrical and Electronics Engineers Inc., 2022. p. 931-941 (Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - IGRP

T2 - 2022 IEEE International Conference on Big Data, Big Data 2022

AU - Harn, Po Wei

AU - Yeddula, Sai Deepthi

AU - Hui, Bo

AU - Zhang, Jie

AU - Sun, Libo

AU - Sun, Min Te

AU - Ku, Wei Shinn

PY - 2022

Y1 - 2022

N2 - Graph Neural Networks (GNNs) have shown promising performance in many applications, yet remain extremely difficult to train over large-scale graph datasets. Existing weight pruning techniques can prune out the layer weights; however, they cannot fully address the high computation complexity of GNN inference, caused by large graph size and complicated node connections. In this paper, we propose an Iterative Gradient Rank Pruning (IGRP) algorithm to find graph lottery tickets (GLT) of GNNs where each GLT includes a pruned adjacency matrix and a sub-network. Our IGRP can avoid layer collapse and the winning ticket achieves Maximal critical compression. We evaluate the proposed method on small-scale (Cora and Citeseer), medium-scale (PubMed and Wiki-CS), and large-scale (Ogbn-ArXiv and Ogbn-Products) graph datasets. We demonstrate that both Single-shot and Multi-shot of IGRP outperform the state-of-the-art unified GNN sparsification (UGS) framework on node classification. The source code can be found in https://github.com/poweiharn/IGRP_GNN.

AB - Graph Neural Networks (GNNs) have shown promising performance in many applications, yet remain extremely difficult to train over large-scale graph datasets. Existing weight pruning techniques can prune out the layer weights; however, they cannot fully address the high computation complexity of GNN inference, caused by large graph size and complicated node connections. In this paper, we propose an Iterative Gradient Rank Pruning (IGRP) algorithm to find graph lottery tickets (GLT) of GNNs where each GLT includes a pruned adjacency matrix and a sub-network. Our IGRP can avoid layer collapse and the winning ticket achieves Maximal critical compression. We evaluate the proposed method on small-scale (Cora and Citeseer), medium-scale (PubMed and Wiki-CS), and large-scale (Ogbn-ArXiv and Ogbn-Products) graph datasets. We demonstrate that both Single-shot and Multi-shot of IGRP outperform the state-of-the-art unified GNN sparsification (UGS) framework on node classification. The source code can be found in https://github.com/poweiharn/IGRP_GNN.

KW - Graph Neural Networks

KW - Lottery Ticket Hypothesis

KW - Network Pruning

UR - http://www.scopus.com/inward/record.url?scp=85147944613&partnerID=8YFLogxK

U2 - 10.1109/BigData55660.2022.10020964

DO - 10.1109/BigData55660.2022.10020964

M3 - 會議論文篇章

AN - SCOPUS:85147944613

T3 - Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022

SP - 931

EP - 941

BT - Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022

A2 - Tsumoto, Shusaku

A2 - Ohsawa, Yukio

A2 - Chen, Lei

A2 - Van den Poel, Dirk

A2 - Hu, Xiaohua

A2 - Motomura, Yoichi

A2 - Takagi, Takuya

A2 - Wu, Lingfei

A2 - Xie, Ying

A2 - Abe, Akihiro

A2 - Raghavan, Vijay

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 17 December 2022 through 20 December 2022

ER -

Harn PW, Yeddula SD, Hui B, Zhang J, Sun L, Sun MT et al. IGRP: Iterative Gradient Rank Pruning for Finding Graph Lottery Ticket. In Tsumoto S, Ohsawa Y, Chen L, Van den Poel D, Hu X, Motomura Y, Takagi T, Wu L, Xie Y, Abe A, Raghavan V, editors, Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022. Institute of Electrical and Electronics Engineers Inc. 2022. p. 931-941. (Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022). doi: 10.1109/BigData55660.2022.10020964

IGRP: Iterative Gradient Rank Pruning for Finding Graph Lottery Ticket

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this