New Horizons of Legal Judgement Predication via Multi-Task Learning and LoRA

Legal Judgment Prediction (LJP) aims to predict the judgement results (such as legal article, charge and penalty) based on the criminal facts of the case. Most previous research in this field was based on criminal statements from court verdicts. However, each verdict actually is based on the content from indictments. For prosecutors, will the case be dismissed or processed? If the case is accepted, is the penalty a jail sentence or a fine? What is the charge and article violated? In this study, we therefore define three novel LJP tasks for prosecutors, including prosecution outcome prediction (LJP#1), imprison prediction (LJP#2) and fine prediction (LJP#3). We explore various multi-task learning (MTL) framework based on Word2Vec and BERT language model (LM) with either topology-based or message-passing mechanism. Moreover, we employed the LoRA (Low-Rank Adaptation) technique to save both computation time and resources during fine-tuning. Experimental results demonstrated that Word2Vec-based model combined with message passing architecture still has the potential to outperform large LM like BERT, while BERT-based models with a simple parallel architecture generally performed well. Finally, using LoRA for fine-tuning not only reduced training time (by 45%) but also improved performance (2.5% F1) in some LJP tasks.