NCU-IISR/AS-GIS: Results of various pre-trained biomedical language models and linear regression model in BioASQ task 9b Phase B

Yu Zhang; Jen Chieh Han; Richard Tzong Han Tsai

NCU-IISR/AS-GIS: Results of various pre-trained biomedical language models and linear regression model in BioASQ task 9b Phase B

Yu Zhang, Jen Chieh Han, Richard Tzong Han Tsai

Department of Computer Science and Information Engineering

Research output: Contribution to journal › Conference article › peer-review

2 Scopus citations

Abstract

Transformer has been widely applied in Natural Language Processing (NLP) field, and it also results in an amount of pre-trained language models like BioBERT, SciBERT, NCBI_Bluebert, and PubMedBERT. In this paper, we introduce our system for the BioASQ Task 9b Phase B. We employed various pre-trained biomedical language models, including BioBERT, BioBERT-MNLI, and PubMedBERT, to generate “exact” answers for the questions, and a linear regression model with our sentence embedding to construct the top-n sentences as a prediction for “ideal” answers.

Original language	English
Pages (from-to)	360-368
Number of pages	9
Journal	CEUR Workshop Proceedings
Volume	2936
State	Published - 2021
Event	2021 Working Notes of CLEF - Conference and Labs of the Evaluation Forum, CLEF-WN 2021 - Virtual, Bucharest, Romania Duration: 21 Sep 2021 → 24 Sep 2021

Keywords

Biomedical question answering
Linear regression
Pre-trained language model

Clinical Question and Answer System for Medication Recommendation and Complication Summary(1/3)
Tsai, T.-H. (PI)
1/08/20 → 31/07/21
Project: Research

Cite this

@article{2ff4a56169c047f6aaebaaa24685f36f,

title = "NCU-IISR/AS-GIS: Results of various pre-trained biomedical language models and linear regression model in BioASQ task 9b Phase B",

abstract = "Transformer has been widely applied in Natural Language Processing (NLP) field, and it also results in an amount of pre-trained language models like BioBERT, SciBERT, NCBI_Bluebert, and PubMedBERT. In this paper, we introduce our system for the BioASQ Task 9b Phase B. We employed various pre-trained biomedical language models, including BioBERT, BioBERT-MNLI, and PubMedBERT, to generate “exact” answers for the questions, and a linear regression model with our sentence embedding to construct the top-n sentences as a prediction for “ideal” answers.",

keywords = "Biomedical question answering, Linear regression, Pre-trained language model",

author = "Yu Zhang and Han, {Jen Chieh} and Tsai, {Richard Tzong Han}",

note = "Publisher Copyright: {\textcopyright} 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).; 2021 Working Notes of CLEF - Conference and Labs of the Evaluation Forum, CLEF-WN 2021 ; Conference date: 21-09-2021 Through 24-09-2021",

year = "2021",

language = "???core.languages.en_GB???",

volume = "2936",

pages = "360--368",

journal = "CEUR Workshop Proceedings",

issn = "1613-0073",

}

TY - JOUR

T1 - NCU-IISR/AS-GIS

T2 - 2021 Working Notes of CLEF - Conference and Labs of the Evaluation Forum, CLEF-WN 2021

AU - Zhang, Yu

AU - Han, Jen Chieh

AU - Tsai, Richard Tzong Han

PY - 2021

Y1 - 2021

N2 - Transformer has been widely applied in Natural Language Processing (NLP) field, and it also results in an amount of pre-trained language models like BioBERT, SciBERT, NCBI_Bluebert, and PubMedBERT. In this paper, we introduce our system for the BioASQ Task 9b Phase B. We employed various pre-trained biomedical language models, including BioBERT, BioBERT-MNLI, and PubMedBERT, to generate “exact” answers for the questions, and a linear regression model with our sentence embedding to construct the top-n sentences as a prediction for “ideal” answers.

AB - Transformer has been widely applied in Natural Language Processing (NLP) field, and it also results in an amount of pre-trained language models like BioBERT, SciBERT, NCBI_Bluebert, and PubMedBERT. In this paper, we introduce our system for the BioASQ Task 9b Phase B. We employed various pre-trained biomedical language models, including BioBERT, BioBERT-MNLI, and PubMedBERT, to generate “exact” answers for the questions, and a linear regression model with our sentence embedding to construct the top-n sentences as a prediction for “ideal” answers.

KW - Biomedical question answering

KW - Linear regression

KW - Pre-trained language model

UR - http://www.scopus.com/inward/record.url?scp=85113512633&partnerID=8YFLogxK

M3 - 會議論文

AN - SCOPUS:85113512633

SN - 1613-0073

VL - 2936

SP - 360

EP - 368

JO - CEUR Workshop Proceedings

JF - CEUR Workshop Proceedings

Y2 - 21 September 2021 through 24 September 2021

ER -

NCU-IISR/AS-GIS: Results of various pre-trained biomedical language models and linear regression model in BioASQ task 9b Phase B

Abstract

Keywords

Fingerprint

Projects

Clinical Question and Answer System for Medication Recommendation and Complication Summary(1/3)

Cite this