Comparison and application of sequence-to-text based language models†
Minkyung Kim1 · Yumi Kim2 · Jihyeon Min3 · Hyeonsu Seong4 · Yoonyoung Cho5 · Yoonsuh Jung6
12346 Department of Statistics, Korea University
5Kakao Games Corporation
5Kakao Games Corporation
Correspondence to: † Jung’s work has been partially supported by National Research Foundation of Korea (NRF) grants funded by the Korean government (MIST)(No. 2022R1F1A1071126 and No. 2022M3J6A1063595).
The contributions of the students are equal.
1 Graduate student, Department of Statistics, Korea University, Seoul 02841, Korea.
2 Graduate student, Department of Statistics, Korea University, Seoul 02841, Korea.
3 Graduate student, Department of Statistics, Korea University, Seoul 02841, Korea.
4 Graduate student, Department of Statistics, Korea University, Seoul 02841, Korea.
5 Data Analytics Labs, Kakao Games Corporation, Gyeonggi 13529, Korea.
6 Corresponding author: Professor, Department of Statistics, Korea University, Seoul 02841, Korea. E-mail: yoons77@korea.ac.kr
The contributions of the students are equal.
1 Graduate student, Department of Statistics, Korea University, Seoul 02841, Korea.
2 Graduate student, Department of Statistics, Korea University, Seoul 02841, Korea.
3 Graduate student, Department of Statistics, Korea University, Seoul 02841, Korea.
4 Graduate student, Department of Statistics, Korea University, Seoul 02841, Korea.
5 Data Analytics Labs, Kakao Games Corporation, Gyeonggi 13529, Korea.
6 Corresponding author: Professor, Department of Statistics, Korea University, Seoul 02841, Korea. E-mail: yoons77@korea.ac.kr
Received September 23, 2024; Revised October 21, 2024; Accepted October 26, 2024.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
- Abstract
- Recently, the active development of language models has garnered significant interest from researchers across various fields. The applications of language models are exceedingly diverse, making them suitable for a multitude of domains. In traditional statistics and machine learning, extensive research has been conducted on binary classification, a task that language models can also perform effectively. Due to the inherent characteristics of language models, the input data required is not merely numerical in tabular form, but rather comprises textual variables. This paper presents an overview of the objectives and features of ten state-of-the-art language models. Additionally, various examples of sequence-to-text methods for transforming sequence data into textual formats are detailed to assist readers who may lack experience in implementing language models. Finally, through a series of simulations utilizing actual data, this study compares the performance of the aforementioned modern language models across several sequence-to-text methods, while also providing results from machine learning models to demonstrate the superior classification performance of language models.
- Keywords : Attention, language model, sequence classification, transformer
Full Text(PDF) Free
