文献类型：期刊文献

英文题名：A BERT-BiLSTM-based categorical representation learning for mixed-attribute data

作者：Li, Qiude Pan, Yinglong Ji, Shengfen Hu, Sigui Yu, Yang Pan, Zhongwen

第一作者：Li, Qiude

通信作者：Ji, SF[1]

机构：[1]Guizhou Med Univ, Sch Biol & Engn, Guiyang 561113, Guizhou, Peoples R China;[2]Guizhou Univ, Sch Math & Stat, Guiyang 550025, Guizhou, Peoples R China;[3]Guizhou Inst Technol, Sch Foreign Language, Guiyang 550003, Guizhou, Peoples R China;[4]Hunan Univ Finance & Econ, Sch Econ, Changsha 410205, Hunan, Peoples R China

第一机构：Guizhou Med Univ, Sch Biol & Engn, Guiyang 561113, Guizhou, Peoples R China

通信机构：corresponding author), Guizhou Inst Technol, Sch Foreign Language, Guiyang 550003, Guizhou, Peoples R China.|贵州理工学院;

年份：2026

卷号：82

期号：4

外文期刊名：JOURNAL OF SUPERCOMPUTING

收录：;EI(收录号：20261320352359);WOS:【SCI-EXPANDED(收录号:WOS:001712656100001)】；

基金：We thank anonymous reviewers for their valuable comments and suggestions. The work was supported by the National Natural Science Foundation of China (Grant No. 62166009), Guizhou Provincial Basic Research Program (Natural Science) (Grant No. MS[2025]553, QKHJC-[2024]Youth 226 and zk[2022]350), Cultivation Project of National Natural Science Foundation of China at Guizhou Medical University (No. 25NSFCP20), the Science and Technology Foundation of Guizhou Provincial Health Commission (Grant No. gzwkj2023-258, 2025GZWJKJXM0914), the Youth Science and Technology Project of Guizhou Provincial Education Department (No. QJJ-[2024] 101), and the Medical Research Union Foundation for High-quality health development of Guizhou Province (No. 2024GZYXKYJJXM0119).

语种：英文

外文关键词：Mixed-attribute data; Representation learning; BERT; Feature extraction; Deep learning

摘要：It is a significant yet challenging task to represent categorical attributes in mixed-attribute data as low-dimensional compact numerical vectors through a two-stage categorical representation learning (i.e., feature extraction and representation learning). Most existing categorical representation learning methods either extract features from a single view or rely on the shallow structure-based representation learning. The former fails to fully capture the feature information of categorical data, while the latter cannot learn deep-level feature representations like deep learning. To address these issues, this paper proposes a BERT-BiLSTM-based Categorical Representation learning method (BBCR) for the mixed-attribute data classification. To be more precise, in the feature extraction stage, the intrinsic features of categorical values are comprehensively extracted from multiple different views, including the Intra-attribute, Inter-attribute, and Attribute-Class. In this stage, the problems of redundancy, noise, and high-dimensional features in the Inter-attribute view are alleviated by using the conditional information entropy extraction method and the symmetric uncertainty dimension reduction method. In the representation learning stage, inspired by the text representation technology in the natural language processing, we introduce the BERT-based deep learning to perform categorical representation learning on the multi-view features obtained in the above stage (i.e., the feature extraction stage). Additionally, BiLSTM is employed to enhance the interaction between long-distance categorical values, ultimately representing categorical values as low-dimensional compact numerical vectors. The representation learning stage of BBCR adopts an end-to-end deep representation learning architecture. Although this incurs certain computational overhead, its powerful representation capabilities make it particularly suitable for high-dimensional, large-scale, and complex mixed-attribute data scenarios, highlighting the inherent demand of this method for high-performance computing (HPC) resources. Extensive experiments on 26 mixed-attribute datasets with diverse characteristics demonstrate that BBCR significantly improves the representation performance compared to state-of-the-art baseline methods.