文献类型：期刊文献

英文题名：Gradient-guided layerwise adaptive noise injection for pre-trained language model fine-tuning

作者：Jiang, Qinglin Zeng, Cheng Chi, Nan

第一作者：Jiang, Qinglin

通信作者：Zeng, C[1];Zeng, C[2];Zeng, C[3];Zeng, C[4]|[1444027956eab4b44a396]曾诚;[14440313afbb7664c3122]曾诚;

机构：[1]Guizhou Univ, Math & Stat, Guiyang 550025, Guizhou, Peoples R China;[2]Guizhou Inst Technol, Coll Innovat & Entrepreneurship, Guiyang 550003, Guizhou, Peoples R China;[3]Guizhou Inst Technol, Coll Sci, Guiyang 550003, Guizhou, Peoples R China;[4]Key Lab New Power Syst Operat Control Guizhou Prov, Guiyang 550003, Guizhou, Peoples R China;[5]Special Key Lab Artificial Intelligence & Intellig, Guiyang 550003, Guizhou, Peoples R China

第一机构：Guizhou Univ, Math & Stat, Guiyang 550025, Guizhou, Peoples R China

通信机构：corresponding author), Guizhou Inst Technol, Coll Innovat & Entrepreneurship, Guiyang 550003, Guizhou, Peoples R China;corresponding author), Guizhou Inst Technol, Coll Sci, Guiyang 550003, Guizhou, Peoples R China;corresponding author), Key Lab New Power Syst Operat Control Guizhou Prov, Guiyang 550003, Guizhou, Peoples R China;corresponding author), Special Key Lab Artificial Intelligence & Intellig, Guiyang 550003, Guizhou, Peoples R China.|贵州理工学院理学院;贵州理工学院;

年份：2026

卷号：16

期号：1

外文期刊名：SCIENTIFIC REPORTS

收录：;Scopus(收录号：2-s2.0-105047425419);WOS:【SCI-EXPANDED(收录号:WOS:001851788800001)】；

基金：This work was supported in part by the National Natural Science Foundation of China under Grant 62163008 and 61763004, Qiankehe Platform ZSYS[2025]007, the Guizhou Provincial Science and Technology Projects [2020]1Z054, and PhD start-up fund of Guizhou Institute of Technology under Grant XJGC20150411.

语种：英文

外文关键词：Pre-trained language models; Adaptive regularization; Natural language processing

摘要：Fine-tuning pre-trained language models (PLMs) is crucial for achieving performance gains in downstream natural language processing tasks, but it is prone to overfitting in low-resource scenarios. Existing noise-based regularization methods typically rely on static parameter statistics or predefined heuristics to mitigate this issue. However, the noise magnitude and injection timing in these approaches depend on fixed hyperparameters or heuristic schedules, making them unable to adapt to the dynamic optimization state of the model during training. To address this limitation, this paper proposes a novel fine-tuning framework named gradient-guided noise injection (GNI). The core idea is to utilize gradient information generated during optimization to dynamically adjust the noise intensity in a layer-wise and real-time manner. Extensive experiments on multiple tasks from the GLUE and SuperGLUE benchmarks demonstrate that GNI generally improves average performance across different pre-trained models and task settings, while exhibiting stable generalizability and robustness. This work provides a simple yet effective solution for optimization-state aware dynamic regularization.