Logo image
Back
Buckle Up: Robustifying LLMs at Every Customization Stage via Data Curation
Preprint   Open access

Buckle Up: Robustifying LLMs at Every Customization Stage via Data Curation

Xiaoqun Liu, Jiacheng Liang, Luoxi Tang, Chenyu You, Muchao Ye and Zhaohan Xi
ArXiv.org
Cornell University
10/03/2024
DOI: 10.48550/arxiv.2410.02220
url
https://doi.org/10.48550/arxiv.2410.02220View
Preprint (Author's original)This preprint has not been evaluated by subject experts through peer review. Preprints may undergo extensive changes and/or become peer-reviewed journal articles. Open Access

Abstract

Large language models (LLMs) are extensively adapted for downstream applications through a process known as "customization," with fine-tuning being a common method for integrating domain-specific expertise. However, recent studies have revealed a vulnerability that tuning LLMs with malicious samples can compromise their robustness and amplify harmful content, an attack known as "jailbreaking." To mitigate such attack, we propose an effective defensive framework utilizing data curation to revise commonsense texts and enhance their safety implication from the perspective of LLMs. The curated texts can mitigate jailbreaking attacks at every stage of the customization process: before customization to immunize LLMs against future jailbreak attempts, during customization to neutralize jailbreaking risks, or after customization to restore the compromised models. Since the curated data strengthens LLMs through the standard fine-tuning workflow, we do not introduce additional modules during LLM inference, thereby preserving the original customization process. Experimental results demonstrate a substantial reduction in jailbreaking effects, with up to a 100% success in generating responsible responses. Notably, our method is effective even with commonsense texts, which are often more readily available than safety-relevant data. With the every-stage defensive framework and supporting experimental performance, this work represents a significant advancement in mitigating jailbreaking risks and ensuring the secure customization of LLMs.
Computer Science - Artificial Intelligence Computer Science - Cryptography and Security

Details

Metrics

23 Record Views
Logo image