Federated Knowledge-Augmented Generation
ZJU-UIUC Institute, China
Advisor: Qiang Zhang (ZJUI Professor)
Jun 2024 - Current
- Motivation: The goal is to develop a personalized and privacy-preserving text generation systems based on large langauge models. Federated Learning enables training machine learning models across multiple devices or servers without sharing raw data. Knowledge-Augmented Generation (KAG) combines retrieval and generation components to enhance text generation tasks.
Teaching Large Language Models to Handle the Composition of Multiple Problems Simultaneously
Remote
Advisor: May Fung (HKUST Professor)
Jun 2024 - Current
- Motivation: Current evaluation of LLM hallucination only focus on single problem setting. Because of this, we investigate how LLM perform and deal with hallucination under multiple problem setting, where it need to response to multiple questions simultaneously.
- Result: We propose a novel fine-tune method called Multiple Answers and Confidence Stepwise Tuning (MAC-Tuning) with up to 12% improvement comparing with baseline and up to 40\% improvement comparing with Zero-shot model under multiple problem setting.
- Contribution: I conduct the entire process of data collection and building the project code. At the same time, I tested various approaches, like LLM-Judge and keyword extraction, to assess the accuracy of LLM-generated outputs. Furthermore, I experimented diverse evaluation metrics including accuracy, AP score and MAP to comprehensively evaluate model performance.
Random Augmentations Cheaply Break LLM Safety Alignment
UIUC, America
Advisor: Gagandeep Singh (UIUC Professor), Jason Vega (UIUC Ph.D)
Jun 2024 - Dec 2024
- Motivation: Current jailbreak methods are rather costly or involve a non-trivial amount of creativity and effort. Since that, we investigate how simple random augmentations to the input prompt affect safety alignment effectiveness in LLMs from different dimensions, including augmentation type, model size, quantization, fine-tuning-based defenses and decoding strategies.
- Result: We show that low-resource and unsophisticated attackers can significantly improve their chances of bypassing alignment with just 25 random augmentations per prompt.
- Contribution: I research and implement different simple data augmentations, including string level and character level. At the same time, I help to check the evaluation metric of the project and do case study as well as labeling the experimental result manually to see how LLM classification align with human evaluation.
LLM Attack Based on Gradient Method
UIUC, America
Advisor: Gagandeep Singh (UIUC Professor), Jason Vega (UIUC Ph.D)
Jan 2024 - Jun 2024
- Motivation: If we can decide the very first output part of LLM generation (which is "prefilling attack"), we can easily bypass the safety training of LLMs. One of the easiest way to do so is utilizing the Greedy Coordinate Gradient (GCG) attack to find the "ignore string" to ignore the "ending token" that separates the input prompt and LLM generation. Also, we can briefly give an explanation to the random string that is generated by gradient method, compared with random token in GCG attack.
- Result: We attack LLaMA2-7B and LLaMA2-13B with 97% attack successful rate.
- Contribution: I develop the code to find "ignore string" based on GCG attack and try different loss functions as well as different place to insert the string.
Galaxy Federated Learning Framework
ZJU, China
Advisor: Chao Wu (ZJU Professor)
Jun 2022 - Aug 2022
- Explore a possible method to obtain training data for machine learning from clients with privacy protections
- Help adding functions for monitoring the status of CPU and GPU while running the program
- Debug the data transporting process so that the model can be delivered and trained