- An understanding of machine learning fundamentals
- Experience working with large language models (LLMs)
- Familiarity with Python and PyTorch
Audience
- Data scientists
- AI engineers
Parameter-Efficient Fine-Tuning (PEFT) is a collection of techniques that enable efficient adaptation of large language models (LLMs) by modifying only a small subset of parameters.
This instructor-led, live training (online or onsite) is aimed at intermediate-level data scientists and AI engineers who wish to fine-tune large language models more affordably and efficiently using methods like LoRA, Adapter Tuning, and Prefix Tuning.
By the end of this training, participants will be able to:
- Understand the theory behind parameter-efficient fine-tuning approaches.
- Implement LoRA, Adapter Tuning, and Prefix Tuning using Hugging Face PEFT.
- Compare performance and cost trade-offs of PEFT methods vs. full fine-tuning.
- Deploy and scale fine-tuned LLMs with reduced compute and storage requirements.
Format of the Course
- Interactive lecture and discussion.
- Lots of exercises and practice.
- Hands-on implementation in a live-lab environment.
Course Customization Options
- To request a customized training for this course, please contact us to arrange.
Introduction to Parameter-Efficient Fine-Tuning (PEFT)
- Motivation and limitations of full fine-tuning
- Overview of PEFT: goals and benefits
- Applications and use cases in industry
LoRA (Low-Rank Adaptation)
- Concept and intuition behind LoRA
- Implementing LoRA using Hugging Face and PyTorch
- Hands-on: Fine-tuning a model with LoRA
Adapter Tuning
- How adapter modules work
- Integration with transformer-based models
- Hands-on: Applying Adapter Tuning to a transformer model
Prefix Tuning
- Using soft prompts for fine-tuning
- Strengths and limitations compared to LoRA and adapters
- Hands-on: Prefix Tuning on an LLM task
Evaluating and Comparing PEFT Methods
- Metrics for evaluating performance and efficiency
- Trade-offs in training speed, memory usage, and accuracy
- Benchmarking experiments and result interpretation
Deploying Fine-Tuned Models
- Saving and loading fine-tuned models
- Deployment considerations for PEFT-based models
- Integrating into applications and pipelines
Best Practices and Extensions
- Combining PEFT with quantization and distillation
- Use in low-resource and multilingual settings
- Future directions and active research areas
Summary and Next Steps
United Arab Emirates - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Qatar - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Egypt - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Saudi Arabia - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
South Africa - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Brasil - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Canada - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
中国 - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
香港 - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
澳門 - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
台灣 - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
USA - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Österreich - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Schweiz - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Deutschland - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Czech Republic - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Denmark - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Estonia - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Finland - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Greece - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Magyarország - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Ireland - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Luxembourg - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Latvia - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
España - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Italia - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Lithuania - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Nederland - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Norway - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Portugal - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
România - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Sverige - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Türkiye - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Malta - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Belgique - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
France - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
日本 - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Australia - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Malaysia - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
New Zealand - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Philippines - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Singapore - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Thailand - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Vietnam - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
India - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Argentina - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Chile - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Costa Rica - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Ecuador - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Guatemala - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Colombia - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
México - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Panama - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Peru - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Uruguay - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Venezuela - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Polska - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
United Kingdom - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
South Korea - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Pakistan - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Sri Lanka - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Bulgaria - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Bolivia - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Indonesia - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Kazakhstan - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Moldova - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Morocco - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Tunisia - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Kuwait - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Oman - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Slovakia - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Kenya - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Nigeria - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Botswana - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Slovenia - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Croatia - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Serbia - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Bhutan - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Nepal - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs
Uzbekistan - Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs