Course Code: ftosllms
Duration: 14 hours
Prerequisites:
  • 了解大型语言模型（LLMs）及其架构
  • 具备Python和PyTorch的经验
  • 对Hugging Face生态系统有基本了解

目标受众

  • 机器学习从业者
  • 人工智慧开发者
Overview:

微调开源LLMs是组织在安全、节省成本且私密的环境中定制AI能力的新兴最佳实践。

这门由讲师指导的培训（线上或线下）面向中级ML从业者和AI开发者，他们希望微调并部署像LLaMA、Mistral和Qwen这样的开源模型，以满足特定业务或内部应用的需求。

在培训结束时，参与者将能够：

  • 理解开源LLMs的生态系统及其差异。
  • 为LLaMA、Mistral和Qwen等模型准备数据集和微调配置。
  • 使用Hugging Face Transformers和PEFT执行微调流程。
  • 在安全环境中评估、保存并部署微调后的模型。

课程形式

  • 互动式讲座与讨论。
  • 大量练习与实践。
  • 在实时实验室环境中进行动手操作。

课程定制选项

  • 如需为本课程定制培训，请联系我们安排。
Course Outline:

开源LLMs简介

  • 什么是开放权重模型及其重要性
  • LLaMA、Mistral、Qwen及其他社群模型概述
  • 私有、本地或安全部署的应用案例

环境设置与工具

  • 安装与配置Transformers、Datasets及PEFT函式库
  • 选择适合的硬体进行微调
  • 从Hugging Face或其他储存库载入预训练模型

数据准备与预处理

  • 数据集格式（指令微调、聊天数据、纯文本）
  • 分词与序列管理
  • 创建自定义数据集与数据加载器

Fine-Tuning技术

  • 标准全微调与参数高效方法对比
  • 应用LoRA与QLoRA进行高效微调
  • 使用Trainer API进行快速实验

模型评估与优化

  • 使用生成与准确度指标评估微调模型
  • 管理过拟合、泛化与验证集
  • 性能调优技巧与日志记录

部署与私有使用

  • 保存与载入模型进行推理
  • 在安全企业环境中部署微调模型
  • 本地与云端部署策略

案例研究与Use Case

  • 企业使用LLaMA、Mistral及Qwen的范例
  • 处理多语言与特定领域的微调
  • 讨论：开放与封闭模型的权衡

总结与下一步

Sites Published:

United Arab Emirates - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Qatar - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Egypt - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Saudi Arabia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

South Africa - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Brasil - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Canada - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

中国 - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

香港 - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

澳門 - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

台灣 - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

USA - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Österreich - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Schweiz - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Deutschland - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Czech Republic - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Denmark - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Estonia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Finland - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Greece - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Magyarország - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Ireland - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Luxembourg - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Latvia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

España - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Italia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Lithuania - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Nederland - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Norway - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Portugal - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

România - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Sverige - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Türkiye - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Malta - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Belgique - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

France - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

日本 - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Australia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Malaysia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

New Zealand - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Philippines - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Singapore - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Thailand - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Vietnam - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

India - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Argentina - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Chile - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Costa Rica - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Ecuador - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Guatemala - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Colombia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

México - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Panama - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Peru - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Uruguay - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Venezuela - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Polska - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

United Kingdom - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

South Korea - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Pakistan - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Sri Lanka - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Bulgaria - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Bolivia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Indonesia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Kazakhstan - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Moldova - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Morocco - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Tunisia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Kuwait - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Oman - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Slovakia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Kenya - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Nigeria - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Botswana - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Slovenia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Croatia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Serbia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Bhutan - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Nepal - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Uzbekistan - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)