Course Code: ftosllms
Duration: 14 hours
Prerequisites:
  • 了解大型語言模型（LLMs）及其架構
  • 具備Python和PyTorch的經驗
  • 對Hugging Face生態系統有基本了解

目標受眾

  • 機器學習從業者
  • 人工智慧開發者
Overview:

微調開源LLMs是組織在安全、節省成本且私密的環境中定制AI能力的新興最佳實踐。

這門由講師指導的培訓（線上或線下）面向中級ML從業者和AI開發者，他們希望微調並部署像LLaMA、Mistral和Qwen這樣的開源模型，以滿足特定業務或內部應用的需求。

在培訓結束時，參與者將能夠：

  • 理解開源LLMs的生態系統及其差異。
  • 為LLaMA、Mistral和Qwen等模型準備數據集和微調配置。
  • 使用Hugging Face Transformers和PEFT執行微調流程。
  • 在安全環境中評估、保存並部署微調後的模型。

課程形式

  • 互動式講座與討論。
  • 大量練習與實踐。
  • 在實時實驗室環境中進行動手操作。

課程定制選項

  • 如需為本課程定制培訓，請聯繫我們安排。
Course Outline:

開源LLMs簡介

  • 什麼是開放權重模型及其重要性
  • LLaMA、Mistral、Qwen及其他社群模型概述
  • 私有、本地或安全部署的應用案例

環境設置與工具

  • 安裝與配置Transformers、Datasets及PEFT函式庫
  • 選擇適合的硬體進行微調
  • 從Hugging Face或其他儲存庫載入預訓練模型

數據準備與預處理

  • 數據集格式（指令微調、聊天數據、純文本）
  • 分詞與序列管理
  • 創建自定義數據集與數據加載器

Fine-Tuning技術

  • 標準全微調與參數高效方法對比
  • 應用LoRA與QLoRA進行高效微調
  • 使用Trainer API進行快速實驗

模型評估與優化

  • 使用生成與準確度指標評估微調模型
  • 管理過擬合、泛化與驗證集
  • 性能調優技巧與日誌記錄

部署與私有使用

  • 保存與載入模型進行推理
  • 在安全企業環境中部署微調模型
  • 本地與雲端部署策略

案例研究與Use Case

  • 企業使用LLaMA、Mistral及Qwen的範例
  • 處理多語言與特定領域的微調
  • 討論：開放與封閉模型的權衡

總結與下一步

Sites Published:

United Arab Emirates - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Qatar - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Egypt - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Saudi Arabia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

South Africa - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Brasil - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Canada - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

中国 - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

香港 - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

澳門 - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

台灣 - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

USA - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Österreich - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Schweiz - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Deutschland - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Czech Republic - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Denmark - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Estonia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Finland - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Greece - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Magyarország - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Ireland - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Luxembourg - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Latvia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

España - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Italia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Lithuania - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Nederland - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Norway - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Portugal - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

România - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Sverige - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Türkiye - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Malta - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Belgique - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

France - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

日本 - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Australia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Malaysia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

New Zealand - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Philippines - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Singapore - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Thailand - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Vietnam - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

India - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Argentina - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Chile - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Costa Rica - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Ecuador - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Guatemala - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Colombia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

México - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Panama - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Peru - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Uruguay - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Venezuela - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Polska - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

United Kingdom - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

South Korea - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Pakistan - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Sri Lanka - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Bulgaria - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Bolivia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Indonesia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Kazakhstan - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Moldova - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Morocco - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Tunisia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Kuwait - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Oman - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Slovakia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Kenya - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Nigeria - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Botswana - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Slovenia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Croatia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Serbia - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Bhutan - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Nepal - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)

Uzbekistan - Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.)