Training and Fine-tuning Large Language Models (LLMs)

Name: Training and Fine-tuning Large Language Models (LLMs)
Author: Yahia Naim

Fine-tune LLMs with W&B

A free course from Weights & Biases covering LLM fine-tuning techniques including LoRA, prefix tuning, evaluation, data curation, and production deployment. Hands-on with W&B experiment tracking.

Data ScienceFreeintermediate4 hours

Submitted by @yahianaim

Roadmap (5 steps)

Foundations

45 min

#LLM Architecture#NeurIPS Efficiency Challenge#Hands-on Fine-tuning#Experiment Tracking

Evaluation

50 min

#LLM Evaluation Metrics#Perplexity#HELM & Benchmarks#Chatbot Arena

Data

50 min

#Dataset Curation#Data Sources & Cost#Data Loading Logistics#Scaling Laws

Training & Fine-tuning Techniques

60 min

#Hardware Requirements#Memory Usage#LoRA & Prefix Tuning#Training Observability

Course Assessment & Next Steps

35 min

#Final Assessment#Further Learning Resources#RLHF#MosaicML Recipes