<aside> 💡

Curriculum

The overall curriculum is based on: https://github.com/mlabonne/llm-course

Some other inspirations:

1. https://github.com/mlabonne/llm-course?search=1

2. https://github.com/louisfb01/start-llms?tab=readme-ov-file (good paper lists)

Deploy Models & Paper Lists:

• https://github.com/Mooler0410/LLMsPracticalGuide?tab=readme-ov-file#practical-guide-for-models

Great Thoughts on LLM:

https://github.com/Mooler0410/LLMsPracticalGuide?tab=readme-ov-file#practical-guide-for-models

Monthly Paper List:

https://github.com/Mooler0410/LLMsPracticalGuide?tab=readme-ov-file#practical-guide-for-models

LLM Fundamentals

Module 1. Mathematics for Machine Learning

Calculus

• Overview: Many machine learning algorithms involve the optimization of continuous functions, which requires an understanding of derivatives, integrals, limits, and series. Multivariable calculus and the concept of gradients are also important.
• Resources
  • 3Blue1Brown's "Essence of Calculus" series (About 4hours in total)
  • Understand Calculus in 35minutes
  • Calculus for Machine Learning, by Jon Krohn (About 4-5hours in total)
  • [Great for teaching] Sage Interaction - Calculus: An interactive tool that covers most of the basics of calculus
  • Khan Academy - Calculus: An interactive course that covers all the basics of calculus. (About 25.5hours)

Probability and Statistics

• Overview: These are crucial for understanding how models learn from data and make predictions. Key concepts include probability theory, random variables, probability distributions, expectations, variance, covariance, correlation, hypothesis testing, confidence intervals, maximum likelihood estimation, and Bayesian inference.
• Resources
  • Seeing Theory: A visual introduction to probability and statistics.
    • https://seeing-theory.brown.edu/#firstPage
  • Sage Interactions - Statistics and Probability
    • https://wiki.sagemath.org/interact/stats
  • Probability Distributions
  • Bayesian Inference
  • Interactive Gaussian Process Visualization

Linear Algebra

• Overview: This is crucial for understanding many algorithms, especially those used in deep learning. Key concepts include vectors, matrices, determinants, eigenvalues and eigenvectors, vector spaces, and linear transformations.
• Resources:
  • Sage Interactions - Linear Algebra
  • geeksforgeeks - Linear Algebra Operations For Machine Learning

Python Basics

• Python Basics: Python programming requires a good understanding of the basic syntax, data types, error handling, and object-oriented programming.
• Resources:
  • Real Python: A comprehensive resource with articles and tutorials for both beginner and advanced Python concepts.
  • freeCodeCamp - Learn Python: Long video that provides a full introduction into all of the core concepts in Python.

Data Science Libraries & Data PreProcessing

• Data Science Libraries: It includes familiarity with NumPy for numerical operations, Pandas for data manipulation and analysis, Matplotlib and Seaborn for data visualization.
• Data Preprocessing: This involves feature scaling and normalization, handling missing data, outlier detection, categorical data encoding, and splitting data into training, validation, and test sets.
• Resources:
  • Python Data Science Handbook: Free digital book that is a great resource for learning pandas, NumPy, Matplotlib, and Seaborn.

Machine Learning Libraries

• Machine Learning Libraries: Proficiency with Scikit-learn, a library providing a wide selection of supervised and unsupervised learning algorithms, is vital. Understanding how to implement algorithms like linear regression, logistic regression, decision trees, random forests, k-nearest neighbors (K-NN), and K-means clustering is important. Dimensionality reduction techniques like PCA and t-SNE are also helpful for visualizing high-dimensional data.
• Resources:
• freeCodeCamp - Machine Learning for Everybody: Practical introduction to different machine learning algorithms for beginners.
• Udacity - Intro to Machine Learning: Free course that covers PCA and several other machine learning concepts. </aside>

LLM Fundamentals Recordings & Materials