CS189 Intro to Machine Learning (Schewchuk) - 2022 Spring

Those are note archives as well as very well-structured final review docs I made for my self. I put it here so that you can study it and also I will be able to review it later. So if you found this useful, make sure you star my repo

Additional Note: The credit of most of those notes are due to the instructor (Prof. Johnathan Shewchuk) and TAs of this course. A great thank you for them for making this course so enjoyable!

Resource List

• Resources
  • Final Review Doc by Yunhao
  • MT Cheatsheet (Made mostly by Rahul Shah, I assisted in the process)
• Notes (Credit to Prof. Shewchuk)
  • Lec 01 - Introduction, Overview and Logistics
  • Lec 02 - Linear Classifier (Perceptrons)
  • Lec 03 - Perceptron Continued, Gradient Descent, weight space transform and SVM
  • Lec 04 - Soft Margin SVM, Feature Lifting
  • Lec 05 - ML Abstractions / Optimization(Linear/Quadratic Program)
  • Lec 06 - Decision Theory / Risk Minimization Technique
  • Lec 07 - Gaussian Discriminant Analysis (GDA => LDA/GDA) with Isotropic Gaussian
  • Lec 08 - Eigenvectors / Quadratic Forms / GDA with Anisotropic Gaussian
  • Lec 09 - More GDA with Anisotropic Gaussian / Data Cleaning Vocabs
  • Lec 10 - Regression(Intro/Logistic)
  • Lec 11 - Regression(Least-Squre Polynomial / Weighted Least Square / Logistic), Newton’s Method, ROC Curves
  • Lec 12 - Statistical Justification for Least Square / Empirical Risk / Logistic Loss, Bias-Variance Decomposition
  • Lec 13 - Shrinkage(Ridge/Lasso Regression, Feature Selection)
  • Lec 14 - Decision Trees
  • Lec 15 - Multivariate Decision Trees, Decision Tree Regression, Pruning, Ensemble Learning(Bagging, Random Forest)
  • Lec 16 - Kernel Trick (example on Ridge Regression / Perceptron / Logistic Regression), RBF and Polynomial Kernel
  • Lec 17 - Neural Nets
  • Lec 18 - Neurobiology and Variations on Neural Networks
  • Lec 19 - Better NN Training, CNN
  • Lec 20 - Unsupervised Learning: Principle Component Analysis (PCA)
  • Lec 21 - SVD & Clustering
  • Lec 22 - High Dimensional Intuition, Random Projection, Pseudoinverse
  • Lec 23 - Learning Theory - Shatter Function / VC Dimension
  • Lec 24 - Adaboost & k-Nearest Neighbour
  • Lec 25 - kNN Algorithms(Exhaustive kNN search, Voronoi Diagrams, k-d trees)
• Alternative Note
  • ML Book by former TA Soroush Nasiriany and Garrett Thomas
• Resources
  • Intuition for Decision Boundary | ML Visualizer by Sagnik Bhattacharya, Colin Zhou, Komila Khamidova, and Aaron Sun
  • Math Review | Math for ML by Garrett Thomas
  • Intuition Bulding for Linear Algebra | 3Blue1Brown - Essence of Linear Algebra
  • Matrix Calculus | First two chapters of The Matrix Cookbook
  • Stanford CS229 ML Probability Review
  • Stanford CS229 Linear Algebra Review
  • Vector Derivatives by Stanford CS231N
  • CS189 - Everything about Gradients
  • Stanford CS231N Linear Backprop Handout
  • Counterintuitive Properties of High Dimensional Space
  • AdaBoost clearly explained by StatQuest
  • Maximum a posteriori (MAP) estimation by mathematicalmonk
  • Maximum Likelihood, clearly explained!!! by StatQuest
  • What are Maximum Likelihood (ML) and Maximum a posteriori (MAP)? by Iain