The fundamentals of machine learning, built from the University of Nebraska CSCE 479/879 course. This section is the roots. It covers what learning from data means, the paradigms, and the training and evaluation machinery that every model shares, from a linear regression to a deep network. The deep learning section grows out of these ideas, and each note there links back to the fundamental it depends on.
Start here
- AI vs ML vs DL - the cornerstone: why the three are different kinds of claim, not a difficulty ladder.
The paradigms
- Supervised Learning - learning a function from labeled examples.
- Unsupervised Learning - finding structure in data with no labels.
- Generalization vs Memorization - why learning means performing on unseen data, not reciting the training set.
- Features and Representations - how a problem is described, and the shift deep learning makes by learning the description itself.
The models
- Regression: Linear, Logistic, and Softmax - the weighted-sum model in three forms, and the output layer of a neural network.
- Logistic Regression - the log-odds model in depth, and why it is a regression that people use as a classifier.
- Decision Trees and Ensembles - rule-based models, and the random forests and gradient boosting that win on tabular data.
- Support Vector Machines - the maximum-margin hyperplane, and the kernel trick.
- k-Nearest Neighbors - the model that stores the training set instead of summarizing it.
- k-Means Clustering - Lloyd’s alternation, and why initialization decides the answer.
- PCA and Dimensionality Reduction - the unsupervised workhorse, and the linear ancestor of the autoencoder.
Training and evaluation
- Loss Functions - how wrong an answer is.
- Gradient Descent - how a model reduces its loss.
- Bias-Variance Tradeoff - overfitting, underfitting, and the sweet spot between them.
- Regularization: Ridge and Lasso - the L2 and L1 penalties, and why only one of them zeroes coefficients out.
- Train, Validation, Test - measuring performance honestly and avoiding leakage.
- Evaluation Metrics - beyond accuracy: precision, recall, and ROC-AUC.
Connects to
The math these notes lean on lives in Statistics (probability, regression, maximum likelihood, Bayes) and Mathematics (linear algebra). The history of how machine learning went mainstream is in The Deep Learning Revolution.