CSCI 145: Data Mining

A course on the mathematical foundations of deep learning.


Instructor: R. Teal Witter. Please call me Teal.

Class Times: Mondays and Wednesdays from 2:45 to 4:00pm in Robert Day 126.

Office Hours: Tuesdays from 1:00 to 4:00pm in Adams Hall 213.

Problem Sets: Problem sets are your primary opportunity to learn the material. You may work with others, but you must write and submit your own solutions and be prepared to explain your reasoning in a brief conversation.

Quizzes: Short five-minute quizzes at the beginning of randomly selected classes test your understanding of the problem sets and concepts from the prior week.

Exams: The two midterm exams are the primary assessment of your understanding of the material.

Project: Choose a useful question, use AI to help execute a reproducible analysis, and tell the resulting story with visual evidence in a public GitHub repository.

Resources: There is no textbook. Each lecture has a reading on this site.

Date Topic Demos Assignments
Mathematical Foundations
Mon Aug 31 Probability Central Limit Theorem (ipynb) Problem 1 (TeX, Solution)
Wed Sep 2 Monte Carlo Estimator Mean Estimation (ipynb) Problem 2 (TeX, Solution)
Mon Sep 7 Labor Day (No Class)
Wed Sep 9 Linear Algebra Warping an Image (ipynb) Problem 3 (TeX, Solution)
Mon Sep 14 Decompositions Power Method (ipynb) Problem 4 (TeX, Solution)
Linear Models
Wed Sep 16 Fitting a Linear Model Fitting a Calibration Line (ipynb) Problem 5 (TeX, Solution)
Mon Sep 21 Least Squares Curves and Redundant Features (ipynb) Problem 6 (TeX, Solution)
Wed Sep 23 Gradient Descent Feature Units and Training Speed (ipynb) Problem 7 (TeX, Solution)
Mon Sep 28 Model Evaluation Selecting a Model on New Data (ipynb) Problem 8 (TeX, Solution)
Wed Sep 30 Likelihood and Loss Functions One Faulty Sensor, Two Losses (ipynb) Problem 9 (TeX, Solution)
Mon Oct 5 Logistic Regression Learning MNIST (ipynb) Problem 10 (TeX, Solution)
Neural Networks
Wed Oct 7 Neural Networks Spiral Decision Regions (ipynb) Problem 11 (TeX, Solution)
Mon Oct 12 Gradient Descent in Practice Training a Neural Network on MNIST (ipynb) Problem 12 (TeX, Solution)
Wed Oct 14 Depth Enablers Gradients Live or Die by Layer (ipynb) Problem 13 (TeX, Solution)
Mon Oct 19 Fall Break (No Class)
Wed Oct 21 Generalization Double Descent (ipynb) Problem 14 (TeX, Solution)
Mon Oct 26 Midterm Exam Project Proposal 1 (TeX)
Low-Rank Structure
Wed Oct 28 Low-Rank Approximation Image and Video Compression (ipynb) Problem 15 (TeX, Solution)
Mon Nov 2 TL;DR
Wed Nov 4 Embeddings Word Embeddings with PCA (ipynb) Problem 16 (TeX, Solution)
Mon Nov 9 Muon Muon vs. Adam (ipynb) Problem 18 (TeX, Solution)
Architectures
Wed Nov 11 Convolutional Networks Class Activation Maps (ipynb) Problem 19 (TeX, Solution)
Mon Nov 16 Self-Attention Attention Heatmaps (ipynb) Problem 20 (TeX, Solution)
Wed Nov 18 Transformer A Transformer That Writes (ipynb) Problem 21 (TeX, Solution)
Mon Nov 23 Positional Embeddings Rotations and Toeplitz Scores (ipynb) Problem 22 (TeX, Solution)
Wed Nov 25 Thanksgiving Break (No Class) Project Proposal 2 (TeX)
Reinforcement Learning
Mon Nov 30 Reinforcement Learning Random vs. Hand-Coded CartPole (ipynb) Problem 23 (TeX, Solution)
Wed Dec 2 Policy Gradients Learning CartPole from Scratch (ipynb) Problem 24 (TeX, Solution)
Mon Dec 7 Baseline and Advantage Three REINFORCEs (ipynb) Problem 25 (TeX, Solution)
Wed Dec 9 Midterm Exam
Dec 14 to Dec 18 Project Presentations (Finals Week) Final Project (TeX)