Lecture 01

information theory history and applications, information measures, entropy

Lecture 02

entropy, convexity, submodularity, divergence

Lecture 03

differential entropy, conditional divergence, mutual information

Lecture 04

mutual information, conditional mutual information

Lecture 05

variational characterization of divergence, sufficient statistics

Lecture 06

variational characterization of divergence, sufficient statistics

Lecture 07

feature selection via information gain, structure learning, density estimation

Lecture 08

information projection, information bottleneck

Lecture 09

source coding, Kraft and McMillan theorems, Huffman codes, prefix codes

Lecture 10

maximum description length principle, rate-distortion theory

Lecture 11

empirical risk minimization, histogram classifiers, decision trees

Lecture 12

histogram regression, universal prediction, unbounded loss functions