information theory history and applications, information measures, entropy

​

entropy, convexity, submodularity, divergence

​

differential entropy, conditional divergence, mutual information

​

mutual information, conditional mutual information, geometric interpretation of mutual information

​

variational characterization of divergence, sufficient statistics

​

statistical decision theory: basics

​

risk functions

​

tensor product of experiments, sample complexity

​

sample complexity, f-divergence, hypothesis testing, connection between f-divergences

​

connection between f-divergences, variational form of f-divergence

​

f-divergence, parameter estimation, HCR bound, CR lower bound, fisher information

​

Fisher information, multivariate HCR bound

​

Bayesian CR lower bound, information bound, local estimators, biased estimators

​

maximum likelihood estimator, high-dimensional unstructured estimation, bowl-shaped loss

​

two-point quantization of the estimation problem (LeCam’s method)

​

two-point per dimension (coordinate) quantization of the estimation problem (Assouad's method)

​

information-theoretic method to analyzing risk; model capacity, geometric interpretation

​

Shannon's method, Fano's method

​

structured high-dimensional estimation, denoising a sparse vector (lower bound)

​

denoising a sparse vector (upper bound); thresholding schemes for sparse recovery

​

linear regression and sparse recovery

​

functional estimation (lower bounds)

​

functional estimation (upper bounds)