Information Sciences Group
Ali Tajer
Professor
Electrical, Computer, and Systems Engineering
Rensselaer Polytechnic Institute
(518) 276-8237
6040 Jonsson Engineering Center (JEC)
110 8th Street, Troy, NY 12180
Information Theory & Coding (High-dimensional Statistics)
Fall 2018
Title | Topic |
|---|---|
Lecture 01 | information theory history and applications, information measures, entropy |
Lecture 02 | entropy, convexity, submodularity, divergence |
Lecture 03 | differential entropy, conditional divergence, mutual information |
Lecture 04 | mutual information, conditional mutual information, geometric interpretation of mutual information |
Lecture 05 | variational characterization of divergence, sufficient statistics |
Lecture 06 | statistical decision theory: basics |
Lecture 07 | risk functions |
Lecture 08 | tensor product of experiments, sample complexity |
Lecture 09 | sample complexity, f-divergence, hypothesis testing, connection between f-divergences |
Lecture 10 | connection between f-divergences, variational form of f-divergence |
Lecture 11 | f-divergence, parameter estimation, HCR bound, CR lower bound, fisher information |
Lecture 12 | Fisher information, multivariate HCR bound |
Lecture 13 | Bayesian CR lower bound, information bound, local estimators, biased estimators |
Lecture 14 | maximum likelihood estimator, high-dimensional unstructured estimation, bowl-shaped loss |
Lecture 15 | two-point quantization of the estimation problem (LeCam’s method) |
Lecture 16 | two-point per dimension (coordinate) quantization of the estimation problem (Assouad's method) |
Lecture 17 | information-theoretic method to analyzing risk; model capacity, geometric interpretation |
Lecture 18 | Shannon's method, Fano's method |
Lecture 19 | structured high-dimensional estimation, denoising a sparse vector (lower bound) |
Lecture 20 | denoising a sparse vector (upper bound); thresholding schemes for sparse recovery |
Lecture 21 | linear regression and sparse recovery |
Lecture 22 | functional estimation (lower bounds) |
Lecture 23 | functional estimation (upper bounds) |