Help support MIT OpenCourseWare by shopping at Amazon.com! MIT OpenCourseWare offers direct links to Amazon.com to purchase the books cited in this course. Click on the book titles and purchase the book from Amazon.com, and MIT OpenCourseWare will receive up to 10% of all purchases you make. Your support will enable MIT to continue offering open access to MIT courses. 
The reading assignments are from the required course textbook, Cover, T. M and J. A. Thomas. Elements of Information Theory. New York, NY: John Wiley & Sons, 2001. ISBN: 0471062596.
Supplemental readings are also provided below the required readings.
1 
Introduction, Review of Random Variables, Entropy, Mutual Information, Chain Rules 
Chapter 1, Sections 2.1  2.5 
2 
Jensen's Inequality, Data Processing Theorem, Fanos's Inequality 
Sections 2.6  2.8 and 2.11 
3 
Markov Chain, Entropy Rate of Random Processes 
Chapter 4 
4 
Different Types of Convergence, Asymptotic Equipartition Property (AEP), Typical Set, Joint Typicality 
Sections 3.1  3.3 and 8.6 
5 
Data Compression, Kraft Inequality, Optimal Codes 
Sections 5.1  5.5 
6 
Huffman Codes, Sensitivity of Distribution, Elias Code 
Sections 5.6  5.9 
7 
Gambling 
Chapter 6 
8 
Channel Capacity, Symmetric and Erasure Channels 
Sections 8.1  8.3 
9 
Coding Theorem 
Sections 8.4  8.9 
10 
Strong Coding Theorem 

11 
Strong Coding Theorem (cont.) 

12 
Feedback Capacity 
Section 8.12 
13 
Joint Source Channel Coding 
Section 8.13 
14 
Differential Entropy 
Chapter 9 
15 
Gaussian Channel 
Sections 10.1  10.3 
16 
Gaussian Channels: Parallel, Colored Noise, Intersymbol Interference 
Sections 10.4  10.5 
17 
Maximizing Entropy 
Sections 11.1  11.6 
18 
Gaussian Channels with Feedback 
Section 10.6 
19 
Fading Channels 

20 
Types, Universal Source Coding, Sanov's Theorem 
Sections 12.1  12.6 
21 
Multiple Access Channels 
Sections 14.1  14.3 
22 
SlepianWolf Coding 
Sections 14.4  14.5 
23 
Broadcast Channels 
Section 14.6 
24 
Channel Side Information, Wideband Channels 


Supplemental Readings
This section provides the optional supplementary readings for the course. Most of the readings are coded for difficulty:
One star (*): Accessible
Two stars (**): Requires significant mathematical maturity
Three stars(***): Expert level
Shannon, Claude. "A Mathematical Theory of Communication.'' Bell System Technical Journal 27 (July and October, 1948): 379423 and 623656.
***Vembu, Sridhar, Sergio Verdu, and Yossef Steinberg. "The SourceChannel Separation Theorem Revisited." IEEE Trans. on Info. Theory 41, no. 1 (January 1995).
***Biglieri, Ezio, John Proakis, and Shlomo Shamai. "Fading Channels: InformationTheoretic and Communications Aspects." IEEE Trans. on Info. Theory 44, no. 6 (October, 1998).
*Rimoldi, Bixio, and Rudiger Urbanke. "A RateSplitting Approach to the Gaussian MultipleAccess Channel." IEEE Trans. on Info. Theory 42, no. 2 (March, 1996).
*Bergmans, Patrick. "A Simple Converse for Broadcast Channels with Additive White Gaussian Noise." IEEE Trans. on Info. Theory 20, no. 2 (March, 1974).
*———. "Random Coding Theorem for Broadcast Channels with Degraded Components" IEEE Trans. on Info. Theory 19, no. 2 (March 1973).
**Cover, Thomas. "Comments on Broadcast Channels." IEEE Trans. on Info. Theory 44, no. 6 (October, 1998).
**Mushkin, Mordechai, and Israel BarDavid. "Capacity and Coding for the GilbertElliott Channels." IEEE Trans. on Info. Theory 35, no. 6 (November, 1989).
***Feder, Meir, and Neri Merhav. "Hierarchical Universal Coding." IEEE Trans. on Info. Theory 42, no. 5 (September, 1986).
***Verdu, Sergio, and Te Sun Han. "A General Formula for Channel Capacity." IEEE Trans. on Info. Theory 40, no. 4 (July, 1994).
*Gallager, Robert. "Variations on a Theme by Huffman." IEEE Trans. on Info. Theory 24, no. 6 (November, 1978).
*Chomsky, Noam. "Three Models for the Description on Language." IEEE Trans. on Info. Theory 2, no. 3 (November, 1956).
**Thomas, Joy. "Feedback Can at Most Double Gaussian Multiple Access Channel Capacity." IEEE Trans. on Info. Theory 33, no. 5 (September, 1987).
**Cheng, Roger, and Sergio Verdu. "Gaussian Multiaccess Channel with ISI: Capacity Region and Multiuser WaterFilling." IEEE Trans. on Info. Theory 39, no. 3 (May, 1993).
***Caire, Giuseppe, and Shlomo Shamai. "On the Capacity of Some Channels with Channel State Information." IEEE Trans. on Info. Theory 45, no. 6 (September, 1999).
*Gray, Robert, Mari Dunham, and R. Gobbi. "Eogodicity of Markov Channels." IEEE Trans. on Info. Theory 33, no. 5 (September, 1987).
***Goldsmith, Andrea, and Pravin Varaiya. "Capacity, Mutual Information, and Coding for FiniteState Markov Channels." IEEE Trans. on Info. Theory 42, no. 3 (May, 1993).
*Gallager, Robert. "A Perspective on Multiaccess Channels." IEEE Trans. on Info. Theory 31, no. 2 (March, 1985).
***Hirt, Walter, and James Massey. "Capacity of the DiscreteTime Gaussian Channel with Intersymbol Interference." IEEE Trans. on Info. Theory 34, no. 3 (May, 1988).
**Shamai, Shlomo, and Aaron Wyner. "InformationTheoretic Considerations for Symmetric Cellular, MultipleAccess Fading Channels, Part I and II." IEEE Trans. on Info. Theory 43, no. 6 (May, 1997).
***Ozarow, Lawrence. "Random Coding for Additive Gaussian Channels with Feedback." IEEE Trans. on Info. Theory 36, no. 1 (January, 1990).
***Hughes, Brian, and Tony Thomas. "On Error Exponents for Arbitrarily Varying Channels." IEEE Trans. on Info. Theory 42, no. 1 (May, 1996).
***Medard, Muriel, and Robert Gallager. "Bandwidth Scaling for Fading Multipath Channels." IEEE Trans. on Info. Theory 48, no. 4 (April, 2002).
***Gallager, Robert. "A Simple Derivation of the Coding Theorem and Some Applications." IEEE Trans. on Info. Theory 11, no. 1 (January, 1965).
***Telatar, Emre, and David Tse. "Capacity and Mutual Information of Wideband Multipath Fading Channels." IEEE Trans. on Info. Theory 46, no. 4 (July, 2000).
**Telatar, I., "Capacity of Multiantenna Gaussian Channels." European Transactions on Telecommunications 10 (Nov/Dec 1999): 585595.
Verdu, Sergio. "On Channel Capacity per Unit Cost." EEE Trans. on Info. Theory 36, no. 5 (September 1990).
———. "Spectral Efficiency in the Wideband Regime." EEE Trans. on Info. Theory 48, no. 6 (June 2002).
Forney, David. "On Exponential Error Bounds for Random Codes on the BSC."
Jindal, Nihar, Sriram Vishwanath, and Andrea Goldsmith. "On The Duality of Gaussian MultipleAccess and Broadcast Channels." Preprint 2002.