Course on Information Theory 1

Purpose:

Short description of the contents

source coding (data compaction)

principes:

entropy, divergence, divergence-inequality, convexity and convex functions.

block-to-block coding:

source coding theorem for block-to-block coding, converse for block-to-block coding.

block-to-variable coding:

uniquely decodable codes, comma code, prefix code, Kraft inequality, Mc-Millan's theorem, source coding theorem for block-to-variable coding, Huffman codes.

stationary sources:

entropy of stationary sources, block entropy, conditional entropy, innovation entropy, chain rule for entropy, unifilar and non-unifilar Markov sources.

rate-distortion:

Rate-distortion function, R(D) for a binary memoryless source with Hamming distortion, achievability of R(D) and converse, the general case.

 

channel coding (error-correction)

principes:

memoryless channels, average mutual information, channel capacity, equivocation, the Fano inequality, the dataprocessing theorem, Markov relation, Shannon's converse for channel coding, convexity of the average mutual information.

computation of channel capacity:

Kuhn-Tucker theorem, some special properties at capacity.

achievability of capacity:

block codes, code-efficiency, stochastic coding, threshold decoding, Hamming distance and Hamming weight, threshold error exponent.

error correction:

correctable error fraction, Hamming codes, Hamming bound, Gilbert bound, sphere packing error exponent.

Cryptography

Traditional cryptographic methods, a probabilistic attack, Shannon's cryptosystem, theoretic secrecy, perfect sececy, random ciphers and unicity distance.

Advanced cources

• Information Theory 2 (5P340).
• Communicatietheorie (5JJ80).
• Coding Theory (2F830).
• Cryptology (2F590).