Math 5251 Information and Coding Theory

Spring 2017

Often transmission of data through a noisy channel is preceded by coding the data in two stages.

Source coding: may be used as a method of data compression.  The raw data may contain redundant information or inessential information or may just be presented in an inefficient manner. You may want the data to be converted to a string of 0’s and 1’s.  There are various source coding algorithms. For example, if the data consists of word documents, you might want the most frequently occurring words, regardless of length, to be represented by the shortest bit strings.  Thus probability enters into the analysis. The extent to which compression is possible is measured by the entropy of the data.

Channel coding is typically used as an error detection and correction tool. The problem is that in transmission of a bit string, some of the 0’s might be converted to 1’s and vice versa.  Codes have been developed to fight against this problem. Channel capacity gives a measure of what is theoretically possible. Study of the class of linear codes leads us into the subject of linear algebra with scalars coming from finite fields.