信息论与编码

Chapter1Introduction

Contents

Beforeitstarts,thereissomethingmustbeknown

1.1WhatisInformation

1.2WhatsInformationTheory？

1.2.1OriginandDevelopmentofInformationTheory

1.2.2TheapplicationandachievementofInformationTheorymethods

1.3FormationandDevelopmentofInformationTheory

QuestionsandExercises

BiographyofClaudeElwoodShannon

Chapter2BasicConceptsofInformationTheory

Contents

Preparationknowledge

2.1Self-informationandconditionalself-information

2.1.1Self-Information

2.1.2ConditionalSelf-Information

2.2Mutualinformationandconditionalmutualinformation

2.3Sourceentropy

2.3.1Introductionofentropy

2.3.2Mathematicsdescriptionofsourceentropy

2.3.3Conditionalentropy

2.3.4Unionentropy(Communalentropy)

2.3.5Basicnatureandtheoremofsourceentropy

2.4Averagemutualinformation

2.4.1Definition

2.4.2Physicssignificanceofaveragemutualinformation

2.4.3Propertiesofaveragemutualinformation

2.5Continuoussource

2.5.1Entropyofthecontinuoussource(alsocalleddifferentialentropy)

2.5.2Mutualinformationofthecontinuousrandomvariable

QuestionsandExercises

Additionalreadingmaterials

Chapter3DiscreteSourceInformation

Contents

3.1Mathematicalmodelandclassificationofthesource

3.2Thediscretesourcewithoutmemory

3.3Multi-marksdiscretesteadysource

3.4Sourceentropyofdiscrete

4.2.4Relationshipbetweenentropy,channeldoubtdegreeandmutualinformation

4.3Thediscretechannelwithoutmemoryanditschannelcapacity

4.4Channelcapacity

4.4.1Conceptofchannelcapacity

4.4.2Discretechannelwithoutmemoryanditschannelcapacity

4.4.3Continuouschannelanditschannelcapacity

Chapter5kosslesssourcecoding

Contents

5.1Losslesscoder

5.2Losslesssourcecoding

5.2.1Fixedlengthcodingtheorem

5.2.2Unfixedlengthsourcecoding

5.3Losslesssourcecodingtheorems

5.3.1Classificationofcodeandmaincodingmethod

5.3.2Krafttheorem

5.3.3Losslessunfixedsourcecodingtheorem(ShannonFirsttheorem)

5.4Pragmaticexamplesoflosslesssourcecoding

5.4.1Huffmancoding

5.4.2ShannoncodingandFanocoding

5.5TheLempel-zivalgorithm

5.6Run-LengthEncodingandthePCXformat

QuestionsandExercises

Chapter6Limiteddistortionsourcecoding

Contents

6.1Thestartpointoflimitdistortiontheory

6.2Distortionmeasurement

6.2.1Distortionfunction

6.2.2Averagedistortion

6.3Informationratedistortionfunction

6.4PropertyofR(D)

6.4.1MinimumofDandR(D)

6.4.2DmaxandR(Dmax)

6.4.3TheunderconvexfunctionofR(D)

6.4.4

Questionsandexercises

Bibliography

　　本书重点介绍经典信息论的基本理论，并力图将信息论的基本理论和工程应用的编码理论联系起来，介绍一些关于这些理论的实际应用。全书分为7章，内容包括信息度量的基本理论、无失真信源编码、限失真信源编码、信道编码及其应用等。本书注重基本概念，并且用通俗易懂的语言对它们加以诠释。在当前信息、通信系统飞速发展的大背景下，本书力图用较多的例子和图表来阐述概念和理论，同时尽量避免纠缠于烦琐难懂的公式证明之中。为了加深读者对所讲述知识的理解，每章最后都配有适量的练习题供读者选用。本书可作为高等院校电子信息类学生双语教学的教材或参考书，也可作为通信、电信、电子等领域从业人员的参考资料。　　Thisbookismainlyaboutfoundationofinformationtheoryandthecodingtheory.Also,somepracticalapplicationsofthesetheoriesarementionedinsomeoccasions.Onthebasisofintroducingthemeasureofinformation,itmainlyfocusesontheintroductionofthetheoryoflosslesssourcecoding,limitedlosssourcecoding,channelcodingandtheirapplications.Itpaysattentiontobasicconceptsandusespopulartexttoexplainthem.Inthebackgroundofthecurrentcommunicationsystem,itincludesmanyexamplesandfigurestoelaboratetheseconceptsandtheorems.Atthesametime,itdoesitsbesttoavoidgettingentangledinprovingtheequations.Todeepentheunderstandingtotheknowledgeittouchesupon,therearesomeexercisesattachedattheendofeverychapter.　　Thisbookcanbeusedastheteachingmaterialforcollegestudents,andalso,itcanbethereferenceforcareermanintheareaofcommunication,telecommunicationandelectronic.