马尔科夫过程导论

Preface

Chapter1RandomWalksAGoodPlacetoBegin

1.1.NearestNeighborRandomWlalksonZ

1.1.1.DistributionatTimen

1.1.2.PassageTimesviatheReflectionPrinciple

1.1.3.SomeRelatedComputations

1.1.4.TimeofFirstReturn

1.1.5.PassageTimesviaFunctionalEquations

1.2.RecurrencePropertiesofRandomWalks

1.2.1.RandomWalksonZd

1.2.2.AnElementaryRecurrenceCriterion

1.2.3.RecurrenceofSymmetricRandomWalkinZz

1.2.4.nansienceinZ3

1.3.Exercises

Chapter2Doeblin’STheoryforMarkovChains

2.1.SomeGeneralities

2.1.1.ExistenceofMarkovChains

2.1.2.TransionProbabilities＆ProbabilityVectors

2.1.3.nansitionProbabilitiesandFunctions

2.1.4.TheMarkovProperty

2.2.Doeblin’STheory

2.2.1.Doeblin’SBasicTheorem

2.2.2.ACoupleofExtensions

2.3.ElementsofErgodicTheory

2.3.1.TheMeanErgodicTheorem

2.3.2.ReturnTimes

2.3.3.Identificationofπ

2.4.Exercises

Chapter3MoreabouttheErgodicTheoryofMarkovChains

3.1.ClassificationofStates

3.1.1.Classification，Recurrence，andTransience

3.1.2.CriteriaforRecurrenceandTransmnge

3.1.3.Periodicity

3.2.ErgodicTheorywithoutDoeblin

3.2.1.ConvergenceofMatrices

3.2.2.AbelConvergence

3.2.3.StructureofStationaryDistributions

3.2.4.ASmallImprovement

3.2.5.TheMcanErgodicTheoremAgain

3.2.6.ARefinementinTheAperiodicCase

3.2.7.PeriodicStructure

3.3.Exercises

Chapter4MarkovProcessesinContinuousTime

4.1.PoissonProcesses

4.1.1.TheSimplePoissonProcess

4.1.2.CompoundPoissonProcessesonZ

4.2.MarkovProcesseswithBoundedRates

4.2.1.BasicConstruction

4.2.2.TheMarkovProperty

4.2.3.TheQMatrixandKolmogorov’SBackwardEquation

4.2.4.Kolmogorov’SForwardEquation

4.2.5.SolvingKolmogorov’SEquation

4.2.6.AMarkovProcessfromitsInfinitesimalCharacteristics

4.3.UnboundedRates

4.3.1.Explosion

4.3.2.CriteriaforNon.explosionorExplosion

4.3.3.WhattoDoWhenExplosionOccurs

4.4.ErgodicProperties.

4.4.1.ClassificationofStates

4.4.2.StationaryMeasuresandLimitTheorems

4.4.3.Interpretingπii

4.5.Exercises

Chapter5ReversibleMarkovProeesses

5.1.R，eversibleMarkovChains

5.1.1.ReversibilityfromInvariance

5.1.2.MeasurementsinQuadraticMean

5.1.3.TheSpectralGap

5.1.4.ReversibilityandPeriodicity

5.1.5.RelationtoConvergenceinVariation

5.2.DirichletFormsandEstimationofβ

5.2.1.TheDirichletFo.rmandPoincar4’SInequality，

5.2.2.Estimatingβ+

5.2.3.Estimatingβ-

5.3.ReversibleMarkovProcessesinContinuousTime

5.3.1.CriterionforReversibility

5.3.2.ConvergenceinL2(π)forBoundedRates

5.3.3.L2(π)ConvergenceRateinGeneral

5.3.4.Estimatingλ

5.4.GibbsStatesandGlauberDynamics

5.4.1.Formulation

5.4.2.TheDirichletForm

5.5.SimulatedAnnealing

5.5.1.TheAlgorithm

5.5.2.ConstructionoftheTransitionProbabilities

5.5.3.DescriptionoftheMarkovProcess

5.5.4.ChoosingaCoolingSchedule

5.5.5.SmallImprovements

5.6.Exercises

Chapter6SomeMildMeasureTheory

6.1.ADescriptionofLebesguesMeasureTheory

6.1.1.MeasureSpaces

6.1.2.SomeConsequencesofCountableAdditivity

6.1.3.Generatinga-Algebras

6.1.4.MeasurableFunctions

6.1.5.LebesgueIntegration

6.1.6.StabilityPropertiesofLebesgueIntegration

6.1.7.LebesgueIntegrationinCountableSpaces

6.1.8.FubinisTheorem

6.2.ModelingProbability

6.2.1.ModelingInfinitelyManyTossesofaFairCoin

6.3.IndependentRandomVariables

6.3.1.ExistenceofLotsofIndependentRandomVariables

6.4.ConditionalProbabilitiesandExpectations

6.4.1.ConditioningwithRespecttoRandomVariables

Notation

References

Index

　　Tosomeextent,itwouldbeaccuratetosummarizethecontentsofthisbookasanintolerablyprotracteddescriptionofwhathappenswheneitheroneraisesatransitionprobabilitymatrixP(i.e.,allentries(P)arenonnegativeandeachrowofPsumsto1)tohigherandhigherpowersoroneexponentiatesR(P-I),whereRisadiagonalmatrixwithnon-negativeentries.Indeed,whenitcomesrightdowntoit,thatisallthatisdoneinthisbook.However,I,andothersofmyilk,wouldtakeoffenseatsuchadismissivecharacterizationofthetheoryofMarkovchainsandprocesseswithvaluesinacountablestatespace,andaprimarygoalofmineinwritingthisbookwastoconvinceitsreadersthatouroffensewouldbewarranted.