统计学习基础

Preface to the Second Edition

Preface to the First Edition

1 Introduction

2 Overview of Supervised Learning

2.1 Introduction

2.2 Variable Types and Terminology

2.3 Two Simple Approaches to Prediction

Least Squares and Nearest Neighbors

2.3.1 Linear Models and Least Squares

2.3.2 Nearest-Neighbor Methods

2.3.3 From Least Squares to Nearest Neighbors

2.4 Statistical Decision Theory

2.5 Local Methods in High Dimensions

2.6 Statistical Models, Supervised Learning and Function Approximation

2.6.1 A Statistical Model for the Joint Distribution Pr(X,Y)

2.6.2 Supervised Learning

2.6.3 Function Approximation

2.7 Structured Regression Models

2.7.1 Difficulty of the Problem

2.8 Classes of Restricted Estimators

2.8.1 Roughness Penalty and Bayesian Methods

2.8.2 Kernel Methods and Local Regression

2.8.3 Basis Functions and Dictionary Methods

2.9 Model Selection and the Bias-Variance rlyadeoff

Bibliographic Notes

Exercises

3 Linear Methods for Regression

3.1 Introduction

3.2 Linear Regression Models and Least Squares

3.2.1 Example: Prostate Cancer

3.2.2 The Gauss-Markov Theorem

3.2.3 Multiple Regression from Simple Univariate Regression

3.2.4 Multiple Outputs

3.3 Subset Selection

3.3.1 Best-Subset Selection

3.3.2 Forward- and Backward-Stepwise Selection

3.3.3 Forward-Stagewise Regression

3.3.4 Prostate Cancer Data Example (Continued)

3.4 Shrinkage Methods

3.4.1 Ridge Regression

3.4.2 The Lasso

3.4.3 Discussion: Subset Selection, Ridge Regression and the Lasso

3.4.4 Least Angle Regression

3.5 Methods Using Derived Input Directions

3.5.1 Principal Components Regression

3.5.2 Partial Least Squares

3.6 Discussion: A Comparison of the Selection and Shrinkage Methods

3.7 Multiple Outcome Shrinkage and Selection

3.8 More on the Lasso and Related Path Algorithms

3.8.1 Incremental Forward Stagewise Regression

3.8.2 Piecewise-Linear Path Algorithms

3.8.3 The Dantzig Selector

3.8.4 The Grouped Lasso

3.8.5 Further Properties of the Lasso

3.8.6 Pathwise Coordinate Optimization

3.9 Computational Considerations

Bibliographic Notes

Exercises

……

4 Linear Methods for Classification

5 Basis Expansions and Regularization

6 Kernel Smoothing Methods

7 Model Assessment and Selection

8 Modellnference and Averaging

9 Additive Models, Trees, and Related Methods

10 Boosting and Additive Trees

11 Neural Networks

12 Support Vector Machines and Flexible Discriminants

13 Prototype Methods and Nearest-Neighbors

14 Unsupervised Learning

15 Random Forests

16 Ensemble Learning

17 Undirected Graphical Models

18 High-Dimensional Problems: p≥N

References

Author Index

Index

This book is our attempt to bring together many of the important new ideas in learning, and explain them in a statistical framework. While some mathematical details are needed, we emphasize the methods and their conceptual underpinnings rather than their theoretical properties. As a result, we hope that this book will appeal not just to statisticians but also to researchers and practitioners in a wide variety of fields.