自然语言标注 : 用于机器学习

Preface

1. The Basics

The Importance of Language Annotation

The Layers of Linguistic Description

What Is Natural Language Processing?

A Brief History of Corpus Linguistics

What Is a Corpus?

Early Use of Corpora

Corpora Today

Kinds of Annotation

Language Data and Machine Learning

Classification

Clustering

Structured Pattern Induction

The Annotation Development Cycle

Model the Phenomenon

Annotate with the Specification

Train and Test the Algorithms over the Corpus

Evaluate the Results

Revise the Model and Algorithms

Summary

2. Defining Your Goal and Dataset

Defining Your Goal

The Statement of Purpose

Refining Your Goal: Informativity Versus Correctness

Background Research

Language Resources

Organizations and Conferences

NLP Challenges

Assembling Your Dataset

The Ideal Corpus: Representative and Balanced

Collecting Data from the Internet

Eliciting Data from People

The Size of Your Corpus

Existing Corpora

Distributions Within Corpora

Summary

3. Corpus Analytics

Basic Probability for Corpus Analytics

Joint Probability Distributions

Bayes Rule

Counting Occurrences

Zipf's Law

N-grams

Language Models

Summary

4. Building Your Model and Specificationl

Some Example Models and Specs

Film Genre Classification

Adding Named Entities

Semantic Roles

Adopting (or Not Adopting) Existing Models

Creating Your Own Model and Specification: Generality Versus Specificity

Using Existing Models and Specifications

Using Models Without Specifications

Different Kinds of Standards

ISO Standards

Community-Driven Standards

Other Standards Affecting Annotation

Summary

5. Applying and Adopting Annotation Standards

Metadata Annotation: Document Classification

Unique Labels: Movie Reviews

Multiple Labels: Film Genres

Text Extent Annotation: Named Entities

Inline Annotation

……

6. Annotation and Adjudication..

7. Training: Machine Learning...

8. Testing and Evaluation.

9. Revising and Reporting.

10. Annotation: TimeML.

11. Automatic Annotation: Generating TimeML.

12. Afterword: The Future of Annotation.

A. List of Available Corpora and Specifications

B. List of Software Resources

C. MAE UserGuide

D. MAI UserGuide

E. Bibliography

Index

是时候创建属于你自己的用于机器学习的自然语言训练语料库了。无论你使用英语、汉语或者其他任何一种自然语言，《自然语言标注：用于机器学习（影印版）》都可以手把手地指导你一种经验证的标注开发周期一一把元语添加到你的训练语料库中来帮助机器学习算法更有效工作的过程。你无需任何编程或者语言学方面的经验就可以上手。
　　通过每一步中的详细示例，你将学到“标注开发过程”是如何帮助你建模、标注、训练、测试、评估和修正你的训练语料库。你也将了解到一个实际标注项目的完整演示。

在收集你的数据集（语料库）之前定义一个清晰的标注目标
　　学习用于分析你的语料库中语言内容的工具
　　搭建用于你的标注项目的模型和规范
　　检查从基本的XML到语言标记框架这样一些不同的标注格式
　　创建适合于训练和测试机器学习算法的黄金标准语料库
　　选择用来处理你的标注数据的机器学习算法
　　评估测试结果并修正你的标注任务
　　学习如何使用用于标注文本和调整标注的轻量级软件