基于数据分析的网络安全

Preface

PartI.Data

1.Sensors and Detectors： An Introduction

Vantages： How Sensor Placement Affects Data Collection

Domains： Determining Data That Can Be Collected

Actions： What a Sensor Does with Data

Conclusion

2.Network Sensors

Network Layering and Its Impact on Instrumentation

Network Layers and Vantage

Network Layers and Addressing

Packet Data

Packet and Frame Formats

Rolling Buffers

Limiting the Data Captured from Each Packet

Filtering SpeciFic Types of Packets

What Iflt's Not Ethernet？

NetFlow

NetFlow v5 Formats and Fields

NetFlow Generation and Collection

Further Reading

3.Host and Service Sensors： Logging Traffic at the Source

Accessing and Manipulating LogFiles

The Contents of Logfiles

The Characteristics of a Good Log Message

Existing Logflles and How to Manipulate Them

Representative Logflle Formats

HTTP： CLF and ELF

SMTP

Microsoft Exchange： Message Tracking Logs

Logfile Transport： Transfers，Syslog，and Message Queues

Transfer and Logfrle Rotation

Syslog

Further Reading

4.Data Storage for Analysis： Relational Databases，Big Data，and Other Options

Log Data and the CRUD Paradigm

Creating a Well—Organized Flat File System： Lessons from SiLK

A Brieflntroduction to NoSQL Systems

What Storage Approach to Use

Storage Hierarchy，Query Times，and Aging

Partll.Tools

5.The SiLK Suite

What Is SiLK and How Does It Work？

Acquiring and Installing SiLK

The DataFiles

Choosing and Formatting Output Field Manipulation： rwcut

Basic Field Manipulation： rwfrlter

Ports and Protocols

Size

IP Addresses

Time

TCP Options

Helper Options

Miscellaneous Filtering Options and Some Hacks

rwfileinfo and Provenance

Combining Information Flows： rwcount

rwset and IP Sets

rwuniq

rwbag

Advanced SiLK Faalities

pmaps

Collecting SiLK Data

YAF

rwptoflow

rwtuc

Further Reading

6.An Introduction to R for Security Analysts

Installation and Setup

Basics of the Language

The R Prompt

R Variables

Writing Functions

Conditionals and Iteration

Using the R Workspace

Data Frames

Visualization

Visualization Commands

Parameters to Visualization

Annotating a Visualization

ExportingVisualization

Analysis： Statistical Hypothesis Testing

Hypothesis Testing

Testing Data

Further Reading

7.Classification and Event Tools： IDS，AV，and SEM

How an IDS Works

Basic Vocabulary

Classifler Failure Rates： Understanding the Base—Rate Fallacy

Applying ClassiFication

Improving IDS Performance

Enhancing IDS Detection

Enhanang IDS Response

Prefetching Data

Further Reading

8.Reference and Lookup： Tools for Figuring Out Who Someone ls

MAC and Hardware Addresses

IP Addressing

IPv4 Addresses，Theu Structure，and Significant Addresses

IPv6 Addresses，Their Structure and Significant Addresses

Checking Connectivity： Using ping to Connect to an Address

Tracerouting

IP Intelligence： Geolocation and Demographics

DNS

DNS Name Structure

Forward DNS Querying Using dig

The DNS Reverse Lookup

Using whois to Find Ownership

Additional Reference Tools

DNSBLs

9，More Tools

Visualization

Graphviz

Communications and Probing

netcat

nmap

Scapy

Packet Inspection and Reference

Wireshark

GeoIP

The NVD，Malware Sites，and the C*Es

Search Engines，Mailing Lists，and People

Further Reading

Partlll.Analytics

10.Exploratory Data Analysis and Visualization

The Goal of EDA： Applying Analysis

EDA Workflow

Variables and Visualization

Univariate Visualization： Histograms，QQ Plots，Boxplots，and Rank Plots

Histograms

Bar Plots（Not Pie Charts）

The Quantile—Quantile（QQ）Plot

The Five—Number Summary and the Boxplot

Generating a Boxplot

Bivariate Description

Scatterplots

Contingency Tables

Multivariate Visualization

Operationalizing Security Visualization

Further Reading

11.On Fumbling

Attack Models

Fumbling： Misconfiguration，Automation，and Scanning

Lookup Failures

Automation

Scanning

Identifying Fumbling

TCP Fumbling： The State Machine

ICMP Messages and Fumbling

Identifying UDP Fumbling

Fumbling at the Service Level

HTTP Fumbling

SMTP Fumbling

Analyzing Fumbling

Building Fumbling Alarms

Forensic Analysis of Fumbling

Engineering a Network to Take Advantage of Fumbling

Further Reading

12.Volume and Time Analysis

The Workday and Its Impact on Network Traffic Volume

Beaconing

File Transfers／Raiding

Locality

DDoS，Flash Crowds，and Resource Exhaustion

DDoS and Routing Infrastructure

Applying Volume and Locality Analysis

Data Selection

Using Volume as an Alarm

Using Beaconing as an Alarm

Using Locality as an Alarm

Engineering Solutions

Further Reading

13.Graph Analysis

Graph Attributes： What Is a Graph？

Labeling，Weight，and Paths

Components and Connectivity

Clustering Coeffiaent

Analyzing Graphs

Using Component Analysis as an Alarm

Using Centrality Analysis for Forensics

Using Breadth—First Searches Forensically

Using Centrality Analysis for Engineering

Further Reading

14.Application Identification

Mechanisms for Application Identification

Port Number

Application Identiflcation by Banner Grabbing

Application Identification by Behavior

Application Identification by Subsidiary Site

Application Banners： Identifying and Classifying

Non—Web Banners

Web Client Banners： The User—Agent String

Further Reading

15.Network Mapping

Creating an Initial Network Inventory and Map

Creating an Inventory： Data，Coverage，and Files

Phase Ⅰ： The First Three Questions

Phase Ⅱ： Examining the IP Space

Phase Ⅲ： Identifying Blind and Confusing Traffic

Phase Ⅳ： Identifying Clients and Servers

Identifying Sensing and Blocking Infrastructure

Updating the Inventory： Toward Continuous Audit

Further Reading

Index

《基于数据分析的网络安全（影印版）》分成3个部分，包括采集和组织数据的流程、多种分析工具以及几个不同的分析场景和技术。它很适合网络管理员和熟悉脚本的运行安全分析员。传统的入侵检测和日志分析已经不足以保护今天的复杂网络。在这本实用指南里，安全研究员Michael Collins为你展示了多个采集和分析网络流量数据集的技术和工具。你将理解你的网络是如何被利用的以及有哪些必要手段来保护和改善它。