云计算与分布式系统 : 从并行处理到物联网

PrefaceAbout the Authors.PART SYSTEMS MODELING, CLUSTERINGAND VIRTUALIZATIONCHAPTER Distributed System Models and Enabling TechnologiesSummary1.1 Scalable Computing over the Internet1.1.1 The Age of Internet Computing1.1.2 Scalable Computing Trends and New Paradigms81.1.3 The Internet of Things and Cyber-Physical Systems1.2 Technologies for Network-Based Systems.131.2.1 Multicore CPUs and Multithreading Technologies1.2.2 GPU Computing to Exascale and Beyond.1.2.3 Memory, Storage, and Wide-Area Networking.1.2.4 Virtual Machines and Virtualization Middleware.1.2.5 Data Center Virtualization for Cloud Computing.1.3 System Models for Distributed and Cloud Computing.1.3.1 Clusters of Cooperative Computers.1.3.2 Grid Computing Infrastructures.1.3.3 Peer-to-Peer Network Families1.3.4 Cloud Computing over the Internet.1.4 Software Environments for Distributed Systems and Clouds.1.4.1 Service-Oriented Architecture (SOA)1.4.2 Trends toward Distributed Operating Systems.1.4.3 Parallel and Distributed Programming Models.1.5 Performance, Security, and Energy Efficiency1.5.1 Performance Metrics and Scalability Analysis.1.5.2 Fault Tolerance and System Availability.1.5.3 Network Threats and Data Integrity1.5.4 Energy Efficiency in Distributed Computing.1.6 Bibliographic Notes and Homework Problems.Acknowledgments.ReferencesHomework Problems.Foreword.CHAPTER Computer Clusters for Scalable Parallel ComputingSummary.2.1 Clustering for Massive Parallelism2.1.1 Cluster Development Trends2.1.2 Design Objectives of Computer Clusters.2.1.3 Fundamental Cluster Design Issues.2.1.4 Analysis of the Top Supercomputers.2.2 Computer Clusters and MPP Architectures2.2.1 Cluster Organization and Resource Sharing2.2.2 Node Architectures and MPP Packaging.2.2.3 Cluster System Interconnects2.2.4 Hardware, Software, and Middleware Support.2.2.5 GPU Clusters for Massive Parallelism2.3 Design Principles of Computer Clusters2.3.1 Single-System Image Features2.3.2 High Availability through Redundancy.2.3.3 Fault-Tolerant Cluster Configurations2.3.4 Checkpointing and Recovery Techniques2.4 Cluster Job and Resource Management2.4.1 Cluster Job Scheduling Methods2.4.2 Cluster Job Management Systems.2.4.3 Load Sharing Facility (LSF) for Cluster Computing2.4.4 MOSIX； An OS for Linux Clusters and Clouds.2.5 Case Studies of Top Supercomputer Systems.2.5.1 Tianhe-1A； The World Fastest Supercomputer in 102.5.2 Cray XT5 Jaguar； The Top Supercomputer in 092.5.3 IBM Roadrunner； The Top Supercomputer in 082.6 Bibliographic Notes and Homework ProblemsAcknowledgments. 1References.Homework Problems.CHAPTER Virtual Machines and Virtualization of Clusters and DataCenters.Summary3.1 Implementation Levels of Virtualization3.1.1 Levels of Virtualization Implementation.3.1.2 VMM Design Requirements and Providers.3.1.3 Virtualization Support at the OS Level3.1.4 Middleware Support for Virtualization3.2 Virtualization Structures/Tools and Mechanisms.3.2.1 Hypervisor and Xen Architecture.3.2.2 Binary Translation with Full Virtualization.3.2.3 Para-Virtualization with Compiler Support.xii Contents3.3 Virtualization of CPU, Memory, and I/O Devices.3.3.1 Hardware Support for Virtualization3.3.2 CPU Virtualization3.3.3 Memory Virtualization.3.3.4 I/O Virtualization1503.3.5 Virtualization in Multi-Core Processors.3.4 Virtual Clusters and Resource Management.3.4.1 Physical versus Virtual Clusters3.4.2 Live VM Migration Steps and Performance Effects.3.4.3 Migration of Memory, Files, and Network Resources.3.4.4 Dynamic Deployment of Virtual Clusters3.5 Virtualization for Data-Center Automation3.5.1 Server Consolidation in Data Centers3.5.2 Virtual Storage Management. 13.5.3 Cloud OS for Virtualized Data Centers.3.5.4 Trust Management in Virtualized Data Centers.3.6 Bibliographic Notes and Homework ProblemsAcknowledgments.References.Homework Problems.PART COMPUTING CLOUDS, SERVICE-ORIENTEDARCHITECTURE, AND PROGRAMMINGCHAPTER Cloud Platform Architecture over Virtualized DataCentersSummary4.1 Cloud Computing and Service Models.4.1.1 Public, Private, and Hybrid Clouds.4.1.2 Cloud Ecosystem and Enabling Technologies.4.1.3 Infrastructure-as-a-Service (IaaS)4.1.4 Platform-as-a-Service (PaaS) and Software-as-a-Service(SaaS).4.2 Data-Center Design and Interconnection Networks2064.2.1 Warehouse-Scale Data-Center Design2064.2.2 Data-Center Interconnection Networks4.2.3 Modular Data Center in Shipping Containers.4.2.4 Interconnection of Modular Data Centers4.2.5 Data-Center Management Issues4.3 Architectural Design of Compute and Storage Clouds.4.3.1 A Generic Cloud Architecture Design4.3.2 Layered Cloud Architectural Development.4.3.3 Virtualization Support and Disaster Recovery.4.3.4 Architectural Design ChallengesContents xiii4.4 Public Cloud Platforms； GAE, AWS, and Azure4.4.1 Public Clouds and Service Offerings.4.4.2 Google App Engine (GAE)2294.4.3 Amazon Web Services (AWS).4.4.4 Microsoft Windows Azure.4.5 Inter-cloud Resource Management4.5.1 Extended Cloud Computing Services.4.5.2 Resource Provisioning and Platform Deployment4.5.3 Virtual Machine Creation and Management.4.5.4 Global Exchange of Cloud Resources4.6 Cloud Security and Trust Management.4.6.1 Cloud Security Defense Strategies.4.6.2 Distributed Intrusion/Anomaly Detection4.6.3 Data and Software Protection Techniques4.6.4 Reputation-Guided Protection of Data Centers4.7 Bibliographic Notes and Homework ProblemsAcknowledgementsReferences.Homework Problems.CHAPTER Service-Oriented Architectures for DistributedComputingSummary5.1 Services and Service-Oriented Architecture5.1.1 REST and Systems of Systems.5.1.2 Services and Web Services.5.1.3 Enterprise Multitier Architecture5.1.4 Grid Services and OGSA.5.1.5 Other Service-Oriented Architectures and Systems.5.2 Message-Oriented Middleware5.2.1 Enterprise Bus.5.2.2 Publish-Subscribe Model and Notification5.2.3 Queuing and Messaging Systems.5.2.4 Cloud or Grid Middleware Applications.5.3 Portals and Science Gateways5.3.1 Science Gateway Exemplars5.3.2 HUBzero Platform for Scientific Collaboration5.3.3 Open Gateway Computing Environments (OGCE).5.4 Discovery, Registries, Metadata, and Databases.5.4.1 UDDI and Service Registries.5.4.2 Databases and Publish-Subscribe5.4.3 Metadata Catalogs3085.4.4 Semantic Web and Grid5.4.5 Job Execution Environments and Monitoring.xiv Contents5.5 Workflow in Service-Oriented Architectures.5.5.1 Basic Workflow Concepts.3155.5.2 Workflow Standards3165.5.3 Workflow Architecture and Specification.5.5.4 Workflow Execution Engine3195.5.5 Scripting Workflow System Swift.5.6 Bibliographic Notes and Homework ProblemsAcknowledgementsReferences.Homework Problems.CHAPTER Cloud Programming and Software Environments.Summary6.1 Features of Cloud and Grid Platforms6.1.1 Cloud Capabilities and Platform Features6.1.2 Traditional Features Common to Grids and Clouds.6.1.3 Data Features and Databases.6.1.4 Programming and Runtime Support3416.2 Parallel and Distributed Programming Paradigms6.2.1 Parallel Computing and Programming Paradigms6.2.2 MapReduce, Twister, and Iterative MapReduce.6.2.3 Hadoop Library from Apache.3556.2.4 Dryad and DryadLINQ from Microsoft.6.2.5 Sawzall and Pig Latin High-Level Languages.6.2.6 Mapping Applications to Parallel and DistributedSystems6.3 Programming Support of Google App Engine6.3.1 Programming the Google App Engine6.3.2 Google File System (GFS).6.3.3 BigTable, Google’s NOSQL System6.3.4 Chubby, Google’s Distributed Lock Service.6.4 Programming on Amazon AWS and Microsoft Azure.6.4.1 Programming on Amazon EC2.6.4.2 Amazon Simple Storage Service (S3).6.4.3 Amazon Elastic Block Store (EBS) and SimpleDB.6.4.4 Microsoft Azure Programming Support.6.5 Emerging Cloud Software Environments.6.5.1 Open Source Eucalyptus and Nimbus.6.5.2 OpenNebula, Sector/Sphere, and OpenStack.6.5.3 Manjrasoft Aneka Cloud and Appliances.6.6 Bibliographic Notes and Homework Problems399AcknowledgementReferences.Homework Problems.Contents xvPART GRIDS, P2P, AND THE FUTURE INTERNETCHAPTER Grid Computing Systems and Resource ManagementSummary 167.1 Grid Architecture and Service Modeling.7.1.1 Grid History and Service Families.7.1.2 CPU Scavenging and Virtual Supercomputers4197.1.3 Open Grid Services Architecture (OGSA)7.1.4 Data-Intensive Grid Service Models4257.2 Grid Projects and Grid Systems Built7.2.1 National Grids and International Projects.7.2.2 NSF TeraGrid in the United States.7.2.3 DataGrid in the European Union7.2.4 The ChinaGrid Design Experiences7.3 Grid Resource Management and Brokering7.3.1 Resource Management and Job Scheduling.7.3.2 Grid Resource Monitoring with CGSP7.3.3 Service Accounting and Economy Model7.3.4 Resource Brokering with Gridbus.7.4 Software and Middleware for Grid Computing7.4.1 Open Source Grid Middleware Packages.7.4.2 The Globus Toolkit Architecture (GT4).7.4.3 Containers and Resources/Data Management.7.4.4 The ChinaGrid Support Platform (CGSP)7.5 Grid Application Trends and Security Measures7.5.1 Grid Applications and Technology Fusion7.5.2 Grid Workload and Performance Prediction.7.5.3 Trust Models for Grid Security Enforcement7.5.4 Authentication and Authorization Methods7.5.5 Grid Security Infrastructure (GSI).7.6 Bibliographic Notes and Homework ProblemsAcknowledgmentsReferences471Homework ProblemsCHAPTER Peer-to-Peer Computing and Overlay NetworksSummary8.1 Peer-to-Peer Computing Systems.8.1.1 Basic Concepts of P2P Computing Systems.8.1.2 Fundamental Challenges in P2P Computing.8.1.3 Taxonomy of P2P Network Systems.8.2 P2P Overlay Networks and Properties8.2.1 Unstructured P2P Overlay Networksxvi Contents8.2.2 Distributed Hash Tables (DHTs)8.2.3 Structured P2P Overlay Networks.8.2.4 Hierarchically Structured Overlay Networks8.3 Routing, Proximity, and Fault Tolerance8.3.1 Routing in P2P Overlay Networks.8.3.2 Network Proximity in P2P Overlays8.3.3 Fault Tolerance and Failure Recovery8.3.4 Churn Resilience against Failures.8.4 Trust, Reputation, and Security Management8.4.1 Peer Trust and Reputation Systems8.4.2 Trust Overlay and DHT Implementation8.4.3 PowerTrust； A Scalable Reputation System.8.4.4 Securing Overlays to Prevent DDoS Attacks.8.5 P2P File Sharing and Copyright Protection8.5.1 Fast Search, Replica, and Consistency8.5.2 P2P Content Delivery Networks8.5.3 Copyright Protection Issues and Solutions8.5.4 Collusive Piracy Prevention in P2P Networks8.6 Bibliographic Notes and Homework ProblemsAcknowledgementsReferencesHomework Problems.CHAPTER Ubiquitous Clouds and the Internet of ThingsSummary9.1 Cloud Trends in Supporting Ubiquitous Computing9.1.1 Use of Clouds for HPC/HTC and Ubiquitous Computing9.1.2 Large-Scale Private Clouds at NASA and CERN9.1.3 Cloud Mashups for Agility and Scalability9.1.4 Cloudlets for Mobile Cloud Computing9.2 Performance of Distributed Systems and the Cloud9.2.1 Review of Science and Research Clouds9.2.2 Data-Intensive Scalable Computing (DISC)9.2.3 Performance Metrics for HPC/HTC Systems9.2.4 Quality of Service in Cloud Computing9.2.5 Benchmarking MPI, Azure, EC2, MapReduce, and Hadoop9.3 Enabling Technologies for the Internet of Things9.3.1 The Internet of Things for Ubiquitous Computing9.3.2 Radio-Frequency Identification (RFID)9.3.3 Sensor Networks and ZigBee Technology9.3.4 Global Positioning System (GPS)9.4 Innovative Applications of the Internet of Things9.4.1 Applications of the Internet of ThingsContents xvii9.4.2 Retailing and Supply-Chain Management9.4.3 Smart Power Grid and Smart Buildings9.4.4 Cyber-Physical System (CPS)9.5 Online Social and Professional Networking9.5.1 Online Social Networking Characteristics9.5.2 Graph-Theoretic Analysis of Social Networks9.5.3 Communities and Applications of Social Networks9.5.4 Facebook； The World’s Largest Social Network9.5.5 Twitter for Microblogging, News, and Alert Services9.6 Bibliographic Notes and Homework ProblemsAcknowledgementsReferences.Homework ProblemsIndex

　　随着信息技术的广泛应用和快速发展，云计算作为一种新兴的商业计算模型日益受到人们的广泛关注。本书是一本完整讲述云计算与分布式系统基本理论及其应用的教材。书中从现代分布式模型概述开始，介绍了并行、分布式与云计算系统的设计原理、系统体系结构和创新应用，并通过开源应用和商业应用例子，阐述了如何为科研、电子商务、社会网络和超级计算等创建高性能、可扩展的、可靠的系统。　　　　《云计算与分布式系统：从并行处理到物联网（英文版）》特色：　　全面覆盖现代分布式计算技术，包括集群、网格、面向服务的体系结构、大规模并行处理器、对等网络和云计算。　　提供的案例研究来自主流分布式计算供应商，如亚马逊、微软、谷歌等。　　解释如何利用虚拟化来促进管理、调试、迁移和灾难恢复。　　专为本科生或研究生的分布式系统课程而设计——每章后都配有习题和进一步阅读建议，并为教师提供配套的PPT等教辅资源。【作者简介】　　KaiHwang（黄铠）美国南加州大学电子工程与计算机科学教授，互联网/云计算研究实验室主任；清华大学IV客座讲席教授；IEEE终身会士。他拥有加州大学伯克利分校EECS博士学位，主要研究领域为云计算、分布式系统、高性能计算、普适计算、信任网格计算等。现已发表论文220多篇，出版8本计算机体系结构、数字运算、并行处理、分布式系统、互联网安全和云计算方面的相关著作。他还创建了《theJournal of Parallel and DistributedComputing》，并获得了中国计算机学会2004杰出成就奖、IEEE2011 IPDPS创立者奖。　　Geoffrey Fox美国印第安那大学计算机科学、信息与物理学杰出教授，社会网格实验室主任。之前曾在加州理工和锡拉丘兹大学任教，并领导多个研究组。他拥有英国剑桥大学的博士学位。Fox在并行体系结构、分布式编程、网格计算、Web服务和互联网应用方面做了广泛的工作并发表了大量作品。　　Jack Dongarra 美国田纳西大学电子工程与计算机科学杰出教授，橡树岭国家实验室杰出研究员，曼彻斯特大学TurningFellow。他是ACM/IEEE/SIAM/AAAS会士，是超级计算机基准测试、数值分析、线性代数解算器和高性能计算领域的先驱。多年以来，他都在负责Top500最快计算机的Linpack基准测试评估。基于他在超级计算和高性能领域的巨大贡献，他被评为美国国家工程院院士。