Big Data Glossary

A former Apple engineer, Pete Warden is the founder of OpenHeatMap, and writes on large-scale data processing and visualization.

To help you navigate the large number of new data tools available, this guide describes 60 of the most recent innovations, from NoSQL databases and MapReduce approaches to machine learning and visualization tools. Descriptions are based on first-hand experience with these tools in a production environment.This handy glossary also includes a chapter of key terms that help define many of these tool categories:* NoSQL Databases—Document-oriented databases using a key/value interface rather than SQL* MapReduce—Tools that support distributed computing on large datasets* Storage—Technologies for storing data in a distributed way* Servers—Ways to rent computing power on remote machines* Processing—Tools for extracting valuable information from large datasets* Natural Language Processing—Methods for extracting information from human-created text* Machine Learning—Tools that automatically perform data analyses, based on results of a one-off analysis* Visualization—Applications that present meaningful data graphically* Acquisition—Techniques for cleaning up messy public data sources* Serialization—Methods to convert data structure or object state into a storable format

Preface; Conventions Used in This Book; Using Code Examples; Safari® Books Online; How to Contact Us;Chapter 1: Terms; 1.1 Document-Oriented; 1.2 Key/Value Stores; 1.3 Horizontal or Vertical Scaling; 1.4 MapReduce; 1.5 Sharding;Chapter 2: NoSQL Databases; 2.1 MongoDB; 2.2 CouchDB; 2.3 Cassandra; 2.4 Redis; 2.5 BigTable; 2.6 HBase; 2.7 Hypertable; 2.8 Voldemort; 2.9 Riak; 2.10 ZooKeeper;Chapter 3: MapReduce; 3.1 Hadoop; 3.2 Hive; 3.3 Pig; 3.4 Cascading; 3.5 Cascalog; 3.6 mrjob; 3.7 Caffeine; 3.8 S4; 3.9 MapR; 3.10 Acunu; 3.11 Flume; 3.12 Kafka; 3.13 Azkaban; 3.14 Oozie; 3.15 Greenplum;Chapter 4: Storage; 4.1 S3; 4.2 Hadoop Distributed File System;Chapter 5: Servers; 5.1 EC2; 5.2 Google App Engine; 5.3 Elastic Beanstalk; 5.4 Heroku;Chapter 6: Processing; 6.1 R; 6.2 Yahoo! Pipes; 6.3 Mechanical Turk; 6.4 Solr/Lucene; 6.5 ElasticSearch; 6.6 Datameer; 6.7 BigSheets; 6.8 Tinkerpop;Chapter 7: NLP; 7.1 Natural Language Toolkit; 7.2 OpenNLP; 7.3 Boilerpipe; 7.4 OpenCalais;Chapter 8: Machine Learning; 8.1 WEKA; 8.2 Mahout; 8.3 scikits.learn;Chapter 9: Visualization; 9.1 Gephi; 9.2 GraphViz; 9.3 Processing; 9.4 Protovis; 9.5 Fusion Tables; 9.6 Tableau;Chapter 10: Acquisition; 10.1 Google Refine; 10.2 Needlebase; 10.3 ScraperWiki;Chapter 11: Serialization; 11.1 JSON; 11.2 BSON; 11.3 Thrift; 11.4 Avro; 11.5 Protocol Buffers;