book

The Definitive Guide to MongoDB: The NoSQL Database for Cloud and Desktop Computing

Name: The Definitive Guide to MongoDB: The NoSQL Database for Cloud and Desktop Computing
ISBN: 9781430230519

by Eelco Plugge, Peter Membrey, Tim Hawkins

September 2010

Intermediate to advanced

327 pages

8h 29m

English

Apress

Read now

Unlock full access

Copyright
About the Authors
About the Technical Reviewer
Acknowledgments
A Special "Thanks" to MongoDB Beijing
Introduction
Our Approach
I. Basics
1. Introduction to MongoDB
1.1. Reviewing the MongoDB Philosophy1.1.1. Using the Right Tool for the Right Job1.1.2. Lacking Innate Support for Transactions1.1.3. Drilling Down on JSON and How It Relates to MongoDB1.1.4. Adopting a Non-Relational Approach1.1.5. Opting for Performance vs. Features1.1.6. Running the Database Anywhere1.2. Fitting Everything Together1.2.1. Generating or Creating a Key1.2.2. Using Keys and Values1.2.3. Implementing Collections1.2.4. Understanding Databases1.3. Reviewing the Feature List1.3.1. Using Document-Orientated Storage (BSON)1.3.2. Supporting Dynamic Queries1.3.3. Indexing Your Documents1.3.4. Leveraging Geospatial Indexes1.3.5. Profiling Queries1.3.6. Updating Information In-Place1.3.7. Storing Binary Data1.3.8. Replicating Data1.3.9. Implementing Auto Sharding1.3.10. Using Map and Reduce Functions1.4. Getting Help1.4.1. Visiting the Website1.4.2. Chatting with the MongoDB Developers1.4.3. Cutting and Pasting MongoDB Code1.4.4. Finding Solutions on Google Groups1.4.5. Leveraging the JIRA Tracking System1.5. Summary
2. Installing MongoDB
2.1. Choosing Your Version2.1.1. Understanding the Version Numbers2.2. Installing MongoDB on Your System2.2.1. Installing MongoDB Under Linux2.2.1.1. Installing MongoDB Through the Repositories2.2.1.2. Installing MongoDB Manually2.2.2. Installing MongoDB Under Windows2.3. Running MongoDB2.3.1. Prerequisites2.3.2. Surveying the Installation Layout2.3.3. Using the MongoDB Shell2.4. Installing Additional Drivers2.4.1. Installing the PHP driver2.4.1.1. Getting MongoDB for PHP2.4.1.2. Installing PHP on Unix-based Platforms Automatically2.4.1.3. Installing PHP on Unix-Based Platforms Manually2.4.1.4. Installing PHP on Windows2.4.2. Confirming Your PHP Installation Works2.4.2.1. Connecting to and Disconnecting from the PHP Driver2.4.3. Installing the Python Driver2.4.3.1. Installing PyMongo under Linux2.4.3.2. Installing PyMongo Automatically2.4.3.3. Installing PyMongo Manually2.4.3.4. Installing PyMongo Under Windows2.4.4. Confirming Your PyMongo Installation Works2.5. Summary
3. The Data Model
3.1. Designing the Database3.1.1. Drilling Down on Collections3.1.2. Using Documents3.1.2.1. Embedding vs. Referencing Information in Documents3.1.3. Creating the _id Field3.2. Building Indexes3.2.1. Impacting Performance with Indexes3.3. Implementing Geospatial Indexing3.3.1. Querying Geospatial Information3.4. Using MongoDB in the Real World3.5. Summary
4. Working with Data
4.1. Navigating Your Databases4.1.1. Viewing Available Databases and Collections4.2. Inserting Data into Collections4.3. Querying for Data4.3.1. Using the Dot Notation4.3.2. Using the Sort, Limit, and Skip Functions4.3.3. Working with Capped Collections, Natural Order, and $natural4.3.4. Retrieving a Single Document4.3.5. Using the Aggregation Commands4.3.5.1. Returning the Number of Documents with Count()4.3.5.2. Retrieving Unique Values with Distinct()4.3.5.3. Grouping Your Results4.3.6. Working with Conditional Operators4.3.6.1. Performing Greater and Less Than Comparisons4.3.6.2. Retrieving All Documents but Those Specified4.3.6.3. Specifying an Array of Matches4.3.6.4. Finding a Value Not in an Array4.3.6.5. Matching all Attributes in a Document4.3.6.6. Searching for Multiple Expressions in a Document4.3.6.7. Retrieving a Document with $slice4.3.6.8. Searching for Odd/Even Integers4.3.6.9. Filtering Results with $size4.3.6.10. Returning a Specific Field Object4.3.6.11. Matching Results Based on the BSON Type4.3.6.12. Matching an Entire Array4.3.6.13. $not (meta-operator)4.3.6.14. Specifying Additional Query Expressions4.3.7. Leveraging Regular Expressions4.4. Updating Data4.4.1. Updating with update()4.4.2. Implementing an Upsert with the save() Command4.4.3. Updating Information Automatically4.4.3.1. Incrementing a Value with $inc4.4.3.2. Setting a Field's Value4.4.3.3. Deleting a Given Field4.4.3.4. Appending a Value to a Specified Field4.4.3.5. Specifying Multiple Values in an Array4.4.3.6. Adding Data to an Array with $addToSet4.4.3.7. Removing Elements from an Array4.4.3.8. Removing Each Occurrence of a Specified Value4.4.3.9. Removing Multiple Elements from an Array4.4.4. Specifying the Position of a Matched Array4.4.5. Atomic Operations4.4.5.1. Using the Update if Current Method4.4.6. Modifying and Returning a Document Atomically4.5. Renaming a Collection4.6. Removing Data4.7. Referencing a Database4.7.1. Referencing Data Manually4.7.2. Referencing Data with DBRef4.8. Implementing Index-Related Functions4.8.1. Surveying Index-Related Commands4.8.2. Forcing a Specified Index to Query Data4.8.3. Constraining Query Matches4.9. Summary

5. GridFS
5.1. Filling in Some Background5.2. Working with GridFS5.3. Getting Started with the Command-Line Tools5.3.1. Using the _id Key5.3.2. Working with Filenames5.3.3. Determining a File's Length5.3.4. Working with Chunk Sizes5.3.5. Tracking the Upload Date5.3.6. Hashing Your Files5.4. Looking Under MongoDB's Hood5.4.1. Using the Search Command5.4.2. Deleting5.4.3. Retrieving Files from MongoDB5.4.4. Summing up mongofiles5.5. Exploiting the Power of Python5.5.1. Connecting to the Database5.5.2. Accessing the Words5.6. Putting Files into MongoDB5.7. Retrieving Files from GridFS5.8. Deleting Files5.9. Summary
II. Developing
6. PHP and MongoDB
6.1. Comparing Documents in MongoDB and PHP6.2. MongoDB Classes6.3. Connecting and Disconnecting6.4. Inserting Data6.5. Listing Your Data6.5.1. Returning a Single Document6.5.2. Listing All Documents6.5.3. Using Query Operators6.5.4. Querying for Specific Information6.5.5. Sorting, Limiting, and Skipping Items6.5.6. Counting the Number of Matching Results6.5.7. Grouping Data with Map/Reduce6.5.8. Specifying the Index with Hint6.5.9. Refining Queries with Conditional Operators6.5.9.1. Using the $lt, $gt, $lte, and $gte Operators6.5.9.2. Finding Documents that Don't Match a Value6.5.9.3. Matching Any of Multiple Values with $in6.5.9.4. Matching All Criteria in a Query with $all6.5.9.5. Searching for Multiple Expressions with $or6.5.9.6. Retrieving a Specified Number of Items with $slice6.5.9.7. Determining Whether a Field Has a Value6.5.10. Regular Expressions6.6. Modifying Data with PHP6.6.1. Updating via update()6.6.2. Saving Time with Modifier Operators6.6.2.1. Increasing the Value of a Specific Key with $inc6.6.2.2. Changing the Value of a Key with $set6.6.2.3. Deleting a Field with $unset6.6.2.4. Appending a Value to a Specified Field with $push6.6.2.5. Adding Multiple Values to a Key with $pushAll6.6.2.6. Adding Data to an Array with $addToSet6.6.2.7. Removing an Element from an Array with $pop6.6.2.8. Removing Each Occurrence of a Value with $pull6.6.2.9. Removing Each Occurrence of Multiple Elements6.6.3. Upserting Data with save()6.6.4. Modifying a Document Atomically6.7. Deleting Data6.8. DBRef6.8.1. Retrieving the Information6.9. GridFS and the PHP Driver6.9.1. Storing Files6.9.2. Adding More Metadata to Stored Files6.9.3. Retrieving Files6.9.4. Deleting Data6.10. Summary
7. Python and MongoDB
7.1. Working with Documents in Python7.2. Using PyMongo Modules7.3. Connecting and Disconnecting7.4. Inserting Data7.5. Finding Your Data7.5.1. Finding a Single Document7.5.2. Finding Multiple Documents7.5.3. Using Dot Notation7.5.4. Returning Fields7.5.5. Simplifying Queries with Sort, Limit, and Skip7.5.6. Aggregating Queries7.5.6.1. Counting Items with Count()7.5.6.2. Counting Unique Items with Distinct()7.5.6.3. Grouping Data with map_reduce()7.5.7. Specifying an Index with Hint()7.5.8. Refining Queries with Conditional Operators7.5.8.1. Using the $lt, $gt, $lte, and $gte Operators7.5.8.2. Searching for Non-Matching Values with $ne7.5.8.3. Specifying an Array of Matches with $in7.5.8.4. Specifying Against an Array of Matches with $nin7.5.8.5. Finding Documents that Match an Array's Values7.5.8.6. Specifying Multiple Expressions to Match with $or7.5.8.7. Retrieving Items from an Array with $slice7.5.9. Conducting Searches with Regular Expression7.6. Modifying the Data7.6.1. Updating Your Data7.6.2. Modifier Operators7.6.2.1. Increasing an Integer Value with $inc7.6.2.2. Changing an Existing Value with $set7.6.2.3. Removing a Key/Value Field with $unset7.6.2.4. Adding a Value to an Array with $push7.6.2.5. Adding Multiple Values to an Array with $pushAll7.6.2.6. Adding a Value to an Existing Array with $addToSet7.6.2.7. Removing an Element from an Array with $pop7.6.2.8. Removing a Specific Value with $pull7.6.3. Saving Documents Quickly with Save()7.6.4. Modifying a Document Atomically7.6.5. Putting the Parameters to Work7.7. Deleting Data7.8. Creating a Link Between Two Documents7.8.1. Retrieving the Information7.9. Summary
8. Creating a Blog Application with the PHP Driver
8.1. Designing the Application8.2. Listing the Posts8.2.1. Paging with PHP and MongoDB8.3. Looking at a Single Post8.3.1. Specifying Additional Variables8.3.2. Viewing and Adding Comments8.4. Searching the Posts8.5. Adding, Deleting, and Modifying Posts8.5.1. Adding a New Post8.5.2. Editing a Post8.5.3. Deleting a Post8.6. Creating the Index Pages8.7. Recapping the blog Application8.8. Summary
III. Advanced
9. Database Administration
9.1. Using Administrative Tools9.1.1. mongo, the MongoDB Console9.1.2. Using Third-Party Administration Tools9.2. Backing up the MongoDB Server9.2.1. Creating a Backup 1019.2.2. Backing up a Single Database9.2.3. Backing up a Single Collection9.3. Digging Deeper into Backups9.4. Restoring Individual Databases or Collections9.4.1. Restoring a Single Database9.4.2. Restoring a Single Collection9.5. Automating Backups9.5.1. Using a Local Datastore9.5.1.1. Installing the Script9.5.2. Using a Remote (Cloud-Based) Datastore9.6. Backing up Large Databases9.6.1. Using a Slave Server for Backups9.6.2. Creating Snapshots with a Journaling Filesystem9.6.3. Disk Layout to Use with Volume Managers9.7. Importing Data into MongoDB9.8. Exporting Data from MongoDB9.9. Securing Your Data9.9.1. Restricting Access to a MongoDB Server9.10. Protecting Your Server with Authentication9.10.1. Adding an Admin User9.10.2. Enabling Authentication9.10.3. Authenticating in the mongo Console9.10.4. Changing a User's Credentials9.10.5. Adding a Read-Only User9.10.6. Deleting a User9.10.7. Using Authenticated Connections in a PHP Application9.11. Managing Servers9.11.1. Starting a Server9.11.2. Reconfiguring a Server9.11.3. Getting the Server's Version9.11.4. Getting the Server's Status9.11.5. Shutting Down a Server9.12. Using MongoDB Logfiles9.13. Validating and Repairing Your Data9.13.1. Repairing a Server9.13.2. Validating a Single Collection9.13.3. Repairing Collection Validation Faults9.13.3.1. Repairing a Collection's Indexes9.13.4. Repairing a Collection's Datafiles9.14. Upgrading MongoDB9.15. Monitoring MongoDB9.15.1. Rolling Your Own Stat Monitoring Tool9.16. Using the mongod Web Interface9.17. Summary
10. Optimization
10.1. Optimizing Your Server Hardware for Performance10.1.1. Understanding How MongoDB Uses Memory10.1.2. Choosing the Right Database Server Hardware10.2. Evaluating Query Performance10.3. MongoDB Profiler10.3.1. Enabling and Disabling the DB Profiler10.3.1.1. Finding Slow Queries10.3.2. Analyzing a Specific Query with explain()10.3.3. Using Profile and explain() to Optimize a Query10.4. Managing Indexes10.4.1. Listing Indexes10.4.2. Creating a Simple Index10.4.3. Creating a Compound Index10.4.3.1. Creating Subdocument Compound Indexes10.4.3.2. Constructing a Compound Index Manually10.5. Specifying Index Options10.5.1. Creating an Index in the Background with {background:true}10.5.1.1. Killing the Indexing Process10.5.2. Creating an Index with a Unique Key {unique:true}10.5.3. Dropping Duplicates Automatically with {dropdups:true}10.5.4. Dropping an Index10.5.5. Re-Indexing a Collection10.6. How MongoDB Selects Which Indexes It Will Use10.7. Using Hint() to Force Using a Specific Index10.8. Optimizing the Storage of Small Objects10.9. Summary
11. Replication
11.1. Spelling Out MongoDB's Replication Goals11.1.1. Improving Scalability11.1.2. Improving Durability/Reliability11.1.3. Providing Isolation11.2. Drilling Down on the Oplog11.3. Implementing Single Master/Single Slave Replication11.3.1. Setting Up a Master/Slave Replication Configuration11.3.1.1. Examining the Slave11.4. Implementing Single Master/Multiple Slave Replication11.5. Configuring a Master/Slave Replication System11.6. Resynchronizing a Master/Slave Replication System11.6.1. Issuing a Manual Resync Command to the Slave11.6.2. Resyncing by Deleting the Slaves Datafiles11.6.3. Resyncing a Slave with the --fastsync Option11.7. Implementing Multiple Master/Single Slave Replication11.7.1. Setting up a Multiple Master/Slave Replication Configuration11.8. Exploring Various Replication Scenarios11.8.1. Implementing Cascade Replication11.8.2. Implementing Master/Master Replication11.8.3. Implementing Interleaved Replication11.9. Using Replica Pairs11.9.1.11.9.1.1. Setting up a Replica Pair11.9.1.2. Coping with Failure11.9.1.3. Connecting Your Application to a Replica Pair11.9.2. Resolving Server Disputes with an Arbiter11.10. Implementing Advanced Clustering with Replica Sets11.10.1. Creating a Replica Set11.10.2. Getting a Replica Set Member Up and Running11.10.3. Adding a Server to a Replica Set11.10.4. Managing Replica Sets11.10.4.1. Inspecting an Instance's Status with rs.status()11.10.4.2. Forcing a New Election with rs.stepDown()11.10.4.3. Determining If a Member is the Primary Server11.10.5. Configuring the Options for Replica Set Members11.10.5.1. Organization of the Members Structure11.10.5.2. Exploring the Options Available in the Settings Structure11.10.6. Determining the Status of Replica Sets11.10.7. Connecting to a Replica Set from Your Application11.10.7.1. Viewing Replica Set Status with the Web Interface11.11. Summary
12. Sharding
12.1. Exploring the Need for Sharding12.2. Partitioning Horizontal and Vertical Data12.2.1. Partitioning Data Vertically12.2.2. Partitioning Data Horizontally12.3. Analyzing a Simple Sharding Scenario12.4. Implementing Sharding with MongoDB12.5. Setting Up a Sharding Configuration12.5.1. Adding a New Shard to the Cluster12.6. Removing a Shard from the Cluster12.7. Determining How You're Connected12.8. Listing the Status of a Sharded Cluster12.9. Using Replica Sets to Implement Shards12.10. Sharding to Improve Performance12.11. Summary

Content preview from The Definitive Guide to MongoDB: The NoSQL Database for Cloud and Desktop Computing

Chapter 5. GridFS

We live in a world of high definition video, 12MP cameras, and storage media that can hold 50GB of data on a disc the size of a CDROM. In that context, the 4MB limit for the maximum size of a MongoDB document might seem laughably inadequate. Indeed, you might wonder why MongoDB, which has been designed as a database for today's high-tech age, has such a strange limitation. The short answer: performance.

If the data were stored in the document itself, it would obviously get very large, which in turn would make the data hard to work with. For example, pulling back the whole document would require loading the files in the document, as well. You could work around this issue, but you would still need to pull back the entire file whenever ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

The Definitive Guide to MongoDB: A complete guide to dealing with Big Data using MongoDB, Second Edition

Publisher Resources

ISBN: 9781430230519Purchase book

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

The Definitive Guide to MongoDB: The NoSQL Database for Cloud and Desktop Computing

by Eelco Plugge, Peter Membrey, Tim Hawkins

Chapter 5. GridFS

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.