Querying collections efficiently

When retrieving large result sets from a MongoDB collection, the same rule that applies to MySQL database result sets also applies here: reading the complete result set into our Node.js process at once isn't going to be efficient resource-wise. Handling a result array with 2,000,000 entries will eat a lot of memory no matter what. This is what the toArray method, which we used until now, does:

collection.find().toArray(function (err, documents) { 
  // We now have one large documents array 
});

What collection.find() returns is a so-called cursor object. It can be transformed into an array, which is very convenient, but when handling lots of documents, it's better to handle each one separately using the cursor's ...

Get The Node Craftsman Book now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.