Last year at MongoDB World 2019, Eliot announced that MongoDB Atlas Data Lake was a new tool available in beta in the MongoDB Cloud Platform.

During this last year, MongoDB has been working closely with many customers to test this new tool and gathered much feedback to make it even better.

Today, after a year of refinement and improvement, MongoDB is proud to announce that MongoDB Atlas Data Lake is now generally available and can be used with confidence in your production environment.

In this tutorial, I will show you a new feature of MongoDB Atlas Data Lake called Federated Querythat allows you to access your archived documents in S3 AND your documents in your MongoDB Atlas cluster with a SINGLE MQL query.

This feature is really amazing because it allows you to have easy access to your archived data in S3 along with your “hot” data in your Atlas cluster. This feature could help you prevent your Atlas clusters from growing in size indefinitely and reduce your costs drastically. It also makes it easier to gain new insights by easily querying data residing in S3 and exposing it to your real-time app.

Finally, I will show you how to use the new version of the $out aggregation pipeline stage to write documents from a MongoDB Atlas cluster into an AWS S3 bucket.

Prerequisistes

In order to follow along this tutorial, you need to:

• create at least an M10 cluster in MongoDB Atlas,
• create a user in the Database Access menu,
• add your IP address in the IP whitelist in the Network Access menu,
• setup your MongoDB Atlas Data Lake environment in MongoDB Atlas,
• have python 3 with pymongo and dnspython libs installed,
• you will also need to have a Mongo Shell installed or maybe the new Mongosh.

If you did these actions correctly, you should have an M10 (or bigger) cluster running in your MongoDB Atlas project.

And your Data Lake page should look like this:

#mongodb