Mongodb multi tenant implementation

By using our site, you acknowledge that you have read and understand our Cookie PolicyPrivacy Policyand our Terms of Service. The dark mode beta is finally here. Change your preferences any time. Stack Overflow for Teams is a private, secure spot for you and your coworkers to find and share information.

I'm thinking of creating a multi-tenant app using MongoDB. I don't have any guesses in terms of how many tenants I'd have yet, but I would like to be able to scale into the thousands. I have the same problem to solve and also considering variants. As I have years of experience creating SaaS multi-tenant applicatios I also was going to select the second option based on my previous experience with the relational databases.

The guys stated to avoid 2nd options at any cost, which as I understand is not particularly specific to mongodb. Collections or buckets or however they call it in different DBs are not the same thing as security schemas in RDBMS despite they behave as container for documents they are useless for applying good tenant separation.

I couldn't find NoSQL database that can apply security restrictions based on collections. If you post additional details about your application, perhaps I can give you more detailed advice.

We decided not to have a database per customer because of the way MongoDB allocates its data files. The first file for a database is dbname. Once the files reach 2GB in size, each successive file is also 2GB. We found this used a massive amount of disk space compared to having several databases for all customers where the disk space can be used to maximum efficiency.

Sharding will be on a per collection basis as standard which presents a problem where the collection never reaches the minimum size to start sharding, as is the case for quite a few of ours e. However, we have requested that this will also be able to be done on a per database level. There are no performance tradeoffs using lots of collections.

There is a reasonable article on MSDN about multi-tenant data architecture which you might wish to refer to. Some key topics touched on by this article:. Additionally, worth a gander is an interesting write-up from the SQL Anywhere guys. That said I'm no pro with MongoDB. I get pretty nervous using a shared "schema" - but I will happily defer to more experienced practitioners.Comment 4.

This tutorial will guide you in the process to connect to multiple MongoDBs. The sample project uses Project Lombok to generate getters, setters, contructors, etc. To create a MongoTemplate instance through Spring Boot, we only need to provide Mongo Server details in Spring Boot specific property keys and Spring Boot on startup automatically creates a Mongo Connection with MongoTemplate wrapper and lets us auto wire wherever we want to use.

The MongoProperties. Then there is another class called as MongoDataAutoConfiguration. It can be achieved just by adding the following property in the application. Now, we already have the properties configuration classes and data model classes, so we need to configure which MongoTemplate will be responsible for each defined repository. To test the application, I just added the following code to the main class in the project.

As this project was built using Spring Bootto run it, just execute the following commands:. This guide requires a MongoDB instance up and running in localhost and port. See the original article here. Over a million developers have joined DZone. Let's be friends:. DZone 's Guide to. Let's take a look at a tutorial that explains how to build an application that connects to multiple MongoDBs.

Also look at creating the data model and repositories. Free Resource. Like 3. Join the DZone community and get the full member experience. Join For Free. Prerequisites JDK 1.

Below are the basic properties required for creating a MongoTemplate. MongoProperties; import org. ConfigurationProperties; import lombok.We are all building apps in the cloud. Accounting apps, to-do apps, word processing apps everything. With clients trusting you with their data, the question becomes how to keep their data safe and separate. This is called multi-tenancy. Multi-tenancy refers to a principle in software architecture where a single instance of the runs on a server, serving multiple customers tenants.

This is a really important part of an important feature of cloud computing. There are three main ways to build multi-tenant databases in MongoDB. The first is by putting all tenants in a single database. The second is putting each tenant in their own individual database, and finally, each tenant in its own collection.

This is the most common form of multi-tenancy and where most web apps start. Putting all your tenants together is a lot simpler and we do not even think about calling it multi-tenancy. Putting all tenants in a single database requires putting the multi-tenancy logic into the application level.

Enforcing security at the application level can be something as simple as placing an enforcing user or customer filters on all data queries, eg. Creating hundreds of databases for hundreds of non paying customers can waste a lot of resources.

This is probably the worst way for MongoDB for a couple of reasons. First, Collections in the same database share the same database Lock.

MongoDB concurrency has been steadily improving, but it is still there. Second, the default MongoDB nssize setting limits the number of collections in a database to 24, You can go up to 3 million by changing the nssize setting in the configuration. The may be the best way depending on your app. Giving each tenant their own databases gives you flexibility in managing and optimizing your MongoDB setup.

By having a separate database per customer, things like great for moving, managing and deleting client databases become trivial. Since each database is separate, you can create different indexes for different clients depending on their needs. The downside to this is that each client takes space.

If your clients are paying customers, this is not a problem. If you have a free service, then each client will use 32 MB of disk space which is quite a lot if you have a lot of inactive clients.

Even with multi-tenancy, it can be hard to pick a shard key. The hashed shard key in MongoDB can provide performance even at scale depending on your application.

Subscribe to RSS

For most things, performance is application specific. Especially with MongoDB, and the advice here needs to be seen in the light of your application. As always, your mileage may vary. Wou are starting an app, for development, you can certainly use one large database while writing your app logic to support One database per tenant.

And one last thing. Shard Early Picking a shard key is something that is hard to change later.Sign In. NET Cincluding modeling for data integrity in a world without joins or transactions, leveraging dynamic schema elements while using the C driver, embracing sharding, ensuring scalability and security, using the Aggregation framework, and working around limitations case sensitivity, numeric precision.

Learn more. Atlas Fully managed cloud database. Pricing Simple, flexible, pay as you go. Atlas Search Simple, integrated search capabilities beta. Stitch Serverless platform. Charts Native visualization for MongoDB data. Community Server A free and open document database. Enterprise Server Advanced features and security. Connectors Easy integrations to your data estate. What is MongoDB? Easy integrations to your data estate. University Free online courses from MongoDB.

Blog Updates, tutorials, people. Developer Hub Developer best practices, trends, insights. Resources Webinars, white papers, datasheets, and more. Training Instructor-led sessions on your schedule. Events Worldwide community events. Customers Who uses MongoDB. Consulting Accelerate success with MongoDB. Partners Find or become a partner. Cloud Atlas, Stitch, and more. Server The database.

Drivers Language APIs.By using our site, you acknowledge that you have read and understand our Cookie PolicyPrivacy Policyand our Terms of Service. The dark mode beta is finally here. Change your preferences any time. Stack Overflow for Teams is a private, secure spot for you and your coworkers to find and share information.

mongodb multi tenant implementation

Im able to create a connection pool to mongodb without specifying any database using the mongo-native driver like so:. Going by the presentation, looks like this was possible to do with older versions of the Java driver.

mongodb multi tenant implementation

Is it even possible for me to use a single connection pool and authenticate tenants to individual databases using the same connections? The alternative we have is to have a connection pool per tenant, which is not attractive to us at this time.

Yeah that doesn't work any more, they killed this magnificent feature like 6 months after we implemented it and we were out with it in Beta at the time. We had to change the way we work. It's a shame since till this day, in Mongo, "connection" network stuff, SSL, cluster identification and authentication are 2 separate actions.

Think about when you run mongo shell, you provide the host, port, replica set if any, and your in, connected! But not authenticated. You can then authenticate to user1, do stuff, and then authenticate to user2 and do stuff only user2 can do. And this is done on the same connection! Back then, the driver let us have a connection pool of "blank" connections that we could authenticate at will to the current tenant in context of that current execution thread.

Then they deprecated this capability, I think it was with Mongo 2. Now they only supported connections that are authenticated at creation. We asked enterprise support, they didn't say why, but to me it looked like they found this way is not secured, "old" authentication may leak, linger on that "not so blank" reusable connection.

We made a change in our multi-tenancy infra implementation, from a large pool of blank connections to many small pools of authenticated connections, a pool per tenant. These pools per tenant can be extremely small, like 3 or 5 connections. This solution scaled nicely to several hundreds of tenants, but to meet thousands of tenants we had to make all kinds of optimizations to create pools as needed, close them after idle time, lazy creation for non-active or dormant tenants, etc.

This allowed us to scale even more We're still looking into solutions and optimizations. You could always go back to a global pool of authenticated connections to a Mongo user that have access to multiple databases. Yes, you can switch database on that same authenticated connection.

You just can't switch authentication. This is an example of pure Mongo Java driver, we used Spring which provide similar functionality:.By using our site, you acknowledge that you have read and understand our Cookie PolicyPrivacy Policyand our Terms of Service.

Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It only takes a minute to sign up. I'm working on the architecture for my upcoming Project Mangement app as an example and I'm seeking clarity on how best to design the MongoDB data layer, with specific regard to multi-tenancy.

Multiple MongoDB Connectors With Spring Boot

The app will have multiple 'sub-apps' e. My current working strategy is to use one database to store the higher-level, 'global' data like Users, Notifications, Messages, Usage, and Preferences.

And then create a new database for each project created on the system. This seems like the ideal approach for many reasons: security each DB has its own credscatastrophic recovery since if one DB Server goes down the entire app doesn't go downand performance I think, since Mongo would have to search far fewer docs to find the one it's looking for.

The application would contain logic that automatically detects available space on any given DB Server and creates the new Project database on the next available DB Server.

According to this article provide by MongoHQ, this is the 'best' strategy, although it consumes a large amount of storage. Especially since each DB takes up 32M even when it's empty. Which gets very expensive using a service like MongoHQ if you're offering a 'Freemium' app that gets Techcrunch'd. So in a scenario where ProjectManager has three projects on the system my data layer would look like so:.

mongodb multi tenant implementation

Each one storing about documents each at most. Given that, there are a few design patterns to follow as you build your system right now. These are items that will make scaling later on easier depending on what you learn along the way. Shard now This forces you to start thinking about good shard keys, since the shard key is a piece of your non-schema that is hard to change down the road. You're not sharding for performance at this point, you're sharding now to ensure your code can handle it and to boot the performance question down the road a few milestones.

Engineer multi-database support in now If you expect you'll need more than one database, or even more than one cluster of Mongo databases, building in data-locality at the early stages will ease putting it in later.

Also, compaction takes nodes offline for a while which makes it a potentially significant impact to normal operations. Also, you don't have to worry about compaction if you never delete documents. Know what you get with separated Collections and separated Databases Collections in the same database share the same Database Lock, which they've been steadily whittling down with each version of MongoDB.

Indexes matter a lot If you don't have enough RAM to keep at least your indexes in memory, performance will be really bad. Depending on how large you get, you may end up sharding or splitting collections in order to get indexes that will fit into RAM again. This is one area where multi-tennancy can be an issue; if you've got a few large, unused tenants in a single collection all those indexes have to be kept in RAM to make the whole system run.

If you split collections based on tenant then the unused indexes can get paged out with little penalty. Sign up to join this community. The best answers are voted up and rise to the top. Home Questions Tags Users Unanswered. How to create an efficient, scalable, multi-tenant data layer using MongoDB?

Asked 6 years, 1 month ago. Active 6 years, 1 month ago. Viewed 9k times. The Questions Should I use one giant, centralized database to store all the application data, or create an individual database for each Project that is created on the system?

If I choose the individual DB strategy, does that obviate the need to shard the data layer given that the DB's are 'naturally' dispersed across several servers, thus 'naturally' spreading the load across several servers? The application would contain logic that tells it which server to access the data for any given Project.

Would using individual DB's for each Project give me better performance given that to find any given document, Mongo would only have to search at most a few thousand docs in the individual Project DB vs.GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.

If nothing happens, download GitHub Desktop and try again. If nothing happens, download Xcode and try again.

mongodb multi tenant implementation

If nothing happens, download the GitHub extension for Visual Studio and try again. This is where the magic happens. That's all there is to it. How you do this may depend on where you are utilizing the same.

As an example scenario i have implemented this in a Spring MVC app. RequestContextHolder is thread safe. We inject the tenantId somewhere during the request life cycle and extract it in the getDb method.

I also have added a default Db. Skip to content. Dismiss Join GitHub today GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together. Sign up. Simple database based multi tenancy using spring data and mongoDB. Java Branch: master. Find file. Sign in Sign up. Go back. Launching Xcode If nothing happens, download Xcode and try again. Latest commit Fetching latest commit….

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window.

Replies to “Mongodb multi tenant implementation”

Leave a Reply

Your email address will not be published. Required fields are marked *