Need a better understanding of MultiTenant configurations

ERPNext
1

Hello all. If you have read any of my past How-To posts, you know I take the details of setting up ERPNext very seriously. And now I am starting to work on how to setup and configure ERPNext in the MulitTenant mode. This process may take me s few weeks to get worked out, but whne I do, I will publish another Step-by-Step guide and a list of important things to understand before starting.

In order to get there, I need a better understanding of how a multi-tenant configuration operates differently from, and similarly to, a free-standing single entity system that we are used familiar with now.

So I have some questions so I can get my head wrapped around this:

  • When operating multi-tenant, does MariaDB run multiple iterations of itself or is it a single running instance and all data across all tenants is in a single database set?

  • Can a tenant of a multi-tenant installation expect to get a backup of only their part of the system (as if it were a stand alone environment) or is their backup data lumped into a single backup file with the rest of the tenants? How are backups handled or are those functions disabled (like setting up Dropbox Backup)?

  • In understand that the multi-tenant system must have a central system manager (or service provider) like myself to manage things like backing up data, and root access server side maintenance. How do these activities differ from the same activities on a single entity server?

  • Will the multi-tenant system be able to support users of the same exact name and email address across multiple tenants of the same system? For example, a hair stylist or a tattoo artist that rent chairs (workspace) in multiple private businesses (salons) that may exist on the same multi-tenant server. They are independent operators and work in several locations, but they all use their own email address as their identity.

  • Does multi-tenancy have some sort of firewall between the accounting data of the individual tenants, or would there be any crossover of some sort?

  • When a tenant uses the Data Import tool to download templates that include their data, would it always be only their data, or is there any crossover in this function?

  • In a multi-tenancy system, there would likely be a need to do server maintenance at times. Is there a way to force the logout of all users on all tenant sites to logout so the server can be rebooted without causing data issues? This would seem to be a very necessary element for systems with 4 or 5 sites running.

  • Are there any helpful guidelines for setting a multi-tenant system up such as requires swap space per tenant, additional RAM memory overhead per tenant, operating system settings that need to be altered to support the configuration, etc.?

  • Does each tenant site have a separate Administrator User, or is there only one for the system? If there is only one for the system, which site would the admin use to login?

  • If a tenant grows their business to the point they are putting an excessive burden on the server for the other tenants, how could they be moved to their own stand alone version of ERPNext with their own server? Would their data be segregated enough to pull it out and easily set up their new server?

  • There is a function in the stand alone version of ERPNext that allows you to discard all transactions ever made in the system to essentially wipe the slate clean after a testing period and start actually running your business with real data. Is this function available to a tenant as well? Or would it wind up wiping out all transactions across all tenants?

  • Would custom fields, or custom apps be per tenant or would they have to apply across all tenants?

  • If multiple tenants the same server wanted to also setup their own online shops to sell their products online, would they each be able to select their own integrations or would they conflict with each other? Would it even be possible to setup multiple online stores (one for each tenant)?

  • Would tenants be able to use the standard integrations in the Integration section of Setup to add things like Strip, PayPal, GSuite, OAuth and so on, or would they possibly interfere with each other (from a tenant perspective)?

  • Would tenants be able to setup different email settings for themselves or would all tenants on the same server have to be on the same email system?

  • When setting up a single server on a VPS service provider, would a multi-tenant ERPNext installation need a separate IP address added for each tenant site, or could it all be run from one IP address and let the DNS records of the URL providers just point to the right place? Can anyone point me to documentation on how to set this up?

  • In the event of trouble in paradise for a single tenant, can bench be restarted for a single site without causing issues with other sites on the same multi-tenant setup? What are the limitations of dealing with single site errors when it is part of a multi-tenant server?

I am sure there will be other questions after I find answers to the ones I have listed above. Stay tuned or bookmark this post if you also want to know about this topic.

I hope to be able to help others figure out this process as I learn myself. I think the multi-tenant configuration has the potential to introduce many new businesses to the power of ERPNext. Just imagine if you used this model to maybe give one small business a site to get started for the first months while the other 4 older tenants fund the rest of the server.

Anyway, I see potential to use this to expand our user base. If tenants of a multi-tenant server grow enough to need their own single entity server, then we have expanded our world by another successful business story. I find that both exciting and challenging.

So I need your help understanding how this configuration works and how to get it up and running. I promise to then publish the Step-by-Step thread for this like I did here:

Thank you in advance to anyone with knowledge around this topic that helps me to help others.

BKM

8 Likes
2

Wowā€¦ I cannot be the only one that wants to know this stuff.

Are there no experienced people here with implementing multi-tenant systems?

BKM

3

Iā€™m a noob here and trying to learn the ins and outs. Iā€™ll try to answer what I know, or think I know, and hopefully I will be corrected if wrong and then we both can learn :slight_smile:

When operating multi-tenant, does MariaDB run multiple iterations of itself or is it a single running instance and all data across all tenants is in a single database set?

From my understanding, one instance, but each site (tenant) has their own database and its own credentials to access their data.

Can a tenant of a multi-tenant installation expect to get a backup of only their part of the system (as if it were a stand alone environment) or is their backup data lumped into a single backup file with the rest of the tenants? How are backups handled or are those functions disabled (like setting up Dropbox Backup)?

Each site/tenant has their own backup and restore procedure. It can be a sql dump or a sql dump with files. Each site can be configured to export the back up at different times. When inside an indiviual site, you can go to ā€œexploreā€ => ā€œintegrationsā€ and have the files for the site (sql dumps and images) sent to Dropbox or Amazon S3. I believe the dumps are placed on the local server at:

 ~/frappe-bench/sites/[sitename]/private/backups/

In the case you wanted to move the backups to a different location yourself.

In understand that the multi-tenant system must have a central system manager (or service provider) like myself to manage things like backing up data, and root access server side maintenance. How do these activities differ from the same activities on a single entity server?

Iā€™m not sure if I understand the question correctly, buy the way I look at it, is this is no different than managing 1 server with multiple sites. The server itself is serving all the sites, so managing the actual server is the same as any other web server. OS updates/patches etc are all managed the same. Each site is its own entity, with the exception of the shared apps ex: frappe, erpNext etc, when those are updated/upgraded, all of the sites get whatever version of the app the server has been upgraded to. This may not be correct if you are running multiple benches on a single server (Iā€™m assuming, need someone to verify and explain more on this as this is something I have yet to explore.

Will the multi-tenant system be able to support users of the same exact name and
email address across multiple tenants of the same system? For example, a hair stylist or a tattoo artist that rent chairs (workspace) in multiple private businesses (salons) that may exist on the same multi-tenant server. They are independent operators and work in several locations, but they all use their own email address as their identity.

if I understand the question correctly, wouldnā€™t each saloon/parlor get a ā€œsiteā€ and each stylist/artist get a login to the the saloon/parlor site for when they are working at the given location? In this case, yes, one email could be the email address login at multiple erpNext sites, however, each site has an independent database, so passwords would not sync up.

If you go the route, where each artist/stylist has thier own erpNext site, then each parlor/saloon would be a location in which the artist has set up in thier site to track timesheets against etc.

Does multi-tenancy have some sort of firewall between the accounting data of the individual tenants, or would there be any crossover of some sort?

erpNext has roles and permissions, each user can only see what thier roles and permissions are set to. With this being said, one member of one erpNext site could never see another sites data as mentioned above, they are held in seperate databases, with different mariadb username and passwords.

When a tenant uses the Data Import tool to download templates that include their data, would it always be only their data, or is there any crossover in this function?

All data is restricted to the respective site. Each site contains a seperate database and has no access to the other sites databases. Each site has a unique autogenerated mariadb username and password. When downloading data, the only data you have access to is data on the given site you are logged into, and only the data on that site that you have been given permission to view.

In a multi-tenancy system, there would likely be a need to do server maintenance at times. Is there a way to force the logout of all users on all tenant sites to logout so the server can be rebooted without causing data issues? This would seem to be a very necessary element for systems with 4 or 5 sites running.

Not 100% sure, but perhaps:

sudo supervisorctl stop all

to start back up

sudo supervisorctl start all

Are there any helpful guidelines for setting a multi-tenant system up such as requires swap space per tenant, additional RAM memory overhead per tenant, operating system settings that need to be altered to support the configuration, etc.?

I have not found one yet, perhaps someone can post some links, I would be interested in knowing this as well.

Does each tenant site have a separate Administrator User, or is there only one for the system? If there is only one for the system, which site would the admin use to login?

Each site has a unique Administrator, when setting up a new site, you will be prompted to enter a password for the administrator of the new site you are setting up.

If a tenant grows their business to the point they are putting an excessive burden on the server for the other tenants, how could they be moved to their own stand alone version of ERPNext with their own server? Would their data be segregated enough to pull it out and easily set up their new server?

Since each site has its own database, itā€™s as simple as backing up the database and images and restoring to the new server instance running erpNext. Bench has commands for these processes. See:

https://frappe.io/docs/user/en/bench/resources/bench-commands-cheatsheet

There is a function in the stand alone version of ERPNext that allows you to discard all transactions ever made in the system to essentially wipe the slate clean after a testing period and start actually running your business with real data. Is this function available to a tenant as well? Or would it wind up wiping out all transactions across all tenants?

Again, all data is segregated by utilizing separate databases with unique credititials. When working with one site, you are only working on that siteā€™s data. So, it will only access and delete data in the database for the site being used.

Would custom fields, or custom apps be per tenant or would they have to apply across all tenants?

Per tenant, you control which apps get installed to which sites, custom fields etc live in the database, and each site has a unqiue database.

If multiple tenants the same server wanted to also setup their own online shops to sell their products online, would they each be able to select their own integrations or would they conflict with each other? Would it even be possible to setup multiple online stores (one for each tenant)?

Each tenant has the capability of online stores, again given the structure and path the erpNext Devs have followed, from the ground up, erpNext has been built with Multi Tenant in mind.

Would tenants be able to use the standard integrations in the Integration section of Setup to add things like Strip, PayPal, GSuite, OAuth and so on, or would they possibly interfere with each other (from a tenant perspective)?

All these integrations are unique per site. No interference to worry about.

Would tenants be able to setup different email settings for themselves or would all tenants on the same server have to be on the same email system?

Not sure I understand the questions. Emails as log ins can be whatever the user and the site admin allows. If you are referring to the system checking and sending emails, you can set up multiple email domains and accounts per site.

When setting up a single server on a VPS service provider, would a multi-tenant ERPNext installation need a separate IP address added for each tenant site, or could it all be run from one IP address and let the DNS records of the URL providers just point to the right place? Can anyone point me to documentation on how to set this up?

Single IPā€¦ You can do either port based or DNS based Multitenant-Setup. I do DNS based. Read more at:

In the event of trouble in paradise for a single tenant, can bench be restarted for a single site without causing issues with other sites on the same multi-tenant setup? What are the limitations of dealing with single site errors when it is part of a multi-tenant server?

I believe all sites would be ā€œbouncedā€ if you restarted bench. However, I should defer to someone with more knowledge on this topic.

I hope I helped and gave correct information. As stated, still pretty new here and trying to come up to speed myself.

8 Likes
4

More : Home Ā· frappe/frappe Wiki Ā· GitHub

8 Likes
5

A big Thank You goes out to @revant_one for pointing out this valuable resource.

:tada: :confetti_ball: :tada:

Hopefully this thread will be useful to others trying to understand the MultiTenant concept much like I was at the time of the posting.

BKM

2 Likes
6

@bkm

Let me clarify on some things

You donā€™t log users out, instead use ā€œMaintenance Modeā€ to prevent users from accessing the site. Use it by doing bench set maintenance-mode on, of course turn it off once youā€™re done changing things. You must do this per-site

sudo supervisorctl stop all works, but itā€™s not the right way to go about it. That will just stop all the processes.

Unfortunately there is no exact metric on this, it really depends. Therefore itā€™s best to have an architecture that is easily scalable. Use a cloud database service, we use Amazon RDS.

There is an option in the ā€œCompanyā€ DocType to do that. However, from a sysadmin point of view, you can simply ā€œreinstallā€ the site by doing bench --site <site-name> reinstall. This will destroy all data.

You can actually setup a configuration in common_site_config.json to make all the sites use the same email configuration, like this : frappe/test_sites/test_site/site_config.json at c1e5e5dabef21969436ec89f931dd5a3850d6135 Ā· frappe/frappe Ā· GitHub. But each site can have their own configuration as well.

Each site uses the same app code. You canā€™t restart a single app, you can however ā€œmigrateā€ the database schema for a single app. But there is only some use cases where you would want to just do that.

6 Likes
7

@bkm We are waiting for the step by step if you have acquired the knowledge.Thanks in advance.
Fred

8

Thanks for the bump Fred. It is on my list of things to do, but I have been swamped with client work lately and havenā€™t had much time.

BKM

1 Like
9

I have followed the exact steps shown

But it is not working under google cloud.
I have tried port multi tenancy.

Any one has same issue on V11?

10

I donā€™t think it will work on GCP (Google Cloud Platform). That environment is predominantly a development environment and only very few ports are even open to be able to use.

GCP intentionally locks most common ports for email, alternate port websites, and even connections to other outside servers. This (they say) is to prevent people spinning up a server quickly and using it to hack other sites, or send out spam.

To this end, I do not think multi-tenant will work on GCP unless you have contacted them and requested all ports to be opened.

I think the only ports that are open by default are 22, and 80. Anything else requires you to get them to loosen the restrictions. BTWā€¦ you would never know the ports are closed to you unless you had an outside service to verify it. There is a process you can do to get some ports open, but it is tedious and requires youto learn your way through their administration panels and forms. Even then, you may not be able to open certain important ports (like email ports, and SMS messaging ports, etc.).

This has been my experience with them last Feb (2018) and I just gave up and started using some of the KVM VPS servers I found on ā€˜lowendbox.comā€™ in order to get all port access and much fewer restrictions.

I never went back to GCP except to do new app testing or new versions testing. I do everything else now on cheap cloud servers.

So, if you are having trouble with port related stuff not working, you may want to start looking through the account settings in GCP to see what is available to you this year. I have not looked in quite a while.

Your Mileage May Varyā€¦

BKM

11

I do all steps in contabo vps cloud but i get page not found when i try to open ip for the host can you help me to solve it