How to make exact (development) copy of a production installation of ERPNext v10

Frappe Framework
1

Hi everyone,

I’ve been trying to “copy” an existing, working production setup:

  • Ubuntu 16.04
  • ERPNext V10.05
  • Installed using the easy install script install.py

I’ve been working on this setup directly, made a custom app where I’ve added about a dozen custom doctypes already and several more custom fields in the native doctypes. Now, we have users already using the system for actual business work.

So I need to make a copy on a separate server for further development. And I need all the data in the database and all my custom code to be brought into this dev server. The problem is I haven’t quite figured out yet how the export customizations and export fixtures really work. And also, I’m pretty sure there have been customizations to native doctypes that I forgot to export.

So, doing a fresh install on the dev server and restoring a backup of the production data into it, means I left some, if not all, of my custom code. I’ve only transferred the database content. I’ve already tried this option and it didn’t work.

The approach I found that gets me closest to what I need to do was this “Reverting” that @James_Robertson described in his excellent documentation here. It’s basically replacing the entire bench folder of a fresh working installation with a copy of the bench folder that you want to have.

Since it is a production setup that I’m trying to copy, the first hurdles I had to overcome were problems with nginx. I got them sorted now, after a couple of days troubleshooting. The webserver is not refusing anymore.

Now, my problem is in the application level. I’m still getting this Internal Server Error. There are no errors in the nginx logs. The only errors I found related to this are the following:

/home/frappe/frappe-bench/logs/web.error.log:

[2018-01-23 12:28:26 +0000] [1974] [ERROR] Error handling request /
Traceback (most recent call last):
  File "/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 135, in handle
	self.handle_request(listener, req, client, addr)
  File "/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request
	respiter = self.wsgi(environ, resp.start_response)
  File "/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/werkzeug/local.py", line 228, in application
	return ClosingIterator(app(environ, start_response), self.cleanup)
  File "/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/werkzeug/wrappers.py", line 308, in application
	resp = f(*args[:-2] + (request,))
  File "/home/frappe/frappe-bench/apps/frappe/frappe/app.py", line 88, in application
	response = handle_exception(e)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/app.py", line 187, in handle_exception
	make_error_snapshot(e)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/error.py", line 21, in make_error_snapshot
	if frappe.conf.disable_error_snapshot:
  File "/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/werkzeug/local.py", line 347, in __getattr__
	return getattr(self._get_current_object(), name)
  File "/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/werkzeug/local.py", line 310, in _get_current_object
	raise RuntimeError('no object bound to %s' % self.__name__)
RuntimeError: no object bound to conf
[2018-01-23 12:28:26 +0000] [1975] [ERROR] Error handling request /favicon.ico
Traceback (most recent call last):
  File "/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 135, in handle
	self.handle_request(listener, req, client, addr)
  File "/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request
	respiter = self.wsgi(environ, resp.start_response)
  File "/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/werkzeug/local.py", line 228, in application
	return ClosingIterator(app(environ, start_response), self.cleanup)
  File "/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/werkzeug/wrappers.py", line 308, in application
	resp = f(*args[:-2] + (request,))
  File "/home/frappe/frappe-bench/apps/frappe/frappe/app.py", line 88, in application
	response = handle_exception(e)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/app.py", line 187, in handle_exception
	make_error_snapshot(e)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/error.py", line 21, in make_error_snapshot
	if frappe.conf.disable_error_snapshot:
  File "/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/werkzeug/local.py", line 347, in __getattr__
	return getattr(self._get_current_object(), name)
  File "/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/werkzeug/local.py", line 310, in _get_current_object
	raise RuntimeError('no object bound to %s' % self.__name__)
RuntimeError: no object bound to conf

My nginx config file looks like this:

upstream frappe-bench-frappe {
	server 127.0.0.1:8000 fail_timeout=0;
}

upstream frappe-bench-socketio-server {
	server 127.0.0.1:9000 fail_timeout=0;
}

server {
	listen 80;
	server_name dev-erp.mydomain.com;
	root /home/frappe/frappe-bench/sites;

	location /assets {
		try_files $uri =404;
	}

	location ~ ^/protected/(.*) {
		internal;
		try_files /dev-erp.mydomain.com/$1 =404;
	}

	location /socket.io {
		proxy_http_version 1.1;
		proxy_set_header Upgrade $http_upgrade;
		proxy_set_header Connection "upgrade";
		proxy_set_header X-Frappe-Site-Name dev-erp.mydomain.com;
		proxy_set_header Origin $scheme://$http_host;
		proxy_set_header Host dev-erp.mydomain.com;

		proxy_pass http://frappe-bench-socketio-server;
	}

	location / {
		try_files /dev-erp.mydomain.com/public/$uri @webserver;
	}

	location @webserver {
		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
		proxy_set_header X-Forwarded-Proto $scheme;
		proxy_set_header X-Frappe-Site-Name dev-erp.mydomain.com;
		proxy_set_header Host $host;
		proxy_set_header X-Use-X-Accel-Redirect True;
		proxy_read_timeout 120;
		proxy_redirect off;

		proxy_pass  http://frappe-bench-frappe;
	}

	# error pages
	error_page 502 /502.html;
	location /502.html {
		root /home/frappe/.bench/bench/config/templates;
		internal;
	}

	# optimizations
	sendfile on;
	keepalive_timeout 15;
	client_max_body_size 50m;
	client_body_buffer_size 16K;
	client_header_buffer_size 1k;

	# enable gzip compresion
	# based on https://mattstauffer.co/blog/enabling-gzip-on-nginx-servers-including-laravel-forge
	gzip on;
	gzip_http_version 1.1;
	gzip_comp_level 5;
	gzip_min_length 256;
	gzip_proxied any;
	gzip_vary on;
	gzip_types
		application/atom+xml
		application/javascript
		application/json
		application/rss+xml
		application/vnd.ms-fontobject
		application/x-font-ttf
		application/font-woff
		application/x-web-app-manifest+json
		application/xhtml+xml
		application/xml
		font/opentype
		image/svg+xml
		image/x-icon
		text/css
		text/plain
		text/x-component
		;
		# text/html is always compressed by HttpGzipModule
}

My site_config.json looks like this:

/home/frappe/frappe-bench/sites/dev-erp.mydomain.com/site_config.json

{
 "db_name": "dbdbdbdbdbdbdbd", 
 "db_password": "dpdpdpdpdpdpdpd", 
 "developer_mode": 1, 
 "encryption_key": "-nnGqasoawehoBAKSBlbLSdflsdflasbglsdgl=", 
 "limits": {
  "space_usage": {
   "backup_size": 6.0, 
   "database_size": 24.6, 
   "files_size": 6.0, 
   "total": 36.6
  }
 }

I have a multi-tenant setup, so the currentsite.txt is empty.

I’ve run out of ideas on where to look and what to look for. I’d greatly appreciate any help.

2

Starting the web server in development and production are totally different.
nginx and gunicorn are not used in development.

You should do a bench init frappe-bench, this will be by default set up as development bench for you. Then, you can restore your database from your production setup.

To copy customizations, you can Import/Export your Custom Field DocType or/and export fixtures. Please go through the following articles to know more.

https://frappe.io/docs/user/en/guides/app-development
https://frappe.io/docs/user/en/guides/app-development/exporting-customizations

3

Thanks for the reply @netchampfaris. I’ve already read those pages before. But what I’m not clear about, is how to export all the customizations I’ve made when if I forgot to export them when I applied them.

What I’m saying is I can’t remember anymore all the native doctypes I’ve customized. Is there a way to find out which doctypes have been applied with customizations?

From what I’ve seen so far, Export Customizations button is only available in the Customize Form page. And from there, I’ll have to select which particular doctype I want to customize or whose customizations I want to export.

Also, as to the dev setup, I think what I want really is not to have a “development bench” I just want to have an exact same setup I have in the production. I want everything to be the same so that I know I’m working on something that’s not the production, but is exactly the same as the production.

So, I actually want nginx and gunicorn and everything else in there. :laughing:

How do you recommend I should go about this?

Or, if this is not a good idea, I’m also interested to know the disadvantages to this approach.

4

It’s reason why I’m using Docker :slight_smile:

5

The concept behind Docker is great. Unfortunately for me, I didn’t start with a Dockerized original installation. And what I’m trying to do now is to reproduce that original instance in an Oracle Virtualbox in my laptop running pretty much the same environment as that original instance.

6

Custom Field contains all the custom field you have added through customize form. Property Setter contains all the modifications you have done to Standard fields.

Development bench is more suited to quick feedback, changes you make to .py and .js files will be reloaded automatically, you just have to refresh your browser to see changes.

In production setup, you have to bench restart everytime you make a change. If you are not making code changes, then having this setup should be fine.

In that case, after you have initialized a development bench setup, just run the command sudo bench setup production

Hope that clears your doubts.

1 Like
7

Oh, there it is! It’s the Property Setter that I was really looking for. (I’ve been looking at the Custom Field list before.)

I see. Well, I haven’t touched any server-side code so far - I was only adding client-side custom scripts. So, the Reload option from the menu bar did the job for me so far. But yes, I’m also looking at adding server-side functionalities soon, so I think I’d better start to figure out how to properly setup a development instance.

Thanks for the explanation @netchampfaris.