Need help in improving development installer

Hi everyone,

We have a new installer for setting up a development environment and would love to get your help on improving it. For now we have tested it on Mac OS X and Ubuntu 14.04+

Here’s the help: GitHub - frappe/bench at master

The centos dev setup is a WIP (I gave up on it for the time being due to other priorities)
and production setups using ansible is pending.

Maybe some of you can pitch in to fix issues and/or develop it further for missing OSes.
Check this folder to get an idea: bench/playbooks at master · frappe/bench · GitHub

Best,
Anand.

6 Likes

@anand I tried it it was successfully installed but ~/frappe/frappe-bench folder does not exist bench command not found

but I have only this /usr/local/frappe/bench-repo. Any tips?

Im using ubuntu 16.04

Any error during install?

@anand no errors but It did not install this part

Runs the Ansible playbook ‘playbooks/develop/install.yml’, which:
Installs
MariaDB and its config
Redis
NodeJS
WKHTMLtoPDF with patched QT
Initializes a new Bench at ~/frappe/frappe-bench with frappe framework already installed under apps.

I run again the command again this is the log

chris@chris:~$ python install.py
[sudo] password for chris: 
Ign:1 http://dl.google.com/linux/chrome/deb stable InRelease                   
Hit:2 http://dl.google.com/linux/chrome/deb stable Release                     
Hit:4 http://ppa.launchpad.net/jtaylor/keepass/ubuntu xenial InRelease         
Hit:5 http://archive.canonical.com xenial InRelease                            
Hit:6 http://apt.insynchq.com/ubuntu xenial InRelease                          
Hit:7 http://security.ubuntu.com/ubuntu xenial-security InRelease        
Get:8 http://us.archive.ubuntu.com/ubuntu xenial InRelease [116 kB]      
Hit:9 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease             
Hit:10 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease
Get:11 http://us.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages [7,249 kB]
Get:12 http://us.archive.ubuntu.com/ubuntu xenial/universe i386 Packages [7,231 kB]
89% [12 Packages 6,411 kB/7,231 kB 89%]                                        
Get:13 http://us.archive.ubuntu.com/ubuntu xenial/universe Translation-en [4,187 kB]
Get:14 http://us.archive.ubuntu.com/ubuntu xenial/universe amd64 DEP-11 Metadata [1,802 kB]
Get:15 http://us.archive.ubuntu.com/ubuntu xenial/universe DEP-11 64x64 Icons [7,493 kB]
68% [15 icons-64x64 4,589 kB/7,493 kB 61%]                                     
68% [15 icons-64x64 4,589 kB/7,493 kB 61%]
Get:16 http://us.archive.ubuntu.com/ubuntu xenial/multiverse amd64 Packages [145 kB]                                                           
Get:17 http://us.archive.ubuntu.com/ubuntu xenial/multiverse i386 Packages [142 kB]                                                            
Get:18 http://us.archive.ubuntu.com/ubuntu xenial/multiverse amd64 DEP-11 Metadata [6,897 B]                                                   
Get:19 http://us.archive.ubuntu.com/ubuntu xenial/multiverse DEP-11 64x64 Icons [240 kB]                                                       
Fetched 28.6 MB in 23min 13s (20.5 kB/s)                                                                                                       
Reading package lists... Done
W: gpgv:/var/lib/apt/lists/dl.google.com_linux_chrome_deb_dists_stable_Release.gpg: The repository is insufficiently signed by key 4CCA1EAF950CEE4AB83976DCA040830F7FAC5991 (weak digest)
W: gpgv:/var/lib/apt/lists/ppa.launchpad.net_jtaylor_keepass_ubuntu_dists_xenial_InRelease: The repository is insufficiently signed by key 57A0E8DEA026F8D8173E90A57858088158B80F90 (weak digest)
W: gpgv:/var/lib/apt/lists/apt.insynchq.com_ubuntu_dists_xenial_InRelease: The repository is insufficiently signed by key 3B158123A580D31A9E86248106BBDC2602DFE7E7 (weak digest)
[sudo] password for chris: 
Sorry, try again.
[sudo] password for chris: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
build-essential is already the newest version (12.1ubuntu2).
git is already the newest version (1:2.7.3-0ubuntu1).
python-dev is already the newest version (2.7.11-1).
python-setuptools is already the newest version (20.1.1-1).
0 upgraded, 0 newly installed, 0 to remove and 26 not upgraded.
--2016-03-24 22:36:08--  https://bootstrap.pypa.io/get-pip.py
Resolving bootstrap.pypa.io (bootstrap.pypa.io)... 103.245.222.175
Connecting to bootstrap.pypa.io (bootstrap.pypa.io)|103.245.222.175|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1522812 (1.5M) [text/x-python]
Saving to: ‘get-pip.py’

get-pip.py                          100%[===================================================================>]   1.45M   233KB/s    in 12s     

2016-03-24 22:36:24 (126 KB/s) - ‘get-pip.py’ saved [1522812/1522812]

The directory '/home/chris/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/chris/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already up-to-date: pip in /usr/local/lib/python2.7/dist-packages
The directory '/home/chris/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/chris/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied (use --upgrade to upgrade): ansible in /usr/local/lib/python2.7/dist-packages
Requirement already satisfied (use --upgrade to upgrade): paramiko in /usr/local/lib/python2.7/dist-packages (from ansible)
Requirement already satisfied (use --upgrade to upgrade): jinja2 in /usr/local/lib/python2.7/dist-packages (from ansible)
Requirement already satisfied (use --upgrade to upgrade): PyYAML in /usr/local/lib/python2.7/dist-packages (from ansible)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/local/lib/python2.7/dist-packages/setuptools-18.1-py2.7.egg (from ansible)
Requirement already satisfied (use --upgrade to upgrade): pycrypto>=2.6 in /usr/local/lib/python2.7/dist-packages (from ansible)
Requirement already satisfied (use --upgrade to upgrade): ecdsa>=0.11 in /usr/local/lib/python2.7/dist-packages (from paramiko->ansible)
Requirement already satisfied (use --upgrade to upgrade): MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2->ansible)

I get the same error as above…in my case, the installation abruptly ends.

1 Like

@ccfiel and @kirthi
Which OS did you try the installation on?

@anand it’s ubuntu 16.04

ubuntu 14.04

@anand 1 issue i found in install.py - The ansible playbook is not getting called. script ends after cloning the bench repo due to this if condition -

if args.develop:
run_playbook(‘develop/install.yml’,sudo=True)

@anand @ccfiel that was it! running that function installs and initiates the bench. The script finished now, I’ll try installing erpnext and other apps, and keep posted.

@anand should we pass some args while calling python install.py to get the ansible function called?

1 Like

@kirthi thanks! and I am really sorry. I forgot to add that argument. I’ve updated the readme.

1 Like

no problem. I seem to have problem with connecting to mariadb as root. During install, it asked for root password, I gave one. I’m not able to use the same pwd to access it again. When I try bench new-site, mysql asks for root pwd, and it doesnt work…any ideas?

@anand Its now installing other packages but got an error

chris@chris:~$ python install.py --develop
Ign:1 http://dl.google.com/linux/chrome/deb stable InRelease                   
Hit:2 http://us.archive.ubuntu.com/ubuntu xenial InRelease                     
Hit:3 http://apt.insynchq.com/ubuntu xenial InRelease                          
Hit:4 http://dl.google.com/linux/chrome/deb stable Release                     
Hit:5 http://security.ubuntu.com/ubuntu xenial-security InRelease              
Hit:6 http://archive.canonical.com xenial InRelease                            
Hit:8 http://ppa.launchpad.net/jtaylor/keepass/ubuntu xenial InRelease         
Hit:9 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease       
Hit:10 http://ppa.launchpad.net/snappy-dev/tools/ubuntu xenial InRelease 
Hit:11 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease
Reading package lists... Done                       
W: gpgv:/var/lib/apt/lists/apt.insynchq.com_ubuntu_dists_xenial_InRelease: The repository is insufficiently signed by key 3B158123A580D31A9E86248106BBDC2602DFE7E7 (weak digest)
W: gpgv:/var/lib/apt/lists/dl.google.com_linux_chrome_deb_dists_stable_Release.gpg: The repository is insufficiently signed by key 4CCA1EAF950CEE4AB83976DCA040830F7FAC5991 (weak digest)
W: gpgv:/var/lib/apt/lists/ppa.launchpad.net_jtaylor_keepass_ubuntu_dists_xenial_InRelease: The repository is insufficiently signed by key 57A0E8DEA026F8D8173E90A57858088158B80F90 (weak digest)
W: gpgv:/var/lib/apt/lists/ppa.launchpad.net_snappy-dev_tools_ubuntu_dists_xenial_InRelease: The repository is insufficiently signed by key 78E1918602959B9C59103100F1831DDAFC42E99D (weak digest)
Reading package lists... Done
Building dependency tree       
Reading state information... Done
build-essential is already the newest version (12.1ubuntu2).
git is already the newest version (1:2.7.4-0ubuntu1).
python-dev is already the newest version (2.7.11-1).
python-setuptools is already the newest version (20.1.1-1).
The following package was automatically installed and is no longer required:
  libqt4-opengl
Use 'sudo apt autoremove' to remove it.
0 upgraded, 0 newly installed, 0 to remove and 83 not upgraded.
The directory '/home/chris/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/chris/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already up-to-date: pip in /usr/local/lib/python2.7/dist-packages
The directory '/home/chris/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/chris/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied (use --upgrade to upgrade): ansible in /usr/local/lib/python2.7/dist-packages
Requirement already satisfied (use --upgrade to upgrade): paramiko in /usr/local/lib/python2.7/dist-packages (from ansible)
Requirement already satisfied (use --upgrade to upgrade): jinja2 in /usr/local/lib/python2.7/dist-packages (from ansible)
Requirement already satisfied (use --upgrade to upgrade): PyYAML in /usr/local/lib/python2.7/dist-packages (from ansible)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/local/lib/python2.7/dist-packages/setuptools-18.1-py2.7.egg (from ansible)
Requirement already satisfied (use --upgrade to upgrade): pycrypto>=2.6 in /usr/local/lib/python2.7/dist-packages (from ansible)
Requirement already satisfied (use --upgrade to upgrade): ecdsa>=0.11 in /usr/local/lib/python2.7/dist-packages (from paramiko->ansible)
Requirement already satisfied (use --upgrade to upgrade): MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2->ansible)
SUDO password: 
 [WARNING]: provided hosts list is empty, only localhost is available

MySQL Root Password: 

PLAY ***************************************************************************

TASK [setup] *******************************************************************
ok: [localhost]

PLAY ***************************************************************************

TASK [setup] *******************************************************************
ok: [localhost]

TASK [install prequisites] *****************************************************
skipping: [localhost] => (item=cmake) 
skipping: [localhost] => (item=redis) 
skipping: [localhost] => (item=mariadb) 
skipping: [localhost] => (item=nodejs) 

TASK [cask installs] ***********************************************************
skipping: [localhost] => (item=wkhtmltopdf) 

TASK [include] *****************************************************************
skipping: [localhost]

TASK [include] *****************************************************************
skipping: [localhost]

PLAY ***************************************************************************

TASK [setup] *******************************************************************
ok: [localhost]

TASK [install prequisites] *****************************************************


changed: [localhost] => (item=[u'build-essential', u'redis-server', u'nodejs', u'npm', u'software-properties-common', u'libxrender1', u'libxext6', u'xfonts-75dpi', u'xfonts-base', u'libjpeg8-dev', u'zlib1g-dev', u'libfreetype6-dev', u'liblcms2-dev', u'libwebp-dev', u'python-tk'])

TASK [install pillow prerequisites for Ubuntu < 14.04] *************************
skipping: [localhost] => (item=[]) 

TASK [install pillow prerequisites for Ubuntu > 14.04] *************************
changed: [localhost] => (item=[u'libtiff5-dev', u'tcl8.6-dev', u'tk8.6-dev'])

TASK [include] *****************************************************************
included: /usr/local/frappe/bench-repo/playbooks/develop/includes/mariadb_ubuntu.yml for localhost

TASK [Add apt key] *************************************************************
ok: [localhost]

TASK [Add apt repositry] *******************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: apt.cache.FetchFailedException: W:gpgv:/var/lib/apt/lists/dl.google.com_linux_chrome_deb_dists_stable_Release.gpg: The repository is insufficiently signed by key 4CCA1EAF950CEE4AB83976DCA040830F7FAC5991 (weak digest), W:gpgv:/var/lib/apt/lists/ppa.launchpad.net_jtaylor_keepass_ubuntu_dists_xenial_InRelease: The repository is insufficiently signed by key 57A0E8DEA026F8D8173E90A57858088158B80F90 (weak digest), W:The repository 'http://nyc2.mirrors.digitalocean.com/mariadb/repo/10.1/ubuntu xenial Release' does not have a Release file., W:Data from such a repository can't be authenticated and is therefore potentially dangerous to use., W:See apt-secure(8) manpage for repository creation and user configuration details., W:gpgv:/var/lib/apt/lists/apt.insynchq.com_ubuntu_dists_xenial_InRelease: The repository is insufficiently signed by key 3B158123A580D31A9E86248106BBDC2602DFE7E7 (weak digest), W:gpgv:/var/lib/apt/lists/ppa.launchpad.net_snappy-dev_tools_ubuntu_dists_xenial_InRelease: The repository is insufficiently signed by key 78E1918602959B9C59103100F1831DDAFC42E99D (weak digest), W:Failed to fetch http://nyc2.mirrors.digitalocean.com/mariadb/repo/10.1/ubuntu/dists/xenial/main/binary-amd64/Packages  404  Not Found [IP: 192.241.164.26 80], W:Failed to fetch http://nyc2.mirrors.digitalocean.com/mariadb/repo/10.1/ubuntu/dists/xenial/main/binary-i386/Packages  404  Not Found [IP: 192.241.164.26 80], E:Some index files failed to download. They have been ignored, or old ones used instead.
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "parsed": false}
	to retry, use: --limit @develop/install.retry

PLAY RECAP *********************************************************************
localhost                  : ok=7    changed=2    unreachable=0    failed=1   

Traceback (most recent call last):
  File "install.py", line 153, in <module>
    install_bench(args)
  File "install.py", line 58, in install_bench
    run_playbook('develop/install.yml', sudo=True)
  File "install.py", line 131, in run_playbook
    success = subprocess.check_call(args, cwd=os.path.join(bench_repo, 'playbooks'))
  File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ansible-playbook', '-c', 'local', 'develop/install.yml', '-K']' returned non-zero exit status 2
chris@chris:~$ 

@ccfiel

From the error it seems you are using Ubuntu 16.04 which does not have apt repository yet in the path that is included in the ansible playbook. We might have to make a change in the playbook to work for the latest Ubuntu release which can happen only after 16.04 gets released.

At present playbook support 15.04, as there is no apt repository added for 16.04.

1 Like

@anand my db error is -

How do I reset or recover the root password for mysql?

Migrating erpnext
Traceback (most recent call last):
File “/usr/lib/python2.7/runpy.py”, line 162, in _run_module_as_main
main”, fname, loader, pkg_name)
File “/usr/lib/python2.7/runpy.py”, line 72, in _run_code
exec code in run_globals
File “/home/kirthi/frappe/frappe-bench/apps/frappe/frappe/utils/bench_helper.py”, line 79, in
main()
File “/home/kirthi/frappe/frappe-bench/apps/frappe/frappe/utils/bench_helper.py”, line 16, in main
click.Group(commands=commands)(prog_name=‘bench’)
File “/home/kirthi/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 716, in call
return self.main(*args, **kwargs)
File “/home/kirthi/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 696, in main
rv = self.invoke(ctx)
File “/home/kirthi/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/home/kirthi/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/home/kirthi/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 889, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/home/kirthi/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 534, in invoke
return callback(*args, **kwargs)
File “/home/kirthi/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/decorators.py”, line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File “/home/kirthi/frappe/frappe-bench/apps/frappe/frappe/commands.py”, line 29, in _func
ret = f(frappe._dict(ctx.obj), *args, **kwargs)
File “/home/kirthi/frappe/frappe-bench/apps/frappe/frappe/commands.py”, line 205, in migrate
migrate(context.verbose, rebuild_website=rebuild_website)
File “/home/kirthi/frappe/frappe-bench/apps/frappe/frappe/migrate.py”, line 27, in migrate
frappe.modules.patch_handler.run_all()
File “/home/kirthi/frappe/frappe-bench/apps/frappe/frappe/modules/patch_handler.py”, line 21, in run_all
executed = [p[0] for p in frappe.db.sql(“”“select patch from tabPatch Log”“”)]
File “/home/kirthi/frappe/frappe-bench/apps/frappe/frappe/database.py”, line 106, in sql
self.connect()
File “/home/kirthi/frappe/frappe-bench/apps/frappe/frappe/database.py”, line 52, in connect
use_unicode=True, charset=‘utf8’)
File “/home/kirthi/frappe/frappe-bench/env/local/lib/python2.7/site-packages/MySQLdb/init.py”, line 81, in Connect
return Connection(*args, **kwargs)
File “/home/kirthi/frappe/frappe-bench/env/local/lib/python2.7/site-packages/MySQLdb/connections.py”, line 193, in init
super(Connection, self).init(*args, **kwargs2)
_mysql_exceptions.OperationalError: (1045, “Access denied for user ‘76cf18db07’@‘localhost’ (using password: YES)”)

@anand This is the log of ansible during installation. It skips setting root password. Is that why I’m having problem with logging into mariadb as root?

TASK [include] ***************************************************************** included: /usr/local/frappe/bench-repo/playbooks/develop/includes/setup_mariadb.yml for localhost

TASK [Install MySQLdb in global env] *******************************************
changed: [localhost]

TASK [Set root Password] *******************************************************
skipping: [localhost] => (item=localhost)

Did you have mariadb already installed on your system? The traceback doesn’t indicate any problem with root password. Try creating a new site and check if it starts.

@anand Same error with new install as well. I checked and also ran a purge of MySQL before running the script. In fact, it asks for mysql root password during install…where is root pwd stored in erpnext? Can I try changing in MySQL and configure in erpnext?

I meant to say - I haven’t installed MySQL. It is a brand new container (vm)