Socket.error: [Errno 104] Connection reset by peer using ftplib

Frappe Framework
1

Hi there,

I’m getting this f**king error while retrieving directory list using ftplib client, but can’t understand the reason …

I’m on VM Ubuntu 16, Python 2.7.12 and frappe on develop:

frappe@ubuntu16-dev:~$ python -V
Python 2.7.12
frappe@ubuntu16-dev:~$ python
Python 2.7.12 (default, Jul  1 2016, 15:12:24)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ftplib
>>> ftp = ftplib.FTP('ftp.sunet.se', 'anonymous', 'anonymous@sunet.se')
>>> ftp.dir()
-rw-r--r--    1 ftp      ftp          1204 Oct 04 15:11 HEADER.html
lrwxrwxrwx    1 ftp      ftp             3 Mar 16  2010 Public -> pub
drwxr-xr-x    3 ftp      ftp            10 Nov 07 19:50 about
drwxr-xr-x   28 ftp      ftp            30 Nov 12 11:28 cdimage
drwxr-xr-x    2 ftp      ftp             3 Jun 14  2006 conspiracy
lrwxrwxrwx    1 ftp      ftp            22 Mar 16  2010 debian -> cdimage/.debian-mirror
drwxr-xr-x    5 ftp      ftp             7 Mar 13  2016 debian-backports
lrwxrwxrwx    1 ftp      ftp            16 Mar 16  2010 debian-cd -> cdimage/release/
-rw-r--r--    1 ftp      ftp           318 Mar 21  2005 favicon.ico
drwxr-xr-x   57 ftp      ftp            64 Oct 25 09:29 mirror
drwxr-xr-x    4 ftp      ftp            12 Feb 05  2016 pub
lrwxrwxrwx    1 ftp      ftp            23 Mar 16  2010 releases -> mirror/ubuntu-releases/
-rw-r--r--    1 ftp      ftp           965 Oct 12 21:30 robots.txt
lrwxrwxrwx    1 ftp      ftp            12 Aug 01 19:23 tails -> mirror/tails
lrwxrwxrwx    1 ftp      ftp            13 Mar 16  2010 ubuntu -> mirror/ubuntu
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/ftplib.py", line 534, in dir
    self.retrlines(cmd, func)
  File "/usr/lib/python2.7/ftplib.py", line 440, in retrlines
    line = fp.readline(self.maxline + 1)
  File "/usr/lib/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)
socket.error: [Errno 104] Connection reset by peer

On MacOS and python 2.7.12 it’s working fine:

MacBook-Pro $ python -V
Python 2.7.12

MacBook-Pro $ python
Python 2.7.12 (default, Sep 17 2016, 18:01:33)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ftplib
>>> ftp = ftplib.FTP('ftp.sunet.se', 'anonymous', 'anonymous@sunet.se')
>>> fpt.dir()
-rw-r--r--    1 ftp      ftp          1204 Oct 04 15:11 HEADER.html
lrwxrwxrwx    1 ftp      ftp             3 Mar 16  2010 Public -> pub
drwxr-xr-x    3 ftp      ftp            10 Nov 07 19:50 about
drwxr-xr-x   28 ftp      ftp            30 Nov 12 11:28 cdimage
drwxr-xr-x    2 ftp      ftp             3 Jun 14  2006 conspiracy
lrwxrwxrwx    1 ftp      ftp            22 Mar 16  2010 debian -> cdimage/.debian-mirror
drwxr-xr-x    5 ftp      ftp             7 Mar 13  2016 debian-backports
lrwxrwxrwx    1 ftp      ftp            16 Mar 16  2010 debian-cd -> cdimage/release/
-rw-r--r--    1 ftp      ftp           318 Mar 21  2005 favicon.ico
drwxr-xr-x   57 ftp      ftp            64 Oct 25 09:29 mirror
drwxr-xr-x    4 ftp      ftp            12 Feb 05  2016 pub
lrwxrwxrwx    1 ftp      ftp            23 Mar 16  2010 releases -> mirror/ubuntu-releases/
-rw-r--r--    1 ftp      ftp           965 Oct 12 21:30 robots.txt
lrwxrwxrwx    1 ftp      ftp            12 Aug 01 19:23 tails -> mirror/tails
lrwxrwxrwx    1 ftp      ftp            13 Mar 16  2010 ubuntu -> mirror/ubuntu
>>>

As well as on VM Ubuntu 14, Python 2.7.6:

frappe@erpnext:~$ python -V
Python 2.7.6
frappe@erpnext:~$ python
Python 2.7.6 (default, Jun 22 2015, 18:00:18)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ftplib
>>> ftp = ftplib.FTP('ftp.sunet.se', 'anonymous', 'anonymous@sunet.se')
>>> ftp.dir()
-rw-r--r--    1 ftp      ftp          1204 Oct 04 15:11 HEADER.html
lrwxrwxrwx    1 ftp      ftp             3 Mar 16  2010 Public -> pub
drwxr-xr-x    3 ftp      ftp            10 Nov 07 19:50 about
drwxr-xr-x   28 ftp      ftp            30 Nov 12 11:28 cdimage
drwxr-xr-x    2 ftp      ftp             3 Jun 14  2006 conspiracy
lrwxrwxrwx    1 ftp      ftp            22 Mar 16  2010 debian -> cdimage/.debian-mirror
drwxr-xr-x    5 ftp      ftp             7 Mar 13  2016 debian-backports
lrwxrwxrwx    1 ftp      ftp            16 Mar 16  2010 debian-cd -> cdimage/release/
-rw-r--r--    1 ftp      ftp           318 Mar 21  2005 favicon.ico
drwxr-xr-x   57 ftp      ftp            64 Oct 25 09:29 mirror
drwxr-xr-x    4 ftp      ftp            12 Feb 05  2016 pub
lrwxrwxrwx    1 ftp      ftp            23 Mar 16  2010 releases -> mirror/ubuntu-releases/
-rw-r--r--    1 ftp      ftp           965 Oct 12 21:30 robots.txt
lrwxrwxrwx    1 ftp      ftp            12 Aug 01 19:23 tails -> mirror/tails
lrwxrwxrwx    1 ftp      ftp            13 Mar 16  2010 ubuntu -> mirror/ubuntu
>>>

I’m not really sure if it’s some of the installed packages from bench that differ from Ubuntu 14 or whatever …can someone help on this?

2

So for some reason the server abruptly quits the session once dir() completes!?

Why not just restart the ftp session to transfer files etc

This says ubuntu - Python handling socket.error: [Errno 104] Connection reset by peer - Stack Overflow

your problem is on the server end and not your end…

3

@clarkej
thx for the hint, but I do not think it is a server fault ( the server i tested is ftp.sunet.se), otherwise I’d face the same error when connecting from macOS (host machine for the VMs) and from the other VM with Ubuntu 14.

I think it’s a problem on some package (socket?) installed by the frappe installer in Ubuntu 16.

4

@vjFaLk sorry to tag you :sweat:, do u have any idea on this?

5

@clarkej

Update: Created a new VM from scratch with Ubuntu 16 LTS, minimal install + openssh-server, python-minimal (aka python 2.7) and ftplib it’s working properly with no socket error:

frappe@ubuntu16-test:~$ python
Python 2.7.12 (default, Jul  1 2016, 15:12:24)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ftplib
>>> ftp = ftplib.FTP("ftp.debian.org")
>>> ftp.login()
'230 Login successful.'
>>> ftp.dir()
drwxr-xr-x    9 1176     1176         4096 Nov 17 08:57 debian
>>> ftp.cwd("/debian")
'250 Directory successfully changed.'
>>> ftp.dir()
-rw-rw-r--    1 1176     1176         1062 Sep 17 09:53 README
-rw-rw-r--    1 1176     1176         1290 Jun 26  2010 README.CD-manufacture
-rw-rw-r--    1 1176     1176         2590 Sep 17 09:53 README.html
-rw-r--r--    1 1176     1176       191243 Oct 31 07:52 README.mirrors.html
-rw-r--r--    1 1176     1176       103591 Oct 31 07:52 README.mirrors.txt
drwxr-sr-x   18 1176     1176         4096 Sep 17 09:55 dists
drwxr-sr-x    4 1176     1176         4096 Nov 17 07:52 doc
-rw-r--r--    1 1176     1176       148241 Nov 17 08:49 extrafiles
drwxr-sr-x    3 1176     1176         4096 Jul 27  2015 indices
-rw-r--r--    1 1176     1176     12165753 Nov 17 08:42 ls-lR.gz
drwxr-sr-x    5 1176     1176         4096 Dec 19  2000 pool
drwxr-sr-x    4 1176     1176         4096 Nov 17  2008 project
drwxr-xr-x    3 1176     1176         4096 Oct 10  2012 tools
drwxr-xr-x    6 1176     1176         4096 May 21 15:36 zzz-dists
>>>

Definitely that’s not a problem on remote server, i think it’s a problem of some package installed by bench install for Ubuntu 16, but i can’t figure out which one …i guess some packages socket related, but not sure which one.

@vjFaLk, again sorry for tagging you, can you please have a look?

Thanks a lot in advance

6

Wow this is quite odd. I can’t think of any package that would break ftplib :confused:

Are the ftplib versions the same for both systems?

Can you install frappe on the new VM that is working properly and try again? Maybe it’s unrelated.

7

@vjFaLk Yeah, that’s quite odd :disappointed: …i’ll try new frappe install on both Ubuntu 16 and CentOS 7 and will let you know.

8

@vjFaLk

Update:

i’ve created two new VM from scratch with CentOS 7 and Ubuntu 16 and installed bench and frappe using easy install:

sudo python install.py --develop

it’s working without any probs on CentOS, while on Ubuntu 16 i’m still getting the error:

frappe@ubuntu16-dev:~$ python -V
Python 2.7.12
frappe@ubuntu16-dev:~$ python
Python 2.7.12 (default, Jul  1 2016, 15:12:24)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ftplib
>>> ftp = ftplib.FTP("ftp.debian.org")
>>> ftp.login()
'230 Login successful.'
>>> ftp.dir()
drwxr-xr-x    9 1176     1176         4096 Nov 17 08:57 debian
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/ftplib.py", line 534, in dir
    self.retrlines(cmd, func)
  File "/usr/lib/python2.7/ftplib.py", line 440, in retrlines
    line = fp.readline(self.maxline + 1)
  File "/usr/lib/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)
socket.error: [Errno 104] Connection reset by peer
>>>

:cry::sweat::confused:

9

@vjFaLk

Update 2: it seems to work on CentOS until i’ll install app to the site using:

bench --site [site-name] install-app [appname]

The app is a basic frappe app, the code:

def validate(self):
    print get_files_path(is_private="is_private")

    print self.ftp_server
    print self.ftp_user
    print self.get_password(fieldname="ftp_password")

    ftp = connectFTP(self.ftp_server, self.ftp_user,
                     self.get_password(fieldname="ftp_password"))

    listing = []
    ftp.dir(listing.append)



def getFile(ftp, file_path, filename):
   try:
       ftp.retrbinary("RETR " + filename,
                   open(file_path + "/" + filename, 'wb').write)
   except Exception, e:
       print("Error: " + str(e))


def connectFTP(server, user, password):
   ftp = ftplib.FTP(server)
   ftp.login(user, password)

   print ftp.getwelcome()

   return ftp

I’m really struggling with this, can’t understand what’s wrong with this :frowning: :sob:

10

It’s quite late in this part of the world, so lemme look at this tomorrow. This is such a weird bug :laughing:

11

@vjFaLk thx …take your time :grin:

12

@vjFaLk i think i got the problem :grin:

Error depends on VM network setting! If set as NAT, for some reason error [Errno 104] Connection reset by peer will be raised, not sure why …but that’s what happening :confused:

1 Like
13

:dizzy_face: :dizzy_face::dizzy_face::dizzy_face:

14

…exately :dizzy_face: :astonished: …at least i found out …u can close the thread …thx