Pure-FTPd SR
This SR is intended to be used in a demo next week. The purpose is to have a running pureftpd server which will call a script "/opt/pureftpd/upload_script" each time a file is uploaded. Pureftpd is used because it is already used in the current working environment of our partner and they don't want to change for now.
The script /opt/pureftpd/upload_script" is left out of the SR on purpose because it will be written by people outside Nexedi at first and contains sensitive information. Also, it is on purpose in /opt directory of the machine because this SR will be deployed on only one machine for now and the script will be shared by all pureftpd instances.
We don't merge this MR yet as we don't really know if this SR will be useful later...
-
TODO: missing dependencies ? probably openssl
-
Edited@jm pureftpd is not using openssl. It uses
libcrypt.soand notlibcrypto.so.libcryptcomes from glibc.$ ldd /srv/slapgrid/slappart61/srv/runner/software/6d89de9ff91a81af09155426afb30041/parts/pure-ftpd/sbin/pure-ftpd linux-vdso.so.1 (0x00007ffc11912000) libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007fd1f15bd000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd1f1212000) /lib64/ld-linux-x86-64.so.2 (0x000055f4dba9a000) $ dpkg -L libc6 | grep libcrypt /lib/x86_64-linux-gnu/libcrypt-2.19.so /lib/x86_64-linux-gnu/libcrypt.so.1 -
-
-
-
-
-
-
software/pureftpd/instance.cfg.in 0 → 100644
52 ipv6 = ${slap-configuration:ipv6-random} 53 ipv4 = ${slap-configuration:ipv4-random} 54 host = ${:ipv6} 55 ftp-port = ${slap-configuration:configuration.port} 56 url = ftp://[${:host}]:${:ftp-port} 57 data-dir = ${directory:pureftpd-dir} 58 pid-file=${directory:run}/pureftpd.pid 59 auth-user-file=${auth-user-file:output} 60 61 recipe =slapos.recipe.template:jinja2 62 # WARNING pure-uploadscript must be launched AFTER pure-ftpd so keep them in the same wrapper 63 # and make sure they are both killed if one of them is killed. 64 template = inline: 65 #!{{ bash_location }}/bin/bash 66 {{ pureftpd_bin }} --uploadscript --customerproof --bind ${:host},${:ftp-port} --login puredb:${:auth-user-file} --pidfile ${:pid-file} & 67 sleep 5 -
For the records:
It's a long time bug in supervisor that we cannot wait for a process to listen on port ( https://github.com/Supervisor/supervisor/issues/122 ). Some people use https://github.com/vishnubob/wait-for-it to workaround it. I don't think we tried using it in slapos.
In grafana SR, we use a trick of running check port listening promise to only start the second process once the process it depends on is started. I did not know about
wait-for-itat that time, so I did this silly "refuse to start when not ready". -
Indeed, Julien already commented here : tomo/slapos@bc0ed7c4 (comment 73101)
I'm waiting for the pipe to be created now but I just realized that the pipe is not removed when pureftpd is killed... so I will need to find a better condiion
-
Maybe the
wait-for-itlogic could be added toslapos.recipe:wrapper, it already has "wait-for-files" it could have "wait-for-tcp-listen" or similar naming. But if the upload controller also have to be restarted when pure-ftp is restarted, then it's more complicated.
-
-
It would be nice to have a few words explaining why we don't use existing proftpd and make a new software release. If I remember @rafael explained that the reason is that it's to be used in a context where the
upload_scriptinfrastructure is already used and switching to proftpd would be a "useless" change in this context. I think that's the reason, isn't it ?Also our proftpd only supports sftp ( a bit "on purpose" ).
Note that proftpd has http://www.proftpd.org/docs/contrib/mod_exec.html that maybe can be used to achieve the same. proftpd was selected because it supports sftp (in that context we needed strong authentication and encryption) and virtual users. In my understanding PureFTP, did not support sftp, so I did not try it.
Maybe the content
/opt/pureftpd/upload_scriptcan be an instance parameter ? this would make unit testing easier.
This SR is intended to be used in a demo next week
good luck :) I left lots of comments, but nothing seems "blocking" for that demo. Also, these are not comments to "request changes", just to "add information on this topic in the hope it would help"
-
Small clarification: I'm perfectly OK that we make another "*ftp" SR, just think we should document why chose to make another one.
-
marked as a Work In Progress
Toggle commit list -
software/pureftpd/instance.cfg.in 0 → 100644
82 wrapper-path =${buildout:bin-directory}/${:_buildout_section_name_} 83 command-line = 84 {{ purepw_bin }} useradd ${pureftpd-password:username} -d ${pureftpd:data-dir} -u ${pureftpd-userinfo:pw-uid} -g ${pureftpd-userinfo:gr-gid} -f ${auth-user-file:passwd-file} "$@" 85 86 87 [auth-user-file] 88 recipe = plone.recipe.command 89 passwd-file = ${directory:etc}/pureftpd.passwd 90 output = ${directory:etc}/pureftpd.pdb 91 command = 92 if [ -f ${:passwd-file} ] && {{ purepw_bin }} show ${pureftpd-password:username} -f ${:passwd-file} 93 then 94 ( echo ${pureftpd-password:passwd} ; echo ${pureftpd-password:passwd}) | {{ purepw_bin }} passwd ${pureftpd-password:username} -f ${:passwd-file} 95 else 96 ( echo ${pureftpd-password:passwd} ; echo ${pureftpd-password:passwd}) | ${pure-pw:wrapper-path} 97 fi -
( echo ${pureftpd-password:passwd} ; echo ${pureftpd-password:passwd}) | if ...
-
-
added 2 commits
Toggle commit list -
added 10 commits
-
02902bd4...22ebf9fd - 9 commits from branch
nexedi:master - a4312787 - Add new SR: pureftpd
Toggle commit list -
02902bd4...22ebf9fd - 9 commits from branch
-
software/pureftpd/instance.cfg.in 0 → 100644
99 else 100 ( echo ${pureftpd-password:passwd} ; echo ${pureftpd-password:passwd}) | ${pure-pw:wrapper-path} 101 fi 102 {{ purepw_bin }} mkdb ${:output} -f ${:passwd-file} 103 update-command = ${:command} 104 105 106 [promises] 107 recipe = 108 instance-promises = 109 ${pureftpd-listen-promise:output} 110 111 112 [publish-connection-parameter] 113 recipe = slapos.cookbook:publish 114 <= monitor-publish -
unmarked as a Work In Progress
Toggle commit list -
We could just merge this, it is ok for a proof of concept and worked for the demo we intend, so if nobody objects, I will just merge.
-
Please register or sign in to post a comment