Commit 783d0ab0 authored by Alain Takoudjou's avatar Alain Takoudjou

slaprunner: fix argument list too long in exporter script when there is too many files

/reviewed-on nexedi/slapos!415
parent 5461cb85
......@@ -30,7 +30,7 @@ md5sum = 7a879739afe55320ee96409bcc8a52ab
[template-runner-export-script]
filename = template/runner-export.sh.jinja2
md5sum = 98ce179badc6af5979a64a7c3d0a2ceb
md5sum = 5877e70b2bd5cfe06aff793125f65d6a
[instance-runner-export]
filename = instance-runner-export.cfg.in
......
......@@ -113,16 +113,18 @@ trap remove_tmp_files EXIT
# Getting files from runner backup directory, as instance backup files may be
# explicitely excluded from the backup, using the srv/exporter.exclude
backup_directory_path="$tmp_directory/backup_files.txt"
cd {{ directory['backup'] }}
backup_directory_path=$(find . -path "./runner/instance/slappart*/srv/backup/*" -type f)
find . -path "./runner/instance/slappart*/srv/backup/*" -type f -print0 > $backup_directory_path
# If no backup found, it's over
if [ -z "$backup_directory_path" ]; then
if [ ! -s "$backup_directory_path" ]; then
exit 0
fi
sleep 5
sha256sum $backup_directory_path | sort -k 66 > "$tmp_backup_sum"
cat $backup_directory_path | xargs -0 sha256sum | sort -k 66 > "$tmp_backup_sum"
rm $backup_directory_path
egrep "instance/slappart.*/srv/backup/" "$backup_directory/backup.signature" > "$tmp_filtered_signature"
# If the diff fails, then the notifier will restart this script
......
  • thanks, if I understood well the command will become :

    <$backup_directory_path | xargs -0 sha256sum | sort -k 66 > "$tmp_backup_sum"
  • No. Either

    <$backup_directory_path xargs -0 sha256sum | sort -k 66 > "$tmp_backup_sum"

    or

    xargs -0 sha256sum <$backup_directory_path | sort -k 66 > "$tmp_backup_sum"

    BTW, sleep 5 deserves a comment if it's actually useful. Otherwise, I'd do without a temporary file:

    find -path "./runner/instance/slappart*/srv/backup/*" -type f -print0 |
    xargs -r -0 sha256sum | sort -k 66 > "$tmp_backup_sum"

    Note the -r option to xargs. And stop if $tmp_backup_sum is empty.

  • Yes, this is good:

    find -path "./runner/instance/slappart*/srv/backup/*" -type f -print0 |
    xargs -r -0 sha256sum | sort -k 66 > "$tmp_backup_sum"

    not sure why there is sleep 5 but I think it can be removed. Maybe @Nicolas can confirm.

  • It was added in 9854c583 and I guess it's there to let enough time to an eventual backup process to do changes.

    We would keep the sleep, but it looks fine to move it before the find.

  • @jm is right, the "sleep 5" is used to increase the probability that files in use change, in order to restart the export process.

  • mentioned in commit 05bdceb9

    Toggle commit list
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment