Commit Graph

5646 Commits

Author SHA1 Message Date
Girish Ramakrishnan 404a019c56 s3: Check IsTruncated before accessing Contents 2017-09-28 10:36:56 -07:00
Girish Ramakrishnan 24dee80aa6 Make box backups always tarball based
this makes cloudron easy to restore. in the future, if required,
we can move out the mail data as a separate virtual app backup
2017-09-28 10:22:10 -07:00
Girish Ramakrishnan ce6df4bf96 Disable encryption for flat-file for now 2017-09-28 09:47:18 -07:00
Girish Ramakrishnan f8f6c7d93e Add progress detail when rotating snapshots 2017-09-28 09:29:46 -07:00
Girish Ramakrishnan bafc6dce98 s3: refactor out directory listing 2017-09-27 21:59:51 -07:00
Girish Ramakrishnan 56ee4d8e25 Remove old cache files when backup settings is changed 2017-09-27 21:04:46 -07:00
Girish Ramakrishnan eeef221b4e Fix race where pipe finishes before file is created
When there are 0 length files, this is easily reproducible.
2017-09-27 19:40:26 -07:00
Girish Ramakrishnan 4674653982 compare size and inode as well 2017-09-27 19:39:03 -07:00
Girish Ramakrishnan a34180c27b Add format to backupsdb
Call remove/removeDir based on the format
2017-09-27 18:02:30 -07:00
Girish Ramakrishnan aa8ce2c62e Use graphite 0.12.0
this fixes an issue where carbon does not startup properly
if a previous pid file was present
2017-09-27 15:35:55 -07:00
Girish Ramakrishnan b3c6b8aa15 do not spawn process just for chown 2017-09-27 15:07:19 -07:00
Girish Ramakrishnan 44a7a2579c rework backup status
* show backup progress even if not initiated by UI
* display backup progress in separate line
2017-09-27 15:07:15 -07:00
Girish Ramakrishnan 39f0e476f2 Start out empty if cache file is missing 2017-09-27 12:09:19 -07:00
Girish Ramakrishnan 003dc0dbaf Add todo 2017-09-27 11:50:49 -07:00
Girish Ramakrishnan e39329218d Make tests work 2017-09-27 11:38:43 -07:00
Girish Ramakrishnan 8d3fbc5432 Save backup logs and fix backup progress 2017-09-26 21:09:00 -07:00
Girish Ramakrishnan 2780de631e writable streams emit finish 2017-09-26 16:43:51 -07:00
Girish Ramakrishnan 399c756735 use exec so that filenames do not have to be escaped 2017-09-26 15:53:42 -07:00
Girish Ramakrishnan 859311f9e5 Process delete commands before add commands
This is required for cases where a dir becomes a file (or vice-versa)
2017-09-26 15:33:54 -07:00
Girish Ramakrishnan a9e89b57d9 merge caas storage into s3 backend 2017-09-26 12:28:33 -07:00
Girish Ramakrishnan 4e68abe51d Handle fs errors 2017-09-26 12:10:58 -07:00
Girish Ramakrishnan 12083f5608 Ignore all special files 2017-09-26 11:41:01 -07:00
Girish Ramakrishnan d1efb2db56 remove bogus mkdir 2017-09-26 11:34:24 -07:00
Girish Ramakrishnan adde28523f Add backup format to the backup UI 2017-09-26 10:46:02 -07:00
Girish Ramakrishnan f122f46fe2 Generate new index file by appending to file 2017-09-26 07:57:20 -07:00
Girish Ramakrishnan ad7fadb4a9 display backup id in the ui 2017-09-26 07:45:23 -07:00
Johannes Zellner be383582e0 Do not rely on external resource in the appstatus page 2017-09-26 15:33:05 +02:00
Girish Ramakrishnan 0a60365143 Initial version of flat-file uploader 2017-09-26 00:17:11 -07:00
Girish Ramakrishnan 2f6cb3e913 set format in the backup ui 2017-09-26 00:01:36 -07:00
Girish Ramakrishnan b0f85678d4 Implement downloadDir for flat-file format 2017-09-23 18:07:26 -07:00
Girish Ramakrishnan e43413e063 implement remove dir in storage backends 2017-09-23 12:34:51 -07:00
Girish Ramakrishnan e39a5c8872 preserve env in backuptask.js 2017-09-22 11:19:44 -07:00
Girish Ramakrishnan fb4b75dd2a Fix typo in comment 2017-09-22 11:19:37 -07:00
Girish Ramakrishnan 3c1ccc5cf4 Add exoscale provider 2017-09-21 17:50:03 -07:00
Girish Ramakrishnan abd66d6524 Add cloudscale as a provider 2017-09-21 17:49:26 -07:00
Girish Ramakrishnan b61b7f80b5 Add DO spaces 2017-09-21 12:25:39 -07:00
Girish Ramakrishnan efa850614d Add a s3-v4-compat provider 2017-09-21 12:13:45 -07:00
Girish Ramakrishnan 21c534c806 Ensure format is set in backupConfig 2017-09-21 09:49:55 -07:00
Girish Ramakrishnan 7e4ff2440c Fix text for manual DNS 2017-09-21 09:10:12 -07:00
Johannes Zellner f415e19f6f Do not unneccesarily mention error in the logs
Not so friendly for log searches
2017-09-21 15:00:35 +02:00
Girish Ramakrishnan 97da8717ca Refactor backup strategy logic into backups.js 2017-09-20 14:09:55 -07:00
Girish Ramakrishnan cbddb79d15 Resolve the id in rotateAppBackup 2017-09-20 09:38:55 -07:00
Johannes Zellner bffb935f0f Also send digest to appstore account owner 2017-09-20 16:33:25 +02:00
Johannes Zellner e50e0f730b Make nginx listen on :: for ipv6 2017-09-20 16:33:25 +02:00
Girish Ramakrishnan 26f33a8e9b Send resolved path to the storage APIs 2017-09-19 21:58:35 -07:00
Girish Ramakrishnan 952b1f6304 Make backuptask call back into backups.js 2017-09-19 20:27:49 -07:00
Girish Ramakrishnan a3293c4c35 Fix tests 2017-09-19 12:43:13 -07:00
Girish Ramakrishnan 4892473eff backupIds do not have extension anymore
this code existed for legacy reasons
2017-09-19 12:34:09 -07:00
Girish Ramakrishnan 221d5f95e1 ensure backupFolder is always set 2017-09-19 12:34:09 -07:00
Girish Ramakrishnan 84649b9471 Bring back backuptask
This is required for various small reasons:

* dir iteration with a way to pass messagein back to the upload() easily
* can be killed independently of box code
* allows us to run sync (blocking) commands in the upload logic
2017-09-19 12:32:38 -07:00