1444bb038f
only upload needs to be retried
...
copy/delete are already retried in the sdk code
2017-10-11 00:08:41 -07:00
ac9e421ecf
improved backup progress and logging
2017-10-10 22:49:38 -07:00
56d794745b
Sprinkle retries in syncer logic
2017-10-10 14:25:03 -07:00
fd3b73bea2
typo in format name
2017-10-10 13:54:54 -07:00
78807782df
Various hacks for exoscale-sos
...
SOS does not like multipart uploads. They just fail randomly.
As a fix, we try to detect filesystem files and skip multipart uploads
for files < 5GB. For > 5GB, we do multipart upload anyways (mostly fails).
The box backup is switched to flat-file for exoscale for the reason
above.
2017-10-10 11:03:20 -07:00
ee1a4411f8
Do not crash if prefix is empty string
...
('' || undefined) will return undefined ...
2017-10-05 11:08:01 -07:00
d70ff7cd5b
Make copy() return event emitter
...
This way the storage logic does not need to rely on progress
2017-10-04 11:02:50 -07:00
8c7223ceed
Fix cleanup logic to use the app backup format
...
box backup and app backup can have different format
2017-10-03 00:56:34 -07:00
7bf70956a1
fix tests
2017-10-02 18:42:13 -07:00
0f543e6703
s3: add progress detail
...
this is a bit of a hack and we should add another way to set the progress
(maybe via backups.setProgress or via a progress callback). this is because
some methods like removeDir can be called from backuptask and from box code.
2017-10-01 18:25:51 -07:00
f9973e765c
Add backup cleanup eventlog
2017-10-01 10:35:50 -07:00
e089851ae9
add debugs
2017-09-30 20:36:08 -07:00
c524d68c2f
fix crash when cleaning up snapshots
2017-09-30 20:31:41 -07:00
5cccb50a31
fix backup cleanup logic
2017-09-30 18:38:45 -07:00
a93d453963
rename flat-file to rsync
...
not a name I like but cannot come up with anything better
https://en.wikipedia.org/wiki/Flat_file_database
the term 'rsync format' seems to be used in a few places
2017-09-30 14:19:19 -07:00
d5ba73716b
add emptydirs test
2017-09-29 15:29:22 -07:00
954224dafb
make syncer track directories
2017-09-29 15:29:18 -07:00
4a5cbab194
Do not remove parent directory in fs.remove()
...
Do the pruning in the cleanup logic instead
2017-09-28 20:55:45 -07:00
19999abc50
s3: fix restore
2017-09-28 14:35:49 -07:00
565c8445e1
make backup progress work for per-app backups
2017-09-28 11:17:48 -07:00
24dee80aa6
Make box backups always tarball based
...
this makes cloudron easy to restore. in the future, if required,
we can move out the mail data as a separate virtual app backup
2017-09-28 10:22:10 -07:00
f8f6c7d93e
Add progress detail when rotating snapshots
2017-09-28 09:29:46 -07:00
56ee4d8e25
Remove old cache files when backup settings is changed
2017-09-27 21:04:46 -07:00
a34180c27b
Add format to backupsdb
...
Call remove/removeDir based on the format
2017-09-27 18:02:30 -07:00
44a7a2579c
rework backup status
...
* show backup progress even if not initiated by UI
* display backup progress in separate line
2017-09-27 15:07:15 -07:00
39f0e476f2
Start out empty if cache file is missing
2017-09-27 12:09:19 -07:00
8d3fbc5432
Save backup logs and fix backup progress
2017-09-26 21:09:00 -07:00
859311f9e5
Process delete commands before add commands
...
This is required for cases where a dir becomes a file (or vice-versa)
2017-09-26 15:33:54 -07:00
a9e89b57d9
merge caas storage into s3 backend
2017-09-26 12:28:33 -07:00
4e68abe51d
Handle fs errors
2017-09-26 12:10:58 -07:00
d1efb2db56
remove bogus mkdir
2017-09-26 11:34:24 -07:00
0a60365143
Initial version of flat-file uploader
2017-09-26 00:17:11 -07:00
2f6cb3e913
set format in the backup ui
2017-09-26 00:01:36 -07:00
b0f85678d4
Implement downloadDir for flat-file format
2017-09-23 18:07:26 -07:00
e43413e063
implement remove dir in storage backends
2017-09-23 12:34:51 -07:00
e39a5c8872
preserve env in backuptask.js
2017-09-22 11:19:44 -07:00
b61b7f80b5
Add DO spaces
2017-09-21 12:25:39 -07:00
efa850614d
Add a s3-v4-compat provider
2017-09-21 12:13:45 -07:00
97da8717ca
Refactor backup strategy logic into backups.js
2017-09-20 14:09:55 -07:00
cbddb79d15
Resolve the id in rotateAppBackup
2017-09-20 09:38:55 -07:00
26f33a8e9b
Send resolved path to the storage APIs
2017-09-19 21:58:35 -07:00
952b1f6304
Make backuptask call back into backups.js
2017-09-19 20:27:49 -07:00
84649b9471
Bring back backuptask
...
This is required for various small reasons:
* dir iteration with a way to pass messagein back to the upload() easily
* can be killed independently of box code
* allows us to run sync (blocking) commands in the upload logic
2017-09-19 12:32:38 -07:00
c351660a9a
Implement backup rotation
...
Always upload to 'snapshot' dir and then rotate it. This will allow
us to keep pushing incrementally to 'snapshot' and do server side
rotations.
2017-09-18 21:17:34 -07:00
f5d32a9178
copyBackup -> copy
2017-09-18 14:29:15 -07:00
7fc45b3215
Refactor out the backup snapshot logic
2017-09-18 12:43:11 -07:00
be03dd0821
remove unused require
2017-09-18 10:38:26 -07:00
781f543e87
Rename API calls in the storage backend
2017-09-17 18:50:29 -07:00
6525a467a2
Rework backuptask into tar.js
...
This makes it easy to integrate another backup strategy
as the next step
2017-09-17 18:50:26 -07:00
6cddd61a24
Fix style
2017-09-17 18:50:23 -07:00
Girish Ramakrishnan