Girish Ramakrishnan
d70ff7cd5b
Make copy() return event emitter
...
This way the storage logic does not need to rely on progress
2017-10-04 11:02:50 -07:00
Johannes Zellner
38331e71e2
Ensure all S3 CopySource properties are URI encoded
2017-10-04 19:07:08 +02:00
Johannes Zellner
322a9a18d7
Use multipart copy for s3 and files larger than 5GB
2017-10-04 18:56:23 +02:00
Dennis Schwerdel
e3f3241966
Added user agent to health checks
2017-10-04 13:05:00 +02:00
Girish Ramakrishnan
b85bc3aa01
s3: Must encode copySource
...
https://github.com/aws/aws-sdk-js/issues/1302
2017-10-03 15:51:05 -07:00
Girish Ramakrishnan
01154d0ae6
s3: better error messages
2017-10-03 14:46:59 -07:00
Girish Ramakrishnan
6494050d66
Make removeDir less noisy
2017-10-03 01:22:37 -07:00
Girish Ramakrishnan
8c7223ceed
Fix cleanup logic to use the app backup format
...
box backup and app backup can have different format
2017-10-03 00:56:34 -07:00
Girish Ramakrishnan
21afc71d89
add tests for storage backends
2017-10-02 23:08:16 -07:00
Girish Ramakrishnan
7bf70956a1
fix tests
2017-10-02 18:42:13 -07:00
Girish Ramakrishnan
9e9b8b095e
Provider dhparams.pem to the mail container
2017-10-02 01:51:28 -07:00
Girish Ramakrishnan
0f543e6703
s3: add progress detail
...
this is a bit of a hack and we should add another way to set the progress
(maybe via backups.setProgress or via a progress callback). this is because
some methods like removeDir can be called from backuptask and from box code.
2017-10-01 18:25:51 -07:00
Girish Ramakrishnan
f9973e765c
Add backup cleanup eventlog
2017-10-01 10:35:50 -07:00
Girish Ramakrishnan
e089851ae9
add debugs
2017-09-30 20:36:08 -07:00
Girish Ramakrishnan
c524d68c2f
fix crash when cleaning up snapshots
2017-09-30 20:31:41 -07:00
Girish Ramakrishnan
5cccb50a31
fix backup cleanup logic
2017-09-30 18:38:45 -07:00
Girish Ramakrishnan
3d375b687a
style: Fix quoting
2017-09-30 18:26:38 -07:00
Girish Ramakrishnan
a93d453963
rename flat-file to rsync
...
not a name I like but cannot come up with anything better
https://en.wikipedia.org/wiki/Flat_file_database
the term 'rsync format' seems to be used in a few places
2017-09-30 14:19:19 -07:00
Girish Ramakrishnan
d5ba73716b
add emptydirs test
2017-09-29 15:29:22 -07:00
Girish Ramakrishnan
954224dafb
make syncer track directories
2017-09-29 15:29:18 -07:00
Johannes Zellner
8b341e2bf8
Only make nginx listen on ipv6 connections if it is supported by the system
...
Could not decide on the ejs formatting, never nice for me
2017-09-29 19:43:37 +02:00
Johannes Zellner
78fb9401ee
Add config.hasIPv6()
2017-09-29 19:43:37 +02:00
Girish Ramakrishnan
4a5cbab194
Do not remove parent directory in fs.remove()
...
Do the pruning in the cleanup logic instead
2017-09-28 20:55:45 -07:00
Girish Ramakrishnan
19999abc50
s3: fix restore
2017-09-28 14:35:49 -07:00
Girish Ramakrishnan
5123b669d7
remove options.concurrency
2017-09-28 12:20:15 -07:00
Girish Ramakrishnan
565c8445e1
make backup progress work for per-app backups
2017-09-28 11:17:48 -07:00
Girish Ramakrishnan
404a019c56
s3: Check IsTruncated before accessing Contents
2017-09-28 10:36:56 -07:00
Girish Ramakrishnan
24dee80aa6
Make box backups always tarball based
...
this makes cloudron easy to restore. in the future, if required,
we can move out the mail data as a separate virtual app backup
2017-09-28 10:22:10 -07:00
Girish Ramakrishnan
ce6df4bf96
Disable encryption for flat-file for now
2017-09-28 09:47:18 -07:00
Girish Ramakrishnan
f8f6c7d93e
Add progress detail when rotating snapshots
2017-09-28 09:29:46 -07:00
Girish Ramakrishnan
bafc6dce98
s3: refactor out directory listing
2017-09-27 21:59:51 -07:00
Girish Ramakrishnan
56ee4d8e25
Remove old cache files when backup settings is changed
2017-09-27 21:04:46 -07:00
Girish Ramakrishnan
eeef221b4e
Fix race where pipe finishes before file is created
...
When there are 0 length files, this is easily reproducible.
2017-09-27 19:40:26 -07:00
Girish Ramakrishnan
4674653982
compare size and inode as well
2017-09-27 19:39:03 -07:00
Girish Ramakrishnan
a34180c27b
Add format to backupsdb
...
Call remove/removeDir based on the format
2017-09-27 18:02:30 -07:00
Girish Ramakrishnan
aa8ce2c62e
Use graphite 0.12.0
...
this fixes an issue where carbon does not startup properly
if a previous pid file was present
2017-09-27 15:35:55 -07:00
Girish Ramakrishnan
b3c6b8aa15
do not spawn process just for chown
2017-09-27 15:07:19 -07:00
Girish Ramakrishnan
44a7a2579c
rework backup status
...
* show backup progress even if not initiated by UI
* display backup progress in separate line
2017-09-27 15:07:15 -07:00
Girish Ramakrishnan
39f0e476f2
Start out empty if cache file is missing
2017-09-27 12:09:19 -07:00
Girish Ramakrishnan
003dc0dbaf
Add todo
2017-09-27 11:50:49 -07:00
Girish Ramakrishnan
e39329218d
Make tests work
2017-09-27 11:38:43 -07:00
Girish Ramakrishnan
8d3fbc5432
Save backup logs and fix backup progress
2017-09-26 21:09:00 -07:00
Girish Ramakrishnan
2780de631e
writable streams emit finish
2017-09-26 16:43:51 -07:00
Girish Ramakrishnan
399c756735
use exec so that filenames do not have to be escaped
2017-09-26 15:53:42 -07:00
Girish Ramakrishnan
859311f9e5
Process delete commands before add commands
...
This is required for cases where a dir becomes a file (or vice-versa)
2017-09-26 15:33:54 -07:00
Girish Ramakrishnan
a9e89b57d9
merge caas storage into s3 backend
2017-09-26 12:28:33 -07:00
Girish Ramakrishnan
4e68abe51d
Handle fs errors
2017-09-26 12:10:58 -07:00
Girish Ramakrishnan
12083f5608
Ignore all special files
2017-09-26 11:41:01 -07:00
Girish Ramakrishnan
d1efb2db56
remove bogus mkdir
2017-09-26 11:34:24 -07:00
Girish Ramakrishnan
f122f46fe2
Generate new index file by appending to file
2017-09-26 07:57:20 -07:00
