Commit Graph

3173 Commits

Author SHA1 Message Date
Girish Ramakrishnan
1e72d2d651 remove debugs (too noisy) 2017-10-16 12:34:09 -07:00
Girish Ramakrishnan
89fc8efc67 Save as empty array if find output is empty 2017-10-16 10:54:48 -07:00
Girish Ramakrishnan
241dbf160e Remove Unused required 2017-10-15 14:07:03 -07:00
Girish Ramakrishnan
e1cb91ca76 bump mail container 2017-10-13 22:36:54 -07:00
Girish Ramakrishnan
709c742c46 Fix tests 2017-10-12 21:14:13 -07:00
Girish Ramakrishnan
ecad9c499c Port binding conflict can never happen in update route 2017-10-12 21:04:38 -07:00
Girish Ramakrishnan
ed0879ffcd Stop the app only after the backup completed
App backup can take a long time or possibly not work at all. For such
cases, do not stop the app or leave it in some errored state.

newConfigJson is the new config to be updated to. This ensures that
the db has correct app info during the update.
2017-10-12 18:10:41 -07:00
Girish Ramakrishnan
61e2878b08 save/restore exec bit in files
this covers the case where user might stash some executable files
that are used by plugins.
2017-10-12 16:18:11 -07:00
Girish Ramakrishnan
d97034bfb2 Follow backup format for box backups as well 2017-10-12 11:02:52 -07:00
Girish Ramakrishnan
dd68c8f91f Various backup fixes 2017-10-12 11:02:48 -07:00
Girish Ramakrishnan
28ce5f41e3 handle errors in log stream 2017-10-11 12:55:56 -07:00
Girish Ramakrishnan
db8c5a116f Typo 2017-10-11 10:30:03 -07:00
Girish Ramakrishnan
1444bb038f only upload needs to be retried
copy/delete are already retried in the sdk code
2017-10-11 00:08:41 -07:00
Girish Ramakrishnan
ac9e421ecf improved backup progress and logging 2017-10-10 22:49:38 -07:00
Girish Ramakrishnan
b60cbe5a55 move constant 2017-10-10 19:47:21 -07:00
Girish Ramakrishnan
56d794745b Sprinkle retries in syncer logic 2017-10-10 14:25:03 -07:00
Girish Ramakrishnan
fd3b73bea2 typo in format name 2017-10-10 13:54:54 -07:00
Girish Ramakrishnan
78807782df Various hacks for exoscale-sos
SOS does not like multipart uploads. They just fail randomly.

As a fix, we try to detect filesystem files and skip multipart uploads
for files < 5GB. For > 5GB, we do multipart upload anyways (mostly fails).

The box backup is switched to flat-file for exoscale for the reason
above.
2017-10-10 11:03:20 -07:00
Girish Ramakrishnan
754b29b263 Start out empty if the previous run errored 2017-10-09 20:12:21 -07:00
Girish Ramakrishnan
9f97f48634 Add note on s3ForcePathStyle 2017-10-09 18:46:14 -07:00
Girish Ramakrishnan
91ec2eaaf5 sos: "/" must separate bucket and key name 2017-10-09 11:50:22 -07:00
Girish Ramakrishnan
f8d3a7cadd Bump mail container (fixes spam crash) 2017-10-06 16:45:21 -07:00
Girish Ramakrishnan
d04a09b015 Add note on bumping major infra version 2017-10-06 15:52:04 -07:00
Girish Ramakrishnan
f0dd90a1f5 listObjectsV2 does not work on some S3 providers
specifically, cloudscale does not support it
2017-10-05 12:07:14 -07:00
Girish Ramakrishnan
ee8ee8e786 KeyCount is not set on some S3 providers 2017-10-05 11:36:54 -07:00
Girish Ramakrishnan
ee1a4411f8 Do not crash if prefix is empty string
('' || undefined) will return undefined ...
2017-10-05 11:08:01 -07:00
Girish Ramakrishnan
df6e6cb071 Allow s3 backend to accept self-signed certs
Fixes #316
2017-10-05 10:14:55 -07:00
Girish Ramakrishnan
ca502a2d55 Display error code 2017-10-04 22:34:44 -07:00
Girish Ramakrishnan
b9efb0b50b Fix callback invokation 2017-10-04 19:28:40 -07:00
Johannes Zellner
3fb5034ebd Ensure we setup the correct OAuth redirectURI if altDomain is used 2017-10-05 01:10:25 +02:00
Girish Ramakrishnan
afed3f3725 Remove duplicate debug 2017-10-04 15:08:26 -07:00
Girish Ramakrishnan
c3d7d867be Do not set logCallback 2017-10-04 12:32:12 -07:00
Girish Ramakrishnan
96c16cd5d2 remove debug 2017-10-04 11:54:17 -07:00
Girish Ramakrishnan
af182e3df6 caas: cache the creds, otherwise we bombard the server 2017-10-04 11:49:38 -07:00
Girish Ramakrishnan
d70ff7cd5b Make copy() return event emitter
This way the storage logic does not need to rely on progress
2017-10-04 11:02:50 -07:00
Johannes Zellner
38331e71e2 Ensure all S3 CopySource properties are URI encoded 2017-10-04 19:07:08 +02:00
Johannes Zellner
322a9a18d7 Use multipart copy for s3 and files larger than 5GB 2017-10-04 18:56:23 +02:00
Dennis Schwerdel
e3f3241966 Added user agent to health checks 2017-10-04 13:05:00 +02:00
Girish Ramakrishnan
b85bc3aa01 s3: Must encode copySource
https://github.com/aws/aws-sdk-js/issues/1302
2017-10-03 15:51:05 -07:00
Girish Ramakrishnan
01154d0ae6 s3: better error messages 2017-10-03 14:46:59 -07:00
Girish Ramakrishnan
6494050d66 Make removeDir less noisy 2017-10-03 01:22:37 -07:00
Girish Ramakrishnan
8c7223ceed Fix cleanup logic to use the app backup format
box backup and app backup can have different format
2017-10-03 00:56:34 -07:00
Girish Ramakrishnan
21afc71d89 add tests for storage backends 2017-10-02 23:08:16 -07:00
Girish Ramakrishnan
7bf70956a1 fix tests 2017-10-02 18:42:13 -07:00
Girish Ramakrishnan
9e9b8b095e Provider dhparams.pem to the mail container 2017-10-02 01:51:28 -07:00
Girish Ramakrishnan
0f543e6703 s3: add progress detail
this is a bit of a hack and we should add another way to set the progress
(maybe via backups.setProgress or via a progress callback). this is because
some methods like removeDir can be called from backuptask and from box code.
2017-10-01 18:25:51 -07:00
Girish Ramakrishnan
f9973e765c Add backup cleanup eventlog 2017-10-01 10:35:50 -07:00
Girish Ramakrishnan
e089851ae9 add debugs 2017-09-30 20:36:08 -07:00
Girish Ramakrishnan
c524d68c2f fix crash when cleaning up snapshots 2017-09-30 20:31:41 -07:00
Girish Ramakrishnan
5cccb50a31 fix backup cleanup logic 2017-09-30 18:38:45 -07:00