Commit Graph

67 Commits

Author SHA1 Message Date
Girish Ramakrishnan
9635f9aa24 Use key to determine if we should encrypt or not
When encrypting we use the .enc extension. When not encrypting, we
use the plain .tar.gz extension.

Fixes #315
2017-04-28 14:50:20 -07:00
Girish Ramakrishnan
893f9d87bc make s3 upload use queueSize of 1 2017-04-28 14:50:08 -07:00
Girish Ramakrishnan
bfda0d4891 drop support for old format backups 2017-04-28 14:45:44 -07:00
Girish Ramakrishnan
ca5723bbc7 more work on tests
disable the migrate tests for now
2017-04-24 15:45:23 -07:00
Girish Ramakrishnan
f79e1993cb use debug() more 2017-04-23 22:10:44 -07:00
Girish Ramakrishnan
aeba8e8fd2 Use retentionSecs to cleanup backups
Part of #310
2017-04-23 19:34:00 -07:00
Girish Ramakrishnan
d91e8bb87b add minio as a separate backend
Fixes #308
2017-04-22 13:34:43 -07:00
Johannes Zellner
a7d7935451 Remove backup download code 2017-04-22 22:17:29 +02:00
Girish Ramakrishnan
1cca0aee6e refactor targz logic into separate file 2017-04-21 15:45:45 -07:00
Girish Ramakrishnan
be73ec4b66 Add oldFormat support to caas and s3 backends 2017-04-21 15:06:54 -07:00
Girish Ramakrishnan
0aea7cc347 Add progress-stream for upload/download progress 2017-04-21 12:07:01 -07:00
Johannes Zellner
8fb1bc29d1 Fix storage tests for filesystem and s3 2017-04-21 17:21:10 +02:00
Johannes Zellner
33492333c7 Fix backupDone() for caas 2017-04-21 10:31:43 +02:00
Johannes Zellner
0471a14894 We always encrypt even with empty key so make that clear with the file extension 2017-04-21 09:59:27 +02:00
Girish Ramakrishnan
1cd36319ff s3: support setting the signatureVersion 2017-04-20 19:56:06 -07:00
Girish Ramakrishnan
a65611a37b Fix error handling in s3 backend 2017-04-20 19:27:12 -07:00
Girish Ramakrishnan
0584ace954 Remove StorageError from backends 2017-04-20 18:12:29 -07:00
Girish Ramakrishnan
83f36981f7 Make box backups work again 2017-04-20 16:26:42 -07:00
Johannes Zellner
788873e858 Add error callbacks for backup in filesystem and s3 2017-04-20 16:18:27 +02:00
Johannes Zellner
9f117fcfdc Also simplify the s3 backend 2017-04-20 15:35:52 +02:00
Johannes Zellner
65af062c33 Cleanup linter issues 2017-04-19 13:20:24 +02:00
Johannes Zellner
d5db9657ca Add s3 mock for the s3 storage tests 2017-04-18 19:15:56 +02:00
Johannes Zellner
ad9cb00f13 Add s3 storage removeBackup() and getDownloadStream() implementations 2017-04-18 17:33:59 +02:00
Johannes Zellner
cec52e14f6 Remove s3 backup test shell script 2017-04-18 16:58:27 +02:00
Johannes Zellner
261d15f0f7 Handle S3 not found errors 2017-04-18 16:47:49 +02:00
Johannes Zellner
14fe1dde58 s3 storage backend with backup and restore implemented 2017-04-18 16:28:39 +02:00
Girish Ramakrishnan
31a9437b2c Add backupDone hook 2017-01-04 16:23:12 -08:00
Johannes Zellner
b2dbb5a100 Fixup bugs with updated backup scripts 2016-12-12 09:51:52 +01:00
Johannes Zellner
816911d071 Make s3 backup scripts aware of endpoints
Part of #123
2016-12-12 09:51:52 +01:00
Johannes
5ba86d5c35 Use aws s3 cli to test credentials
This allows us to test the exact same usage of the api
through the cli tool, not the javascript api
2016-10-28 16:36:05 +02:00
Johannes Zellner
6e6d4f7413 Actually verify s3 credentials by using the api 2016-10-11 11:46:28 +02:00
Johannes Zellner
cac85b17bc Add backup config test for each backend 2016-10-11 11:36:25 +02:00
Johannes Zellner
c7e5c09bb9 Adjust removeBackup() api 2016-10-10 15:45:48 +02:00
Johannes Zellner
1b3ae1f178 Add new storage.removeBackup() api
This currently is only used in the filesystem backend,
but may be expanded to also cleanup S3 in the future
2016-10-10 15:45:48 +02:00
Johannes Zellner
4a9a6dc232 Move backup config fetching into storage backend 2016-09-20 09:25:48 +02:00
Johannes Zellner
6352064e6c Add backup download route if backend supports it 2016-09-20 09:25:48 +02:00
Johannes Zellner
c9c1964e09 The storage backends dont need a backup listing function 2016-09-20 09:25:48 +02:00
Johannes Zellner
e8be76f2e8 Fixup typos 2016-09-20 09:25:48 +02:00
Johannes Zellner
bf177473fe Rename getBackupDetails() -> getBoxBackupDetails() 2016-09-20 09:25:48 +02:00
Johannes Zellner
2ce768e29a Refactor getAppBackupCredentials() 2016-09-20 09:25:48 +02:00
Johannes Zellner
96c8f96c52 Group exports 2016-09-20 09:25:48 +02:00
Johannes Zellner
83ed87a8eb Refactor getBackupCredentials() 2016-09-20 09:25:48 +02:00
Johannes Zellner
10e7f27b16 Actually we need to specify the signatureVersion... 2016-09-13 12:07:09 +02:00
Johannes Zellner
8de0746ac8 Revert "Use S3 signature versoin 4 to support all regions"
If we set the correct region name, the signature version is selected
automatically

This reverts commit 1e22cc3236.
2016-09-12 14:27:47 +02:00
Johannes Zellner
1e22cc3236 Use S3 signature versoin 4 to support all regions
http://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html
2016-09-09 20:07:57 +02:00
Girish Ramakrishnan
22d86ff5b9 create signed urls that are valid for a day
sometimes the downloads take overly long and it's annoying that they
expire so soon.
2016-08-24 17:53:31 -07:00
Girish Ramakrishnan
4ed368cdd8 remove getBackupUrl 2016-04-10 22:12:06 -07:00
Girish Ramakrishnan
08b5d7003d expose getBackupCredentials from storage api 2016-04-10 10:55:59 -07:00
Girish Ramakrishnan
8aacc503a6 Revert "getRestoreUrl now uses caas restore api"
This reverts commit f9fc9325a8995dc0a9cb1dfcf22fb27eca697a89.

For now, we can simply assume that caas is s3 based.
2016-04-04 15:57:32 -07:00
Girish Ramakrishnan
ec160fe45f make getBackupUrl return id as well 2016-04-04 12:45:09 -07:00