Commit Graph

167 Commits

Author SHA1 Message Date
Girish Ramakrishnan 9635f9aa24 Use key to determine if we should encrypt or not
When encrypting we use the .enc extension. When not encrypting, we
use the plain .tar.gz extension.

Fixes #315
2017-04-28 14:50:20 -07:00
Girish Ramakrishnan 893f9d87bc make s3 upload use queueSize of 1 2017-04-28 14:50:08 -07:00
Girish Ramakrishnan bfda0d4891 drop support for old format backups 2017-04-28 14:45:44 -07:00
Girish Ramakrishnan ca5723bbc7 more work on tests
disable the migrate tests for now
2017-04-24 15:45:23 -07:00
Girish Ramakrishnan f79e1993cb use debug() more 2017-04-23 22:10:44 -07:00
Girish Ramakrishnan aeba8e8fd2 Use retentionSecs to cleanup backups
Part of #310
2017-04-23 19:34:00 -07:00
Girish Ramakrishnan d91e8bb87b add minio as a separate backend
Fixes #308
2017-04-22 13:34:43 -07:00
Johannes Zellner a7d7935451 Remove backup download code 2017-04-22 22:17:29 +02:00
Girish Ramakrishnan 1cca0aee6e refactor targz logic into separate file 2017-04-21 15:45:45 -07:00
Girish Ramakrishnan be73ec4b66 Add oldFormat support to caas and s3 backends 2017-04-21 15:06:54 -07:00
Girish Ramakrishnan 0aea7cc347 Add progress-stream for upload/download progress 2017-04-21 12:07:01 -07:00
Johannes Zellner 8fb1bc29d1 Fix storage tests for filesystem and s3 2017-04-21 17:21:10 +02:00
Johannes Zellner 33492333c7 Fix backupDone() for caas 2017-04-21 10:31:43 +02:00
Johannes Zellner 0471a14894 We always encrypt even with empty key so make that clear with the file extension 2017-04-21 09:59:27 +02:00
Girish Ramakrishnan 1cd36319ff s3: support setting the signatureVersion 2017-04-20 19:56:06 -07:00
Girish Ramakrishnan a65611a37b Fix error handling in s3 backend 2017-04-20 19:27:12 -07:00
Girish Ramakrishnan 0584ace954 Remove StorageError from backends 2017-04-20 18:12:29 -07:00
Girish Ramakrishnan 83f36981f7 Make box backups work again 2017-04-20 16:26:42 -07:00
Johannes Zellner 788873e858 Add error callbacks for backup in filesystem and s3 2017-04-20 16:18:27 +02:00
Johannes Zellner 9f117fcfdc Also simplify the s3 backend 2017-04-20 15:35:52 +02:00
Johannes Zellner 65af062c33 Cleanup linter issues 2017-04-19 13:20:24 +02:00
Johannes Zellner d5db9657ca Add s3 mock for the s3 storage tests 2017-04-18 19:15:56 +02:00
Johannes Zellner ad9cb00f13 Add s3 storage removeBackup() and getDownloadStream() implementations 2017-04-18 17:33:59 +02:00
Johannes Zellner cec52e14f6 Remove s3 backup test shell script 2017-04-18 16:58:27 +02:00
Johannes Zellner 261d15f0f7 Handle S3 not found errors 2017-04-18 16:47:49 +02:00
Johannes Zellner 14fe1dde58 s3 storage backend with backup and restore implemented 2017-04-18 16:28:39 +02:00
Girish Ramakrishnan 31a9437b2c Add backupDone hook 2017-01-04 16:23:12 -08:00
Johannes Zellner b2dbb5a100 Fixup bugs with updated backup scripts 2016-12-12 09:51:52 +01:00
Johannes Zellner 816911d071 Make s3 backup scripts aware of endpoints
Part of #123
2016-12-12 09:51:52 +01:00
Johannes 5ba86d5c35 Use aws s3 cli to test credentials
This allows us to test the exact same usage of the api
through the cli tool, not the javascript api
2016-10-28 16:36:05 +02:00
Johannes Zellner 6e6d4f7413 Actually verify s3 credentials by using the api 2016-10-11 11:46:28 +02:00
Johannes Zellner cac85b17bc Add backup config test for each backend 2016-10-11 11:36:25 +02:00
Johannes Zellner c7e5c09bb9 Adjust removeBackup() api 2016-10-10 15:45:48 +02:00
Johannes Zellner 1b3ae1f178 Add new storage.removeBackup() api
This currently is only used in the filesystem backend,
but may be expanded to also cleanup S3 in the future
2016-10-10 15:45:48 +02:00
Johannes Zellner 4a9a6dc232 Move backup config fetching into storage backend 2016-09-20 09:25:48 +02:00
Johannes Zellner 6352064e6c Add backup download route if backend supports it 2016-09-20 09:25:48 +02:00
Johannes Zellner c9c1964e09 The storage backends dont need a backup listing function 2016-09-20 09:25:48 +02:00
Johannes Zellner e8be76f2e8 Fixup typos 2016-09-20 09:25:48 +02:00
Johannes Zellner bf177473fe Rename getBackupDetails() -> getBoxBackupDetails() 2016-09-20 09:25:48 +02:00
Johannes Zellner 2ce768e29a Refactor getAppBackupCredentials() 2016-09-20 09:25:48 +02:00
Johannes Zellner 96c8f96c52 Group exports 2016-09-20 09:25:48 +02:00
Johannes Zellner 83ed87a8eb Refactor getBackupCredentials() 2016-09-20 09:25:48 +02:00
Johannes Zellner 10e7f27b16 Actually we need to specify the signatureVersion... 2016-09-13 12:07:09 +02:00
Johannes Zellner 8de0746ac8 Revert "Use S3 signature versoin 4 to support all regions"
If we set the correct region name, the signature version is selected
automatically

This reverts commit 1e22cc3236.
2016-09-12 14:27:47 +02:00
Johannes Zellner 1e22cc3236 Use S3 signature versoin 4 to support all regions
http://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html
2016-09-09 20:07:57 +02:00
Girish Ramakrishnan 22d86ff5b9 create signed urls that are valid for a day
sometimes the downloads take overly long and it's annoying that they
expire so soon.
2016-08-24 17:53:31 -07:00
Girish Ramakrishnan 4ed368cdd8 remove getBackupUrl 2016-04-10 22:12:06 -07:00
Girish Ramakrishnan 08b5d7003d expose getBackupCredentials from storage api 2016-04-10 10:55:59 -07:00
Girish Ramakrishnan 8aacc503a6 Revert "getRestoreUrl now uses caas restore api"
This reverts commit f9fc9325a8995dc0a9cb1dfcf22fb27eca697a89.

For now, we can simply assume that caas is s3 based.
2016-04-04 15:57:32 -07:00
Girish Ramakrishnan ec160fe45f make getBackupUrl return id as well 2016-04-04 12:45:09 -07:00