Commit Graph

58 Commits

Author SHA1 Message Date
Girish Ramakrishnan 9635f9aa24 Use key to determine if we should encrypt or not
When encrypting we use the .enc extension. When not encrypting, we
use the plain .tar.gz extension.

Fixes #315
2017-04-28 14:50:20 -07:00
Girish Ramakrishnan 893f9d87bc make s3 upload use queueSize of 1 2017-04-28 14:50:08 -07:00
Girish Ramakrishnan bfda0d4891 drop support for old format backups 2017-04-28 14:45:44 -07:00
Johannes Zellner a723203b28 Fix typo of missing data argument 2017-04-25 13:48:12 +02:00
Girish Ramakrishnan ca5723bbc7 more work on tests
disable the migrate tests for now
2017-04-24 15:45:23 -07:00
Girish Ramakrishnan f79e1993cb use debug() more 2017-04-23 22:10:44 -07:00
Girish Ramakrishnan aeba8e8fd2 Use retentionSecs to cleanup backups
Part of #310
2017-04-23 19:34:00 -07:00
Johannes Zellner a7d7935451 Remove backup download code 2017-04-22 22:17:29 +02:00
Girish Ramakrishnan 1cca0aee6e refactor targz logic into separate file 2017-04-21 15:45:45 -07:00
Girish Ramakrishnan be73ec4b66 Add oldFormat support to caas and s3 backends 2017-04-21 15:06:54 -07:00
Girish Ramakrishnan 0aea7cc347 Add progress-stream for upload/download progress 2017-04-21 12:07:01 -07:00
Johannes Zellner 33492333c7 Fix backupDone() for caas 2017-04-21 10:31:43 +02:00
Johannes Zellner 0471a14894 We always encrypt even with empty key so make that clear with the file extension 2017-04-21 09:59:27 +02:00
Girish Ramakrishnan a6c2c608e4 storage/caas: bring it upto speed for new backup design 2017-04-20 20:12:45 -07:00
Girish Ramakrishnan 0584ace954 Remove StorageError from backends 2017-04-20 18:12:29 -07:00
Johannes Zellner 40f5d0412b Bring the caas storage backend up to speed
This is mostly the same as the s3 backend at the moment
2017-04-19 13:30:05 +02:00
Girish Ramakrishnan 794efb5ef5 Merge backupDone webhook into caas storage backend 2017-01-04 16:29:25 -08:00
Girish Ramakrishnan 31a9437b2c Add backupDone hook 2017-01-04 16:23:12 -08:00
Johannes Zellner 6e1751d0ed Set empty endpoint url for caas storage backend
Caas storage backend also uses the s3 code branches
2016-12-12 14:00:07 +01:00
Johannes Zellner cac85b17bc Add backup config test for each backend 2016-10-11 11:36:25 +02:00
Johannes Zellner c7e5c09bb9 Adjust removeBackup() api 2016-10-10 15:45:48 +02:00
Johannes Zellner 1b3ae1f178 Add new storage.removeBackup() api
This currently is only used in the filesystem backend,
but may be expanded to also cleanup S3 in the future
2016-10-10 15:45:48 +02:00
Johannes Zellner 4a9a6dc232 Move backup config fetching into storage backend 2016-09-20 09:25:48 +02:00
Johannes Zellner 6352064e6c Add backup download route if backend supports it 2016-09-20 09:25:48 +02:00
Johannes Zellner c9c1964e09 The storage backends dont need a backup listing function 2016-09-20 09:25:48 +02:00
Johannes Zellner f348fedd50 Caas backend has to use the AWS credentials provided by appstore 2016-09-20 09:25:48 +02:00
Johannes Zellner 2a92d4772c Fix typo 2016-09-20 09:25:48 +02:00
Johannes Zellner bf177473fe Rename getBackupDetails() -> getBoxBackupDetails() 2016-09-20 09:25:48 +02:00
Johannes Zellner 2ce768e29a Refactor getAppBackupCredentials() 2016-09-20 09:25:48 +02:00
Johannes Zellner 96c8f96c52 Group exports 2016-09-20 09:25:48 +02:00
Johannes Zellner 83ed87a8eb Refactor getBackupCredentials() 2016-09-20 09:25:48 +02:00
Girish Ramakrishnan c1bb264065 Set a timeout for superagent
The default is 'no timeout' and it will wait for the response forever.

https://github.com/visionmedia/superagent/issues/17#issuecomment-207742985
2016-09-12 13:06:18 -07:00
Girish Ramakrishnan 22d86ff5b9 create signed urls that are valid for a day
sometimes the downloads take overly long and it's annoying that they
expire so soon.
2016-08-24 17:53:31 -07:00
Girish Ramakrishnan 4ed368cdd8 remove getBackupUrl 2016-04-10 22:12:06 -07:00
Girish Ramakrishnan 4565291c1c use aws-cli to upload box backups 2016-04-10 18:22:05 -07:00
Girish Ramakrishnan 08b5d7003d expose getBackupCredentials from storage api 2016-04-10 10:55:59 -07:00
Girish Ramakrishnan 497eaea65e bump expiry to 60 mins 2016-04-04 16:02:13 -07:00
Girish Ramakrishnan 8aacc503a6 Revert "getRestoreUrl now uses caas restore api"
This reverts commit f9fc9325a8995dc0a9cb1dfcf22fb27eca697a89.

For now, we can simply assume that caas is s3 based.
2016-04-04 15:57:32 -07:00
Girish Ramakrishnan ec160fe45f make getBackupUrl return id as well 2016-04-04 12:45:09 -07:00
Girish Ramakrishnan c9ef0056e0 rename getSignedUploadUrl to getBackupUrl 2016-04-04 12:01:47 -07:00
Girish Ramakrishnan efb228cf5e getRestoreUrl now uses caas restore api 2016-04-04 11:57:29 -07:00
Girish Ramakrishnan 3135783fe3 rename getSignedDownloadUrl to getRestoreUrl 2016-04-04 11:43:56 -07:00
Girish Ramakrishnan 496f530b9f sessionToken is required in credentials (when signing) 2016-04-04 11:23:38 -07:00
Girish Ramakrishnan d839f0b762 remove redundant session token 2016-04-03 23:23:23 -07:00
Girish Ramakrishnan 16a65fb185 drop configJson
The initial idea was to store exactly where the backups are stored.
But this only causes problems for migrations where the bucket might
change and clones where the prefix (box.id) changes.

Thus, it's best to leave the url creation to the caas side. (That
has to be done in another change)
2016-04-03 22:55:08 -07:00
Girish Ramakrishnan 357f6f0552 use same region as what we uploaded to 2016-04-02 13:32:14 -07:00
Girish Ramakrishnan b16aa4c007 check for region as well 2016-04-02 13:31:12 -07:00
Girish Ramakrishnan 3ea2070cdb pass filename (it is not part of the config!) 2016-03-31 09:53:56 -07:00
Girish Ramakrishnan fc11484b51 pick region from apiConfig if present 2016-03-31 09:48:38 -07:00
Girish Ramakrishnan b4ddfa94cc rename to apiConfig 2016-03-31 09:48:13 -07:00