Commit Graph

60 Commits

Author SHA1 Message Date
Girish Ramakrishnan 20d6da8230 add debugs 2016-04-20 19:40:58 -07:00
Girish Ramakrishnan f159cacfbb Use same timestamp for archive and config
This fixes a very curious case:
1. App has backup.
2. App dies.
3. Box backs up. This make it reuse the backup. But it generates wrong config file timestamp.
4. Box cannot update anymore. This is because the backup of app fails - it tries to reuse
   the backup and that fails with AccessDenied because the timestamp above is wrong!
2016-04-20 19:37:00 -07:00
Girish Ramakrishnan 3bdbcff811 add debug 2016-04-10 22:34:55 -07:00
Girish Ramakrishnan ea3bd6d71d remove trailing comma 2016-04-10 22:29:09 -07:00
Girish Ramakrishnan d5cc96b1ff clean up backups code 2016-04-10 22:24:01 -07:00
Girish Ramakrishnan 4ed368cdd8 remove getBackupUrl 2016-04-10 22:12:06 -07:00
Girish Ramakrishnan 5229222014 getBackupCredentials is never used 2016-04-10 22:09:29 -07:00
Girish Ramakrishnan 9b0aa331e1 remove unused function 2016-04-10 22:08:11 -07:00
Girish Ramakrishnan 70cc073b1c only add to backupdb when the backup succeeded 2016-04-10 21:55:08 -07:00
Girish Ramakrishnan 29502fd8af remove unused exports 2016-04-10 21:52:01 -07:00
Girish Ramakrishnan b2668579d6 pass appid to backup script 2016-04-10 21:41:53 -07:00
Girish Ramakrishnan ba663faa64 fix debug 2016-04-10 21:39:45 -07:00
Girish Ramakrishnan 8db76f6b70 backup swap is not required anymore 2016-04-10 20:55:59 -07:00
Girish Ramakrishnan 322e9faee7 rework backup code
move all the backup code into backups.js
2016-04-10 20:41:08 -07:00
Girish Ramakrishnan af9d489395 backup apps use aws-cli 2016-04-10 18:47:25 -07:00
Girish Ramakrishnan 4565291c1c use aws-cli to upload box backups 2016-04-10 18:22:05 -07:00
Girish Ramakrishnan 8b3a44b33c Add getBackupCredentials to backups API 2016-04-10 11:01:59 -07:00
Johannes Zellner 0b1dcd2940 Use userdb.getByUsername() instead of get() 2016-04-05 09:27:32 +02:00
Girish Ramakrishnan ec160fe45f make getBackupUrl return id as well 2016-04-04 12:45:09 -07:00
Girish Ramakrishnan bbff195863 rename filename to id 2016-04-04 12:20:56 -07:00
Girish Ramakrishnan e528dbcfc0 creationTime is redundant 2016-04-04 12:13:54 -07:00
Girish Ramakrishnan c9ef0056e0 rename getSignedUploadUrl to getBackupUrl 2016-04-04 12:01:47 -07:00
Girish Ramakrishnan 3135783fe3 rename getSignedDownloadUrl to getRestoreUrl 2016-04-04 11:43:56 -07:00
Girish Ramakrishnan d839f0b762 remove redundant session token 2016-04-03 23:23:23 -07:00
Girish Ramakrishnan 16a65fb185 drop configJson
The initial idea was to store exactly where the backups are stored.
But this only causes problems for migrations where the bucket might
change and clones where the prefix (box.id) changes.

Thus, it's best to leave the url creation to the caas side. (That
has to be done in another change)
2016-04-03 22:55:08 -07:00
Girish Ramakrishnan 3ea2070cdb pass filename (it is not part of the config!) 2016-03-31 09:53:56 -07:00
Girish Ramakrishnan 9e7ae1a4f7 we really need a better linter 2016-03-31 09:38:40 -07:00
Girish Ramakrishnan d27159275b pick region as well 2016-03-31 09:34:57 -07:00
Girish Ramakrishnan 6c2ae756f1 fix usage of backupInfo 2016-03-31 09:23:41 -07:00
Girish Ramakrishnan 39cc5d07d1 use the backupdb config to determine bucket and prefix 2016-03-31 00:50:56 -07:00
Girish Ramakrishnan df39384056 do not save backup secrets in database 2016-03-30 23:39:48 -07:00
Girish Ramakrishnan c097651a88 store backup configuration as part of backups table 2016-03-30 15:04:39 -07:00
Girish Ramakrishnan e2b7ec3ffd store filename with tar.gz extension 2016-03-08 16:47:53 -08:00
girish@cloudron.io d157bf30f3 remove box backups from the database 2016-03-08 08:52:20 -08:00
girish@cloudron.io 7996b32022 add backups to the database
ideally, these should be done _after_ the backup is successful and not when
the backup url is generated.

we had a discussion on why need backupdb to start with. Some rationale includes:
1. we can use it as a FK constraint (like make sure you delete backups when you delete app)
2. have labels for backups
3. dependancy relation from box backup to apps
4. s3 reverse sort is little painful and requires us to get all items in bulk and sort in JS
   (also requires us to change our backup filename format)
5. any metadata storage requires database

The cons include:
1. s3 and this db go out of sync
2. db entry is useless if s3 file is missing
2016-03-08 08:42:00 -08:00
Girish Ramakrishnan 4dd82d10ad backup: ensure same timestamp for app data and config 2016-03-07 12:13:54 -08:00
Johannes Zellner ea4862d351 Strictly assert on app 2016-01-29 11:54:40 +01:00
Johannes Zellner 3e4d62329e Also copy the backup config json 2016-01-29 11:54:15 +01:00
Johannes Zellner 7b1d906494 Add backups.getAppBackupConfigUrl() 2016-01-29 11:44:24 +01:00
Johannes Zellner 030faaa5d1 Remove unused information within backup listing 2015-12-30 20:31:00 +01:00
Girish Ramakrishnan a5ef9ff372 Add getAllPaged to storage api 2015-11-08 22:41:13 -08:00
Girish Ramakrishnan 8df7f17186 load backup config from settingsdb 2015-11-07 22:06:09 -08:00
Girish Ramakrishnan 12200f2e0d split aws code into storage backends 2015-11-06 18:22:29 -08:00
Girish Ramakrishnan a853afc407 backups: add api call 2015-11-06 18:14:59 -08:00
Girish Ramakrishnan de471b0012 return BackupError and not SubdomainError 2015-11-06 18:08:25 -08:00
Girish Ramakrishnan 7510c9fe29 Fix typo 2015-09-21 15:57:06 -07:00
Girish Ramakrishnan 78a1d53728 copy old backup as failed/errored apps
This ensures that
a) we don't get emails from janitor about bad app backups
b) that the backups are persisted over the s3 lifecycle

Fixes #493
2015-09-21 15:03:10 -07:00
Girish Ramakrishnan e9b078cd58 add backups.copyLastBackup 2015-09-21 14:14:43 -07:00
Girish Ramakrishnan baecbf783c journalctl seems to barf on this debug 2015-09-15 20:50:22 -07:00
Girish Ramakrishnan b4a34e6432 Explicity debug the fields
for some reason, journalctl barfs on this line
2015-09-15 14:55:20 -07:00