Commit Graph

188 Commits

Author SHA1 Message Date
Girish Ramakrishnan 4a5cbab194 Do not remove parent directory in fs.remove()
Do the pruning in the cleanup logic instead
2017-09-28 20:55:45 -07:00
Girish Ramakrishnan 19999abc50 s3: fix restore 2017-09-28 14:35:49 -07:00
Girish Ramakrishnan 565c8445e1 make backup progress work for per-app backups 2017-09-28 11:17:48 -07:00
Girish Ramakrishnan 24dee80aa6 Make box backups always tarball based
this makes cloudron easy to restore. in the future, if required,
we can move out the mail data as a separate virtual app backup
2017-09-28 10:22:10 -07:00
Girish Ramakrishnan f8f6c7d93e Add progress detail when rotating snapshots 2017-09-28 09:29:46 -07:00
Girish Ramakrishnan 56ee4d8e25 Remove old cache files when backup settings is changed 2017-09-27 21:04:46 -07:00
Girish Ramakrishnan a34180c27b Add format to backupsdb
Call remove/removeDir based on the format
2017-09-27 18:02:30 -07:00
Girish Ramakrishnan 44a7a2579c rework backup status
* show backup progress even if not initiated by UI
* display backup progress in separate line
2017-09-27 15:07:15 -07:00
Girish Ramakrishnan 39f0e476f2 Start out empty if cache file is missing 2017-09-27 12:09:19 -07:00
Girish Ramakrishnan 8d3fbc5432 Save backup logs and fix backup progress 2017-09-26 21:09:00 -07:00
Girish Ramakrishnan 859311f9e5 Process delete commands before add commands
This is required for cases where a dir becomes a file (or vice-versa)
2017-09-26 15:33:54 -07:00
Girish Ramakrishnan a9e89b57d9 merge caas storage into s3 backend 2017-09-26 12:28:33 -07:00
Girish Ramakrishnan 4e68abe51d Handle fs errors 2017-09-26 12:10:58 -07:00
Girish Ramakrishnan d1efb2db56 remove bogus mkdir 2017-09-26 11:34:24 -07:00
Girish Ramakrishnan 0a60365143 Initial version of flat-file uploader 2017-09-26 00:17:11 -07:00
Girish Ramakrishnan 2f6cb3e913 set format in the backup ui 2017-09-26 00:01:36 -07:00
Girish Ramakrishnan b0f85678d4 Implement downloadDir for flat-file format 2017-09-23 18:07:26 -07:00
Girish Ramakrishnan e43413e063 implement remove dir in storage backends 2017-09-23 12:34:51 -07:00
Girish Ramakrishnan e39a5c8872 preserve env in backuptask.js 2017-09-22 11:19:44 -07:00
Girish Ramakrishnan b61b7f80b5 Add DO spaces 2017-09-21 12:25:39 -07:00
Girish Ramakrishnan efa850614d Add a s3-v4-compat provider 2017-09-21 12:13:45 -07:00
Girish Ramakrishnan 97da8717ca Refactor backup strategy logic into backups.js 2017-09-20 14:09:55 -07:00
Girish Ramakrishnan cbddb79d15 Resolve the id in rotateAppBackup 2017-09-20 09:38:55 -07:00
Girish Ramakrishnan 26f33a8e9b Send resolved path to the storage APIs 2017-09-19 21:58:35 -07:00
Girish Ramakrishnan 952b1f6304 Make backuptask call back into backups.js 2017-09-19 20:27:49 -07:00
Girish Ramakrishnan 84649b9471 Bring back backuptask
This is required for various small reasons:

* dir iteration with a way to pass messagein back to the upload() easily
* can be killed independently of box code
* allows us to run sync (blocking) commands in the upload logic
2017-09-19 12:32:38 -07:00
Girish Ramakrishnan c351660a9a Implement backup rotation
Always upload to 'snapshot' dir and then rotate it. This will allow
us to keep pushing incrementally to 'snapshot' and do server side
rotations.
2017-09-18 21:17:34 -07:00
Girish Ramakrishnan f5d32a9178 copyBackup -> copy 2017-09-18 14:29:15 -07:00
Girish Ramakrishnan 7fc45b3215 Refactor out the backup snapshot logic 2017-09-18 12:43:11 -07:00
Girish Ramakrishnan be03dd0821 remove unused require 2017-09-18 10:38:26 -07:00
Girish Ramakrishnan 781f543e87 Rename API calls in the storage backend 2017-09-17 18:50:29 -07:00
Girish Ramakrishnan 6525a467a2 Rework backuptask into tar.js
This makes it easy to integrate another backup strategy
as the next step
2017-09-17 18:50:26 -07:00
Girish Ramakrishnan 6cddd61a24 Fix style 2017-09-17 18:50:23 -07:00
Girish Ramakrishnan 52095cb8ab add debugs for timing backup and restore 2017-09-12 15:37:35 -07:00
Girish Ramakrishnan 547a80f17b make shell.exec options non-optional 2017-09-09 19:54:31 -07:00
Girish Ramakrishnan e67ecae2d2 typo 2017-09-07 22:01:37 -07:00
Girish Ramakrishnan 75b3e7fc78 resolve symlinks correctly for deletion
part of #394
2017-09-07 21:57:08 -07:00
Girish Ramakrishnan 61f7c1af48 Remove unused error codes 2017-08-28 15:27:17 -07:00
Girish Ramakrishnan 78ec8e5c0c Add field to skip backup for an app
This skips the app from a backup when doing a full box backup and
simply reuses the previous backup.

The app can still be explicitly backed up using 'cloudron backup'
and explicitly restored using 'cloudron restore --backup'.

When restoring the box, it all depends on the app's last backup.

Fixes #311
2017-08-16 16:36:50 -07:00
Girish Ramakrishnan 4f1a596123 Add support for exoscale SOS 2017-07-25 15:23:04 -07:00
Girish Ramakrishnan 414e9bdf05 Do not use lastBackupId in cleanup logic
lastBackupId is only used as a "message" passing field for apptask restore.

Theoretically, this code somehow protects a race between the cleanup logic
and the restore apptask. this is unlikely to happen and adds unnecessary
complexity.
2017-06-01 14:47:57 -07:00
Girish Ramakrishnan c342e52e7d Record copyLastBackup in the backupdb 2017-06-01 14:08:55 -07:00
Girish Ramakrishnan 78aa9c66f7 Add a note why we do not cleanup more aggressively yet 2017-06-01 10:33:49 -07:00
Girish Ramakrishnan 986ec02ac6 Add debug on what backup is preserved 2017-06-01 09:38:39 -07:00
Girish Ramakrishnan f34d429052 kill the backup process if it runs for too long 2017-05-30 16:11:12 -07:00
Girish Ramakrishnan b04a417cfc Cleanup errored and creating backups
Fixes #330
2017-05-30 15:16:08 -07:00
Girish Ramakrishnan 77641f4b51 Add backupdb.getByState and backupdb.getByTypeAndState
part of #330
2017-05-30 14:30:06 -07:00
Girish Ramakrishnan 765d20c8be Add backup states to track unfinished backups
part of #330
2017-05-30 13:43:30 -07:00
Girish Ramakrishnan d2420de594 refactor backup cleanup logic 2017-05-30 13:43:30 -07:00
Girish Ramakrishnan 6fc69c05ca Add noop storage backend
This is sometimes useful when an update gets stuck because of some
bug in backup logic.

Note that you cannot restore from this backend because nothing is
saved.
2017-05-22 10:45:01 -07:00