Commit Graph

304 Commits

Author SHA1 Message Date
Girish Ramakrishnan 98dc160886 remove the progress and watchdog timers
not needed since we will track these processes now and can be killed
by the user.
2018-11-17 19:38:02 -08:00
Girish Ramakrishnan a869c88b43 Remove setBackupProgress
the upload task can just output to stdout. for progress status, we
can just use a tailstream on the ui side
2018-11-17 19:38:02 -08:00
Girish Ramakrishnan 0b86070fe9 run backuptask as separate process 2018-11-17 19:38:02 -08:00
Girish Ramakrishnan d7a11ef394 runBackupTask -> runBackupUpload 2018-11-16 09:21:07 -08:00
Girish Ramakrishnan 6abd48d480 rename backuptask to backupupload 2018-11-15 12:00:51 -08:00
Girish Ramakrishnan a504759b95 more debugs and comments 2018-09-26 12:39:33 -07:00
Girish Ramakrishnan ec9c96da6f Fix comments 2018-09-04 10:48:54 -07:00
Girish Ramakrishnan 8bd6c9933f Add interval secs
Part of #568
2018-08-13 22:35:38 -07:00
Girish Ramakrishnan 1ed4710c68 Better error message 2018-08-05 23:38:49 -07:00
Girish Ramakrishnan ef5dc7311f Do not encrypt empty path 2018-07-31 19:41:03 -07:00
Girish Ramakrishnan 498642b670 Fix debugs 2018-07-31 19:32:56 -07:00
Girish Ramakrishnan a1070b7da3 storage: fix file not found message 2018-07-30 07:47:52 -07:00
Girish Ramakrishnan 3067d87ca9 handle decryption errors 2018-07-29 21:01:20 -07:00
Girish Ramakrishnan f546d53ca2 gcs: add listDir
part of #440
2018-07-28 03:51:32 -07:00
Girish Ramakrishnan a5daad2e1a rsync: encrypted download (fs backend)
Part of #440
2018-07-27 16:06:13 -07:00
Girish Ramakrishnan b3c8767d79 rsync: encrypted upload (fs backend)
Part of #440
2018-07-27 15:34:48 -07:00
Girish Ramakrishnan f97f528f05 backup: Implement downloadDir using listDir 2018-07-27 15:24:35 -07:00
Girish Ramakrishnan df98847535 Remove backupDone from storage interface
This is only used by caas
2018-07-27 12:08:19 -07:00
Girish Ramakrishnan 6adf88a6e5 Make uploads work with very slow upload speeds
chunk uploads get a timeout of 2mins (derived from http.timeout).
On servers like kimsufi, uploads takes forever (100 MB/sec limit).
Currently, our upload code does not dynamically adapt itself to
changing the concurrency when network is slow.
2018-03-20 19:37:45 -07:00
Girish Ramakrishnan 7699f6721d Add hack to figure out the position in the queue
this helps us track the progress a bit in the logs
2018-03-20 19:37:35 -07:00
Girish Ramakrishnan 565eed015f Add better backup logs 2018-03-20 16:41:45 -07:00
Girish Ramakrishnan 98e19e6df5 fix upload errors causing double callback 2018-02-27 19:16:03 -08:00
Girish Ramakrishnan f5d426fd69 debug out the progress message 2018-02-22 11:11:36 -08:00
Girish Ramakrishnan e022dbf8a6 Revert "merge the done callback into the main code"
This reverts commit c39bec8cc1.

This was committed with extra stuff by mistake
2018-02-22 10:58:56 -08:00
Girish Ramakrishnan c39bec8cc1 merge the done callback into the main code 2018-02-21 20:17:58 -08:00
Johannes Zellner b6384d5025 Remove intrinsicFqdn 2018-02-08 15:07:49 +01:00
Johannes Zellner b1939e73f4 Remove all occurances of altDomain in the code
Tests are pending
2018-02-08 09:00:31 +01:00
Girish Ramakrishnan 56f4cbe44a Fix double callback invocation in cleanupBackup 2018-01-15 20:08:55 -08:00
Girish Ramakrishnan efc0a3b68d Remove usage of config.appFqdn() 2018-01-10 13:58:05 -08:00
Girish Ramakrishnan 61b56d4679 gcs: keyFilename is not used
also, some linter fixes
2017-12-16 21:36:29 +05:30
Aleksandr Bogdanov 5a2de0bcbb Merge remote-tracking branch 'origin/master' into feature/gcs
# Conflicts:
#	webadmin/src/views/certs.js
#	webadmin/src/views/settings.js
2017-12-06 22:47:26 +01:00
Girish Ramakrishnan 998f736e6f Add database.exportToFile 2017-11-24 15:29:56 -08:00
Girish Ramakrishnan 0f191324fa Add backups.restore to import from box backup
Part of #439
2017-11-22 23:08:59 -08:00
Girish Ramakrishnan dc1d10837b split read/parse of fsmetadata 2017-11-22 23:07:52 -08:00
Girish Ramakrishnan 136ee363a8 Make backups.download take backupConfig 2017-11-22 10:38:04 -08:00
Johannes Zellner e43e904622 Refactor all app.location usages to config.appFqdn(app) 2017-11-20 20:01:50 +01:00
Johannes Zellner 143ba831f4 Make appFqdn() multidomain aware 2017-11-20 19:59:40 +01:00
Girish Ramakrishnan 998bc36673 remove manifest arg to backupApp 2017-11-19 17:58:04 -08:00
Girish Ramakrishnan 360fa058ea store format information for restoring
fixes #483
2017-11-16 15:01:27 -08:00
Girish Ramakrishnan 92acb2954f Rename restoreConfig to manifest in backup table
Only the manifest needs to be preserved in the backup table
2017-11-16 11:25:40 -08:00
Aleksandr Bogdanov 994f771d4d Merge remote-tracking branch 'origin/master' into feature/gcs 2017-11-14 20:16:12 +01:00
Girish Ramakrishnan a7296a0339 Rename filename to backupId in backup eventlog 2017-11-02 18:17:08 -07:00
Aleksandr Bogdanov 051d04890b Adding Google Cloud Storage support 2017-10-28 15:14:23 +02:00
Girish Ramakrishnan 89fc8efc67 Save as empty array if find output is empty 2017-10-16 10:54:48 -07:00
Girish Ramakrishnan 61e2878b08 save/restore exec bit in files
this covers the case where user might stash some executable files
that are used by plugins.
2017-10-12 16:18:11 -07:00
Girish Ramakrishnan d97034bfb2 Follow backup format for box backups as well 2017-10-12 11:02:52 -07:00
Girish Ramakrishnan dd68c8f91f Various backup fixes 2017-10-12 11:02:48 -07:00
Girish Ramakrishnan 28ce5f41e3 handle errors in log stream 2017-10-11 12:55:56 -07:00
Girish Ramakrishnan db8c5a116f Typo 2017-10-11 10:30:03 -07:00
Girish Ramakrishnan 1444bb038f only upload needs to be retried
copy/delete are already retried in the sdk code
2017-10-11 00:08:41 -07:00