jjkiers/cloudron-box
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
9,436 Commits 33 Branches 583 Tags
dd44edde0a2fb2e57401e0cbbe871a1fa891daf0
Commit Graph

405 Commits

Author SHA1 Message Date
Girish Ramakrishnan
9cdf5dd0f3 backups: time the rotation and total as well 2020-08-11 10:28:11 -07:00
Girish Ramakrishnan
05026771e1 add memoryLimit, copyConcurrency, downloadConcurrency to backup config 2020-08-10 22:12:01 -07:00
Girish Ramakrishnan
99f989c384 run apptask and backup task with a nice 
A child process inherits whatever nice value is held by the parent at the time that it is forked
 2020-08-06 16:46:39 -07:00
Girish Ramakrishnan
2112c7d096 sudo: remove the nice support 2020-08-06 16:44:35 -07:00
Girish Ramakrishnan
d564003c87 backup cleaner: referenced backups must be counted as part of period 
otherwise, we end up in a state where box backups keeps referencing
app backups and app backup cleanup is only performed on the remaining
app backups.
 2020-08-03 21:22:27 -07:00
Girish Ramakrishnan
d2882433a5 run backup uploader with a nice of 15 
the gzip takes a lot of cpu processing and hogs the CPU. With a nice
level, we give other things higher priority.

An alternate idea that was explored was to use cpulimit. This is to
send SIGSTOP and SIGCONT periodically but this will not make use of the
CPU if it's idle (unlike nice).

Another idea is to use cgroups, but it's not clear how to use it with
the dynamic setup we have.

part of #691
 2020-07-31 18:23:36 -07:00
Girish Ramakrishnan
a94b175805 Add timing information for backups 2020-07-31 12:59:15 -07:00
Girish Ramakrishnan
f511a610b5 backups: take a pattern instead of interval secs 
part of #699
 2020-07-28 21:54:56 -07:00
Girish Ramakrishnan
05dcbee7e3 backups: add b2 provider 
part of #508
 2020-07-13 14:52:35 -07:00
Johannes Zellner
60f1b2356a Also make nfs storage provider same as cifs and sshfs 2020-06-22 15:51:05 +02:00
Girish Ramakrishnan
129cbb5beb backups: fix cleanup 
The various changes are:
* Latest backup is always kept for box and app backups
* If the latest backup is part of the policy, it is not counted twice
* Latest backup comes into action only when all backups are outside the retention policy
* For uninstalled apps, latest backup is not preserved
* This way the latest backup of apps that are not referenced in box backup is preserved.
  (for example, for stopped apps)

fixes #692
 2020-06-14 22:06:00 -07:00
Girish Ramakrishnan
f6cb1a0863 backups: query using identifier instead of type 
this allows us to move the enums into backups.js instead of backupdb.js
 2020-06-14 12:27:41 -07:00
Girish Ramakrishnan
4f964101a0 add identifier to backups table 2020-06-14 11:39:44 -07:00
Girish Ramakrishnan
f6dcba025f auditSource is not used in the worker 2020-06-14 09:09:41 -07:00
Girish Ramakrishnan
c855d75f35 remove mkdirp use 
node 10.12 has { recursive: true }
 2020-06-11 08:27:48 -07:00
Girish Ramakrishnan
8f5cdcf439 backups: some logs for debugging 2020-06-10 23:00:23 -07:00
Johannes Zellner
ef764c2393 Merge sshfs.js into filesystem.js 2020-06-08 17:08:26 +02:00
Johannes Zellner
8624e2260d add storage api to make preflight checks 
Currently there is only disk space checking but sshfs and cifs need
mount point checking as well
 2020-06-08 16:25:05 +02:00
Johannes Zellner
330b4a613c Retrieve the backupPath from the storage provider itself 2020-06-05 13:27:18 +02:00
Johannes Zellner
6f60495d4d Initial version of sshfs storage backend 2020-06-05 11:39:51 +02:00
Girish Ramakrishnan
4fa181b346 re-use the latest backup id for non-backupable apps 
for stopped apps, as an example
 2020-05-28 14:16:38 -07:00
Johannes Zellner
ea47fb7305 Properly check for backup policy in testConfig() 2020-05-28 20:44:44 +02:00
Girish Ramakrishnan
e1a718c78f remove redundant call to canBackupApp 2020-05-27 22:48:48 -07:00
Girish Ramakrishnan
fca3f606d2 Do not backup stopped apps 2020-05-27 21:04:01 -07:00
Girish Ramakrishnan
04a1aa38b4 Add CIFS as storage provider 
part of #686
 2020-05-26 15:31:45 -07:00
Girish Ramakrishnan
ce927a2247 Set dmode in tar extract 2020-05-24 20:08:17 -07:00
Girish Ramakrishnan
bdcc5c0629 Mbps -> MBps 
Fixes #682
 2020-05-23 13:31:23 -07:00
Girish Ramakrishnan
90c24cf356 add cleanup policy test 2020-05-21 14:30:21 -07:00
Girish Ramakrishnan
54abada561 backups: add progressCallback to cleanup funcs 2020-05-21 13:46:16 -07:00
Girish Ramakrishnan
795e3c57da Add a header for encrypted backup files 
this is required to identify old backups and new backups for decryption
 2020-05-20 22:44:26 -07:00
Girish Ramakrishnan
99475c51e8 fix encryption of 0-length files 2020-05-15 16:05:12 -07:00
Girish Ramakrishnan
cc9b4e26b5 use done event to signal write success (just like in extract) 2020-05-15 15:24:12 -07:00
Girish Ramakrishnan
32f232d3c0 destroy input stream on error 2020-05-15 15:21:24 -07:00
Girish Ramakrishnan
235047ad0b bind to source stream error event immediately 
download() is async and the source stream error is missed
 2020-05-15 14:54:05 -07:00
Girish Ramakrishnan
228f75de0b better error messages 2020-05-15 14:35:19 -07:00
Girish Ramakrishnan
af9e3e38ce apply backup retention policy 
part of #441
 2020-05-14 21:31:24 -07:00
Girish Ramakrishnan
d992702b87 rename to keepWithinSecs 
part of #441
 2020-05-14 16:45:28 -07:00
Girish Ramakrishnan
6a9fe1128f move retentionSecs inside retentionPolicy 
part of #441
 2020-05-14 16:33:29 -07:00
Johannes Zellner
573da29a4d Once upon a time where settings worked 2020-05-14 23:35:03 +02:00
Johannes Zellner
9bdeff0a39 Always use constants.SECRET_PLACEHOLDER 2020-05-14 23:02:02 +02:00
Johannes Zellner
f52c16b209 Ensure encryption property on backup config always exists 2020-05-14 20:22:10 +02:00
Girish Ramakrishnan
4faf880aa4 Fix crash with unencrypted backups 2020-05-14 11:18:41 -07:00
Girish Ramakrishnan
f417a49b34 Add encryptionVersion to backups 
this will identify the old style backups and warn user that a restore
doesn't work anymore
 2020-05-13 22:37:02 -07:00
Girish Ramakrishnan
66fd713d12 rename version to packageVersion 2020-05-13 21:55:50 -07:00
Girish Ramakrishnan
f5bb76333b do hmac validation on filename iv as well 
also, pass encryption object instead of config
 2020-05-13 10:11:07 -07:00
Girish Ramakrishnan
2c52668a74 remove format validation in provider config 2020-05-12 22:00:01 -07:00
Girish Ramakrishnan
37dfa41e01 Add hmac to the file data 
https://stackoverflow.com/questions/10279403/confused-how-to-use-aes-and-hmac
https://en.wikipedia.org/wiki/Padding_oracle_attack

part of #579
 2020-05-12 19:59:06 -07:00
Girish Ramakrishnan
ea8a3d798e create encryption keys from password during app import & restore 2020-05-12 15:53:18 -07:00
Girish Ramakrishnan
1df94fd84d backups: generate keys from password 
this also removes storage of password from db

part of #579
 2020-05-12 15:14:51 -07:00
Girish Ramakrishnan
21073c627e rename backup key to password 
Fixes #579
 2020-05-12 10:55:10 -07:00