72daaa9ff0
ionos: add profitbricks object storage
2021-02-04 10:14:35 -08:00
6bd87485c6
rename addons.js to services.js
...
services is the named container (services view)
addons is more like a heroku concept
2021-01-21 11:31:35 -08:00
78752fde7a
app: add export route
...
Currently, the export route only creates the snapshot (the other side
of in-place import). In the future, the export route can export to a
custom backup config (like import).
2020-12-06 19:57:26 -08:00
a113ece22b
Still have to preserveEnv for the env vars to make it across sudo
2020-11-05 16:13:42 -08:00
02eb362f37
Set the heap size with large backup memory limits
...
I had to also give the server some more swap for the backup to succeed
2020-11-05 16:06:12 -08:00
d9f88985fe
rsync: create destination file only when source is available
...
if the source disappears, the upload() in the backend creates the file
as 'root'. the chown is never done because the read stream errored.
As a result of permissions, cp fails to hardlink because the hardlink
is run as yellowtent user.
fixes #741
2020-09-30 20:12:17 -07:00
933918ea27
Fix docs url
2020-09-15 14:46:22 -07:00
e791084793
bump timeout to 24 hours
2020-09-02 18:19:25 -07:00
9b67ab9713
typo
2020-08-31 08:58:38 -07:00
dd5dfd98b7
ensure box update backups are also preserved for 3 weeks
2020-08-30 21:38:13 -07:00
9cdf5dd0f3
backups: time the rotation and total as well
2020-08-11 10:28:11 -07:00
05026771e1
add memoryLimit, copyConcurrency, downloadConcurrency to backup config
2020-08-10 22:12:01 -07:00
99f989c384
run apptask and backup task with a nice
...
A child process inherits whatever nice value is held by the parent at the time that it is forked
2020-08-06 16:46:39 -07:00
2112c7d096
sudo: remove the nice support
2020-08-06 16:44:35 -07:00
d564003c87
backup cleaner: referenced backups must be counted as part of period
...
otherwise, we end up in a state where box backups keeps referencing
app backups and app backup cleanup is only performed on the remaining
app backups.
2020-08-03 21:22:27 -07:00
d2882433a5
run backup uploader with a nice of 15
...
the gzip takes a lot of cpu processing and hogs the CPU. With a nice
level, we give other things higher priority.
An alternate idea that was explored was to use cpulimit. This is to
send SIGSTOP and SIGCONT periodically but this will not make use of the
CPU if it's idle (unlike nice).
Another idea is to use cgroups, but it's not clear how to use it with
the dynamic setup we have.
part of #691
2020-07-31 18:23:36 -07:00
a94b175805
Add timing information for backups
2020-07-31 12:59:15 -07:00
f511a610b5
backups: take a pattern instead of interval secs
...
part of #699
2020-07-28 21:54:56 -07:00
05dcbee7e3
backups: add b2 provider
...
part of #508
2020-07-13 14:52:35 -07:00
60f1b2356a
Also make nfs storage provider same as cifs and sshfs
2020-06-22 15:51:05 +02:00
129cbb5beb
backups: fix cleanup
...
The various changes are:
* Latest backup is always kept for box and app backups
* If the latest backup is part of the policy, it is not counted twice
* Latest backup comes into action only when all backups are outside the retention policy
* For uninstalled apps, latest backup is not preserved
* This way the latest backup of apps that are not referenced in box backup is preserved.
(for example, for stopped apps)
fixes #692
2020-06-14 22:06:00 -07:00
f6cb1a0863
backups: query using identifier instead of type
...
this allows us to move the enums into backups.js instead of backupdb.js
2020-06-14 12:27:41 -07:00
4f964101a0
add identifier to backups table
2020-06-14 11:39:44 -07:00
f6dcba025f
auditSource is not used in the worker
2020-06-14 09:09:41 -07:00
c855d75f35
remove mkdirp use
...
node 10.12 has { recursive: true }
2020-06-11 08:27:48 -07:00
8f5cdcf439
backups: some logs for debugging
2020-06-10 23:00:23 -07:00
ef764c2393
Merge sshfs.js into filesystem.js
2020-06-08 17:08:26 +02:00
8624e2260d
add storage api to make preflight checks
...
Currently there is only disk space checking but sshfs and cifs need
mount point checking as well
2020-06-08 16:25:05 +02:00
330b4a613c
Retrieve the backupPath from the storage provider itself
2020-06-05 13:27:18 +02:00
6f60495d4d
Initial version of sshfs storage backend
2020-06-05 11:39:51 +02:00
4fa181b346
re-use the latest backup id for non-backupable apps
...
for stopped apps, as an example
2020-05-28 14:16:38 -07:00
ea47fb7305
Properly check for backup policy in testConfig()
2020-05-28 20:44:44 +02:00
e1a718c78f
remove redundant call to canBackupApp
2020-05-27 22:48:48 -07:00
fca3f606d2
Do not backup stopped apps
2020-05-27 21:04:01 -07:00
04a1aa38b4
Add CIFS as storage provider
...
part of #686
2020-05-26 15:31:45 -07:00
ce927a2247
Set dmode in tar extract
2020-05-24 20:08:17 -07:00
bdcc5c0629
Mbps -> MBps
...
Fixes #682
2020-05-23 13:31:23 -07:00
90c24cf356
add cleanup policy test
2020-05-21 14:30:21 -07:00
54abada561
backups: add progressCallback to cleanup funcs
2020-05-21 13:46:16 -07:00
795e3c57da
Add a header for encrypted backup files
...
this is required to identify old backups and new backups for decryption
2020-05-20 22:44:26 -07:00
99475c51e8
fix encryption of 0-length files
2020-05-15 16:05:12 -07:00
cc9b4e26b5
use done event to signal write success (just like in extract)
2020-05-15 15:24:12 -07:00
32f232d3c0
destroy input stream on error
2020-05-15 15:21:24 -07:00
235047ad0b
bind to source stream error event immediately
...
download() is async and the source stream error is missed
2020-05-15 14:54:05 -07:00
228f75de0b
better error messages
2020-05-15 14:35:19 -07:00
af9e3e38ce
apply backup retention policy
...
part of #441
2020-05-14 21:31:24 -07:00
d992702b87
rename to keepWithinSecs
...
part of #441
2020-05-14 16:45:28 -07:00
6a9fe1128f
move retentionSecs inside retentionPolicy
...
part of #441
2020-05-14 16:33:29 -07:00
573da29a4d
Once upon a time where settings worked
2020-05-14 23:35:03 +02:00
9bdeff0a39
Always use constants.SECRET_PLACEHOLDER
2020-05-14 23:02:02 +02:00
Girish Ramakrishnan