jjkiers/cloudron-box
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
9,751 Commits 33 Branches 583 Tags
a20bcbd570d6b8a7dd1b44c121c13ec4384ba97a
Commit Graph

289 Commits

Author SHA1 Message Date
Girish Ramakrishnan
f55300eba5 reduce DO spaces copy part size 2020-12-15 14:37:18 -08:00
Girish Ramakrishnan
a68ddcbbc4 Fix progress message 2020-12-14 19:58:44 -08:00
Girish Ramakrishnan
0723b7d672 reduce copy concurrency to keep most providers happy 2020-12-14 17:26:44 -08:00
Girish Ramakrishnan
ca17afc734 volumes: better hostPath validation 2020-12-03 23:05:06 -08:00
Girish Ramakrishnan
41053d6857 validate backup folder and prefix 2020-12-01 12:46:02 -08:00
Girish Ramakrishnan
3d5599cdd9 b2: reduce copy part size 
'Error copying snapshot/app_8b22dc8f-3e15-4314-8108-bcf1908a24df.tar.gz.enc (482405284 bytes): InternalError InternalError: too busy to complete copy - please try again' }
 2020-11-29 11:16:05 -08:00
Girish Ramakrishnan
bedcd6fccf Disable the timeout altogether for chunk to upload 2020-11-06 14:47:14 -08:00
Girish Ramakrishnan
df8a71cd8b Each chunk can take up to 30 mins to upload 2020-11-06 00:05:53 -08:00
Girish Ramakrishnan
e68db4ce57 Aim for 60% used space 2020-11-02 23:42:53 -08:00
Girish Ramakrishnan
1da2450b10 gcs: use copy concurrency 2020-09-28 22:03:08 -07:00
Girish Ramakrishnan
166c06c628 log the partSize 2020-09-10 00:09:54 -07:00
Girish Ramakrishnan
fc08f9823e s3: copy parts in parallel 2020-09-03 14:31:56 -07:00
Girish Ramakrishnan
e25ccc5e9a Double the timeout for upload now that chunks can have custom sizes 2020-08-20 16:50:58 -07:00
Girish Ramakrishnan
0e00492f54 backups: make part size configurable 2020-08-19 14:39:20 -07:00
Girish Ramakrishnan
df6e3eb1e6 Add deleteConcurrency setting 2020-08-11 09:14:09 -07:00
Girish Ramakrishnan
91288c96b1 s3: set queue size to 3 
fixes #691
 2020-08-07 00:28:00 -07:00
Johannes Zellner
493f1505f0 Check also for mountpoint on filesystem with external disk 2020-07-02 19:08:27 +02:00
Johannes Zellner
a96da20536 TODO is done for filesystem backend moutnpoint check 2020-06-26 17:57:26 +02:00
Johannes Zellner
60f1b2356a Also make nfs storage provider same as cifs and sshfs 2020-06-22 15:51:05 +02:00
Girish Ramakrishnan
c855d75f35 remove mkdirp use 
node 10.12 has { recursive: true }
 2020-06-11 08:27:48 -07:00
Johannes Zellner
89494ced41 Check for sshfs and cifs backup backends, if they are mounted 2020-06-08 17:46:52 +02:00
Johannes Zellner
ef764c2393 Merge sshfs.js into filesystem.js 2020-06-08 17:08:26 +02:00
Johannes Zellner
8624e2260d add storage api to make preflight checks 
Currently there is only disk space checking but sshfs and cifs need
mount point checking as well
 2020-06-08 16:25:05 +02:00
Johannes Zellner
3557fcd129 Add sshfs quirks to shared code in filesytstem.js 2020-06-05 13:45:25 +02:00
Johannes Zellner
330b4a613c Retrieve the backupPath from the storage provider itself 2020-06-05 13:27:18 +02:00
Johannes Zellner
7ba3412aae Add some sshfs config tests 2020-06-05 12:43:09 +02:00
Johannes Zellner
6f60495d4d Initial version of sshfs storage backend 2020-06-05 11:39:51 +02:00
Girish Ramakrishnan
4a0a934a76 start using vhost style for accessing s3 style storage 
if bucket name has a '.', accept self-signed

fixes #680
 2020-05-27 17:50:37 -07:00
Girish Ramakrishnan
f7c406bec9 s3: bucket name cannot contain _ or capitals or . 
we can make it more elaborate, but not sure if it's needed

https://blogs.easydynamics.com/2016/10/24/aws-s3-bucket-name-validation-regex/
 2020-05-27 17:01:42 -07:00
Girish Ramakrishnan
f84622efa1 fs: add create/unlink tests 2020-05-26 15:31:41 -07:00
Johannes Zellner
9bdeff0a39 Always use constants.SECRET_PLACEHOLDER 2020-05-14 23:02:02 +02:00
Girish Ramakrishnan
101dc3a93c s3: do not retry when testing config 2020-05-12 22:45:01 -07:00
Girish Ramakrishnan
4d050725b7 storage: done events must be called next tick 
It seems that listDir() returns synchronously (!), not sure how.
This results in the done event getting called with an error but
the EE event handlers are not setup yet.
 2020-02-11 11:48:49 -08:00
Girish Ramakrishnan
57597bd103 s3: bucket name cannot contain / 2020-02-11 11:19:47 -08:00
Girish Ramakrishnan
905bb92bad s3: ensure BoxError return 2019-12-05 21:50:44 -08:00
Girish Ramakrishnan
85e17b570b Use whilst instead of forever 
this gets rid of the Error object
 2019-12-04 11:17:44 -08:00
Girish Ramakrishnan
30eccfb54b Use BoxError instead of Error in all places 
This moves everything other than the addon code and some 'done' logic
 2019-12-04 11:02:54 -08:00
Girish Ramakrishnan
d0a3d00492 Use NOT_IMPLEMENTED error code 2019-12-04 10:22:22 -08:00
Girish Ramakrishnan
37c6a96a3a s3: if etag is not present, flag as error 2019-11-20 12:53:36 -08:00
Girish Ramakrishnan
8c9ce30d29 Move BackupsError to BoxError 2019-10-22 21:03:47 -07:00
Girish Ramakrishnan
29ab3e91b3 gcs: remove concurrency logic 
this is more complicated than necessary
 2019-10-18 18:54:25 -07:00
Girish Ramakrishnan
4fba216af9 scaleway: try to keep part numbers low 2019-09-30 20:42:37 -07:00
Girish Ramakrishnan
05d3f8a667 gcs: fix crash 2019-05-12 18:05:48 -07:00
Girish Ramakrishnan
800e25a7a7 Fix crash because params was undefined 2019-05-10 13:07:29 -07:00
Girish Ramakrishnan
0835a6d690 caas: remove storage backend 2019-05-08 15:33:23 -07:00
Girish Ramakrishnan
3ded411c15 caas: migrate from fqdn to boxId in backup api 2019-04-26 10:55:50 -07:00
Girish Ramakrishnan
706c970314 caas: fixup routes 2019-04-26 10:19:37 -07:00
Girish Ramakrishnan
d1582a435d readdirp args have changed 2019-04-24 10:47:04 -07:00
Girish Ramakrishnan
1119a62576 fs: emit progress messages for copy and removeDir 2019-04-03 11:54:46 -07:00
Girish Ramakrishnan
9a3603fea3 Add progress messages for multi-part copy 2019-03-26 11:58:32 -07:00