Girish Ramakrishnan
f7c406bec9
s3: bucket name cannot contain _ or capitals or .
...
we can make it more elaborate, but not sure if it's needed
https://blogs.easydynamics.com/2016/10/24/aws-s3-bucket-name-validation-regex/
2020-05-27 17:01:42 -07:00
Johannes Zellner
9bdeff0a39
Always use constants.SECRET_PLACEHOLDER
2020-05-14 23:02:02 +02:00
Girish Ramakrishnan
101dc3a93c
s3: do not retry when testing config
2020-05-12 22:45:01 -07:00
Girish Ramakrishnan
4d050725b7
storage: done events must be called next tick
...
It seems that listDir() returns synchronously (!), not sure how.
This results in the done event getting called with an error but
the EE event handlers are not setup yet.
2020-02-11 11:48:49 -08:00
Girish Ramakrishnan
57597bd103
s3: bucket name cannot contain /
2020-02-11 11:19:47 -08:00
Girish Ramakrishnan
905bb92bad
s3: ensure BoxError return
2019-12-05 21:50:44 -08:00
Girish Ramakrishnan
85e17b570b
Use whilst instead of forever
...
this gets rid of the Error object
2019-12-04 11:17:44 -08:00
Girish Ramakrishnan
30eccfb54b
Use BoxError instead of Error in all places
...
This moves everything other than the addon code and some 'done' logic
2019-12-04 11:02:54 -08:00
Girish Ramakrishnan
37c6a96a3a
s3: if etag is not present, flag as error
2019-11-20 12:53:36 -08:00
Girish Ramakrishnan
8c9ce30d29
Move BackupsError to BoxError
2019-10-22 21:03:47 -07:00
Girish Ramakrishnan
4fba216af9
scaleway: try to keep part numbers low
2019-09-30 20:42:37 -07:00
Girish Ramakrishnan
800e25a7a7
Fix crash because params was undefined
2019-05-10 13:07:29 -07:00
Girish Ramakrishnan
0835a6d690
caas: remove storage backend
2019-05-08 15:33:23 -07:00
Girish Ramakrishnan
3ded411c15
caas: migrate from fqdn to boxId in backup api
2019-04-26 10:55:50 -07:00
Girish Ramakrishnan
706c970314
caas: fixup routes
2019-04-26 10:19:37 -07:00
Girish Ramakrishnan
9a3603fea3
Add progress messages for multi-part copy
2019-03-26 11:58:32 -07:00
Girish Ramakrishnan
a88893b10a
remove/inject backups secret fields
...
follow same pattern as dns code
fixes #615
2019-02-09 20:44:05 -08:00
Girish Ramakrishnan
ee65089eb7
s3: make copying and uploading significantly faster
2019-01-14 13:47:07 -08:00
Girish Ramakrishnan
8efd496579
storage: add access denied function (unused)
2018-09-10 10:41:08 -07:00
Girish Ramakrishnan
ef5dc7311f
Do not encrypt empty path
2018-07-31 19:41:03 -07:00
Girish Ramakrishnan
a1070b7da3
storage: fix file not found message
2018-07-30 07:47:52 -07:00
Girish Ramakrishnan
99ad3e499d
s3: do not rely on entry.s3
2018-07-29 09:00:57 -07:00
Girish Ramakrishnan
0b10e2b332
s3: add listDir
...
Fixes #440
2018-07-28 09:05:44 -07:00
Girish Ramakrishnan
737541f707
storage: standardize the listDir API
2018-07-27 15:23:26 -07:00
Girish Ramakrishnan
df98847535
Remove backupDone from storage interface
...
This is only used by caas
2018-07-27 12:08:19 -07:00
Girish Ramakrishnan
2a18070016
do-spaces: Force retry of 4xx error codes when copying
2018-03-21 15:41:21 -07:00
Girish Ramakrishnan
e0ece06b26
s3: improved copy logging
2018-03-21 14:22:41 -07:00
Girish Ramakrishnan
6adf88a6e5
Make uploads work with very slow upload speeds
...
chunk uploads get a timeout of 2mins (derived from http.timeout).
On servers like kimsufi, uploads takes forever (100 MB/sec limit).
Currently, our upload code does not dynamically adapt itself to
changing the concurrency when network is slow.
2018-03-20 19:37:45 -07:00
Girish Ramakrishnan
ce33681c37
Dump etag info
2018-03-20 18:19:14 -07:00
Girish Ramakrishnan
565eed015f
Add better backup logs
2018-03-20 16:41:45 -07:00
Girish Ramakrishnan
98e19e6df5
fix upload errors causing double callback
2018-02-27 19:16:03 -08:00
Girish Ramakrishnan
b6fb49956f
s3: better debug output when copying
2018-02-22 12:41:18 -08:00
Girish Ramakrishnan
3c8c15db01
s3: use a constant backoff since it takes forever to fail otherwise
2018-02-22 12:30:44 -08:00
Girish Ramakrishnan
cea83889ec
s3: Fix issue where it takes forever to timeout if the backend is down
2018-02-22 12:19:23 -08:00
Girish Ramakrishnan
2ecb66afd7
s3: cleanup code
2018-02-22 12:16:01 -08:00
Girish Ramakrishnan
e6c07fc148
merge the done callback into the main code
2018-02-22 11:06:28 -08:00
Girish Ramakrishnan
1f30a4f3ea
Make s3.deleteObjects return error
2018-02-22 11:05:29 -08:00
Girish Ramakrishnan
0bfdaeb2fb
rename to chunkSize
2018-02-22 11:01:04 -08:00
Girish Ramakrishnan
e022dbf8a6
Revert "merge the done callback into the main code"
...
This reverts commit c39bec8cc1
2018-02-22 10:58:56 -08:00
Girish Ramakrishnan
6075a7a890
typo
2018-02-22 10:34:48 -08:00
Girish Ramakrishnan
28b864c346
sos: Copy in 96M chunks as recommended by exoscale
2018-02-22 10:31:56 -08:00
Girish Ramakrishnan
c39bec8cc1
merge the done callback into the main code
2018-02-21 20:17:58 -08:00
Girish Ramakrishnan
727a25f491
DO Spaces: multipart copy now works
2018-02-20 14:48:03 -08:00
Girish Ramakrishnan
77fe595970
lint
2018-02-04 00:15:41 -08:00
Girish Ramakrishnan
14529d313a
sos: CopySource requires fancier encoding rules
2018-02-04 00:12:28 -08:00
Girish Ramakrishnan
f607010396
sos: remove exoscale specific hacks which are obsolete now
2018-02-02 20:13:51 -08:00
Girish Ramakrishnan
568397ec19
caas: send ids in backupDone instead of filenames
2017-11-29 12:39:10 -08:00
Girish Ramakrishnan
693bc094cc
caas: make fqdn part of dns and s3 credentials
2017-11-28 22:44:40 -08:00
Girish Ramakrishnan
76312495fd
Add debug
2017-11-28 15:00:33 -08:00
Girish Ramakrishnan
f9dda85a38
Fix error code handling
2017-11-22 21:03:34 -08:00
