30eccfb54b
Use BoxError instead of Error in all places
...
This moves everything other than the addon code and some 'done' logic
2019-12-04 11:02:54 -08:00
37c6a96a3a
s3: if etag is not present, flag as error
2019-11-20 12:53:36 -08:00
8c9ce30d29
Move BackupsError to BoxError
2019-10-22 21:03:47 -07:00
4fba216af9
scaleway: try to keep part numbers low
2019-09-30 20:42:37 -07:00
800e25a7a7
Fix crash because params was undefined
2019-05-10 13:07:29 -07:00
0835a6d690
caas: remove storage backend
2019-05-08 15:33:23 -07:00
3ded411c15
caas: migrate from fqdn to boxId in backup api
2019-04-26 10:55:50 -07:00
706c970314
caas: fixup routes
2019-04-26 10:19:37 -07:00
9a3603fea3
Add progress messages for multi-part copy
2019-03-26 11:58:32 -07:00
a88893b10a
remove/inject backups secret fields
...
follow same pattern as dns code
fixes #615
2019-02-09 20:44:05 -08:00
ee65089eb7
s3: make copying and uploading significantly faster
2019-01-14 13:47:07 -08:00
8efd496579
storage: add access denied function (unused)
2018-09-10 10:41:08 -07:00
ef5dc7311f
Do not encrypt empty path
2018-07-31 19:41:03 -07:00
a1070b7da3
storage: fix file not found message
2018-07-30 07:47:52 -07:00
99ad3e499d
s3: do not rely on entry.s3
2018-07-29 09:00:57 -07:00
0b10e2b332
s3: add listDir
...
Fixes #440
2018-07-28 09:05:44 -07:00
737541f707
storage: standardize the listDir API
2018-07-27 15:23:26 -07:00
df98847535
Remove backupDone from storage interface
...
This is only used by caas
2018-07-27 12:08:19 -07:00
2a18070016
do-spaces: Force retry of 4xx error codes when copying
2018-03-21 15:41:21 -07:00
e0ece06b26
s3: improved copy logging
2018-03-21 14:22:41 -07:00
6adf88a6e5
Make uploads work with very slow upload speeds
...
chunk uploads get a timeout of 2mins (derived from http.timeout).
On servers like kimsufi, uploads takes forever (100 MB/sec limit).
Currently, our upload code does not dynamically adapt itself to
changing the concurrency when network is slow.
2018-03-20 19:37:45 -07:00
ce33681c37
Dump etag info
2018-03-20 18:19:14 -07:00
565eed015f
Add better backup logs
2018-03-20 16:41:45 -07:00
98e19e6df5
fix upload errors causing double callback
2018-02-27 19:16:03 -08:00
b6fb49956f
s3: better debug output when copying
2018-02-22 12:41:18 -08:00
3c8c15db01
s3: use a constant backoff since it takes forever to fail otherwise
2018-02-22 12:30:44 -08:00
cea83889ec
s3: Fix issue where it takes forever to timeout if the backend is down
2018-02-22 12:19:23 -08:00
2ecb66afd7
s3: cleanup code
2018-02-22 12:16:01 -08:00
e6c07fc148
merge the done callback into the main code
2018-02-22 11:06:28 -08:00
1f30a4f3ea
Make s3.deleteObjects return error
2018-02-22 11:05:29 -08:00
0bfdaeb2fb
rename to chunkSize
2018-02-22 11:01:04 -08:00
e022dbf8a6
Revert "merge the done callback into the main code"
...
This reverts commit c39bec8cc1
2018-02-22 10:58:56 -08:00
6075a7a890
typo
2018-02-22 10:34:48 -08:00
28b864c346
sos: Copy in 96M chunks as recommended by exoscale
2018-02-22 10:31:56 -08:00
c39bec8cc1
merge the done callback into the main code
2018-02-21 20:17:58 -08:00
727a25f491
DO Spaces: multipart copy now works
2018-02-20 14:48:03 -08:00
77fe595970
lint
2018-02-04 00:15:41 -08:00
14529d313a
sos: CopySource requires fancier encoding rules
2018-02-04 00:12:28 -08:00
f607010396
sos: remove exoscale specific hacks which are obsolete now
2018-02-02 20:13:51 -08:00
568397ec19
caas: send ids in backupDone instead of filenames
2017-11-29 12:39:10 -08:00
693bc094cc
caas: make fqdn part of dns and s3 credentials
2017-11-28 22:44:40 -08:00
76312495fd
Add debug
2017-11-28 15:00:33 -08:00
f9dda85a38
Fix error code handling
2017-11-22 21:03:34 -08:00
f09c89e33f
Remove confusing batchSize logic from listDir
...
This also fixes a bug in removeDir in DO spaces
thanks to @syn for reporting
2017-10-29 19:04:10 -07:00
d127b25f0f
Only set the custom https agent for HTTPS minio
...
Otherwise, we get a Cannot set property ‘agent’ of undefined error
2017-10-26 18:38:45 -07:00
dd68c8f91f
Various backup fixes
2017-10-12 11:02:48 -07:00
1444bb038f
only upload needs to be retried
...
copy/delete are already retried in the sdk code
2017-10-11 00:08:41 -07:00
ac9e421ecf
improved backup progress and logging
2017-10-10 22:49:38 -07:00
78807782df
Various hacks for exoscale-sos
...
SOS does not like multipart uploads. They just fail randomly.
As a fix, we try to detect filesystem files and skip multipart uploads
for files < 5GB. For > 5GB, we do multipart upload anyways (mostly fails).
The box backup is switched to flat-file for exoscale for the reason
above.
2017-10-10 11:03:20 -07:00
9f97f48634
Add note on s3ForcePathStyle
2017-10-09 18:46:14 -07:00
Girish Ramakrishnan