cb6d531300
storage: automatically abort old multipart uploads in s3
2025-02-13 11:09:15 +01:00
a138425298
storage: start migration of s3 api
2025-02-12 23:04:37 +01:00
9888aa8c08
storage: rework listDir api to be a generator (like) function
2025-02-12 19:34:57 +01:00
ea72cef7f9
storage: remove getProviderStatus
2024-09-09 17:36:51 +02:00
ad4e0ba9aa
tests: fix storage test
2024-07-08 22:29:45 +02:00
1dc6b40a68
tgz: extract using tar-stream directly
...
we used have a fork of tar-fs. using tar-stream directly gives us
more control
2024-07-08 13:06:56 +02:00
aa8c23c8b3
rework backup root
...
notes:
* backup root cannot come from backend. for dynamic mounts backend cannot know where it is mounted
* backupConfig is 3 parts - format / mount / password . there is also this rootPath (which should not be in db)
* password should be stored separately in settings at some point
* format has to be passed along everywhere because we allow restore from same backupConfig but different format. we do this by saving the format in the backups table
fixes #819
2023-08-15 22:51:45 +05:30
e6b85c2df7
remount does not need a backend hook
2023-08-15 08:55:38 +05:30
3d5c21d9ca
backups: encrypted backups must have .enc extension
2023-07-24 22:25:06 +05:30
050a82039a
getBackupProviderStatus -> getProviderStatus
2023-07-15 11:00:45 +05:30
c86059e070
backups: move limits into a sub object
...
fixes #817
2023-07-13 12:17:57 +05:30
c4f4f3e914
logs: use %o to format error
...
otherwise, they are printed as multi-line and this messes up tail+date formatting
2023-04-16 10:49:59 +02:00
c3793da5bb
split checkPrecondition so it can be used in cleaner as well
2022-10-02 17:41:21 +02:00
074e9cfd93
rename getRootPath to getBackupRootPath
2022-10-02 16:26:27 +02:00
9d17c6606b
rename to checkBackupPreconditions
...
since this is called only by the backup logic
2022-10-02 16:20:14 +02:00
ea01586b52
storage: make copy async
2022-04-30 16:24:45 -07:00
f7ed044a40
fix storage test
2022-04-15 07:49:38 -05:00
685bda35b9
storage: make remove and removeDir async
2022-04-14 16:16:20 -05:00
d54c03f0a0
storage: make exists async
2022-04-14 12:24:34 -05:00
11f7be2065
storage: verifyConfig is now async
2022-04-14 12:24:30 -05:00
a39e0ab934
storage: make remount async
2022-04-14 09:57:31 -05:00
b51082f7e4
storage: checkPreconditions is now async
2022-04-14 07:40:19 -05:00
452a4d9a75
backups: add remotePath
...
the main motivation is that id can be used in REST API routes. previously,
the id was a path and this had a "/" in it. This made /api/v1/backups/:backupId
not work.
2022-04-04 20:40:40 -07:00
54934c41a7
storage: rename getBackupPath to getBasePath
2022-04-04 14:08:24 -07:00
d1a7172895
Add remount route for mountlike backup storages
2021-10-11 18:12:11 +02:00
c4dffa393b
backups: remove entries from database that don't exist in storage
...
fixes #772
2021-02-19 11:34:22 -08:00
382ae7424d
async 3: the whilst and doWhilst test funcs are async
2021-02-04 16:39:47 -08:00
282040ed1b
gcs: use delete concurrency
2021-02-01 14:23:15 -08:00
1da2450b10
gcs: use copy concurrency
2020-09-28 22:03:08 -07:00
8624e2260d
add storage api to make preflight checks
...
Currently there is only disk space checking but sshfs and cifs need
mount point checking as well
2020-06-08 16:25:05 +02:00
330b4a613c
Retrieve the backupPath from the storage provider itself
2020-06-05 13:27:18 +02:00
9bdeff0a39
Always use constants.SECRET_PLACEHOLDER
2020-05-14 23:02:02 +02:00
4d050725b7
storage: done events must be called next tick
...
It seems that listDir() returns synchronously (!), not sure how.
This results in the done event getting called with an error but
the EE event handlers are not setup yet.
2020-02-11 11:48:49 -08:00
85e17b570b
Use whilst instead of forever
...
this gets rid of the Error object
2019-12-04 11:17:44 -08:00
8c9ce30d29
Move BackupsError to BoxError
2019-10-22 21:03:47 -07:00
29ab3e91b3
gcs: remove concurrency logic
...
this is more complicated than necessary
2019-10-18 18:54:25 -07:00
05d3f8a667
gcs: fix crash
2019-05-12 18:05:48 -07:00
a88893b10a
remove/inject backups secret fields
...
follow same pattern as dns code
fixes #615
2019-02-09 20:44:05 -08:00
f546d53ca2
gcs: add listDir
...
part of #440
2018-07-28 03:51:32 -07:00
df98847535
Remove backupDone from storage interface
...
This is only used by caas
2018-07-27 12:08:19 -07:00
c8a6294772
lint
2018-02-22 12:24:16 -08:00
e4b12f0c4e
gcs: make testfile deletion work
...
It seems there is a race where the delete gets triggered even before the
file upload is complete. as a result, the delete succeeds but the file
is left on gcs.
2017-12-16 21:36:33 +05:30
61b56d4679
gcs: keyFilename is not used
...
also, some linter fixes
2017-12-16 21:36:29 +05:30
5c4ae6066d
gcs: lint
2017-12-15 17:34:09 +05:30
a35e048665
gcs: oldFilePath is not defined
2017-12-15 17:31:05 +05:30
48f6c39ae5
gcs: Make requires alphabetical
2017-12-15 17:28:45 +05:30
2cad93dfd2
Fixing UI to not require credentials be set (GCP use-case has no credentials field)
2017-10-31 12:33:15 +01:00
9b1f8febf1
Fixing listDir to support batchSize = -1 for non-chunked listings. Also strings extrapolation fix (ES6)
2017-10-31 11:40:00 +01:00
51ca1c7384
Refactoring gcs to match the new storage interface
2017-10-29 11:10:50 +01:00
051d04890b
Adding Google Cloud Storage support
2017-10-28 15:14:23 +02:00
Girish Ramakrishnan