Commit Graph

63 Commits

Author SHA1 Message Date
Girish Ramakrishnan 4fcaae8053 backup: fix listDir to return path instead of fullPath 2025-08-02 10:24:51 +02:00
Girish Ramakrishnan c935744f4c backups: root ~~canal~~ path surgery
remove rootPath and getBackupFilePath from the backup target and
make this backend specific.
2025-08-02 10:15:34 +02:00
Girish Ramakrishnan 53e9925880 backups: make id, provider a backend specific setting
the backend can stash whatever values it wants in the config.
just like the DNS backends, we make verifyConfig return a sanitized config.
added benefit is that extra user fields (via API) are also not dumped into the db.
2025-08-01 18:55:04 +02:00
Girish Ramakrishnan ea419509f1 backups: add setup/teardown
1. add setup, teardown hooks
2. move the managed mount setup and teardown to filesystem backend
3. remove this vague storage.js

we should convert storageApi into a real object, so we don't have to
keep passing apiConfig around
2025-08-01 15:36:25 +02:00
Girish Ramakrishnan cb6d531300 storage: automatically abort old multipart uploads in s3 2025-02-13 11:09:15 +01:00
Girish Ramakrishnan 9888aa8c08 storage: rework listDir api to be a generator (like) function 2025-02-12 19:34:57 +01:00
Girish Ramakrishnan ea72cef7f9 storage: remove getProviderStatus 2024-09-09 17:36:51 +02:00
Girish Ramakrishnan 1dc6b40a68 tgz: extract using tar-stream directly
we used have a fork of tar-fs. using tar-stream directly gives us
more control
2024-07-08 13:06:56 +02:00
Girish Ramakrishnan aa8c23c8b3 rework backup root
notes:
* backup root cannot come from backend. for dynamic mounts backend cannot know where it is mounted
* backupConfig is 3 parts - format / mount / password . there is also this rootPath (which should not be in db)
* password should be stored separately in settings at some point
* format has to be passed along everywhere because we allow restore from  same backupConfig but different format. we do this by saving the format in the backups table

fixes #819
2023-08-15 22:51:45 +05:30
Girish Ramakrishnan e6b85c2df7 remount does not need a backend hook 2023-08-15 08:55:38 +05:30
Girish Ramakrishnan 3d5c21d9ca backups: encrypted backups must have .enc extension 2023-07-24 22:25:06 +05:30
Girish Ramakrishnan 050a82039a getBackupProviderStatus -> getProviderStatus 2023-07-15 11:00:45 +05:30
Girish Ramakrishnan c3793da5bb split checkPrecondition so it can be used in cleaner as well 2022-10-02 17:41:21 +02:00
Girish Ramakrishnan 074e9cfd93 rename getRootPath to getBackupRootPath 2022-10-02 16:26:27 +02:00
Girish Ramakrishnan 9d17c6606b rename to checkBackupPreconditions
since this is called only by the backup logic
2022-10-02 16:20:14 +02:00
Girish Ramakrishnan 94e70aca33 storage: downloadDir is not part of interface 2022-04-30 16:24:49 -07:00
Girish Ramakrishnan ea01586b52 storage: make copy async 2022-04-30 16:24:45 -07:00
Girish Ramakrishnan 685bda35b9 storage: make remove and removeDir async 2022-04-14 16:16:20 -05:00
Girish Ramakrishnan d54c03f0a0 storage: make exists async 2022-04-14 12:24:34 -05:00
Girish Ramakrishnan 11f7be2065 storage: verifyConfig is now async 2022-04-14 12:24:30 -05:00
Girish Ramakrishnan a39e0ab934 storage: make remount async 2022-04-14 09:57:31 -05:00
Girish Ramakrishnan b51082f7e4 storage: checkPreconditions is now async 2022-04-14 07:40:19 -05:00
Girish Ramakrishnan 452a4d9a75 backups: add remotePath
the main motivation is that id can be used in REST API routes. previously,
the id was a path and this had a "/" in it. This made /api/v1/backups/:backupId
not work.
2022-04-04 20:40:40 -07:00
Girish Ramakrishnan 54934c41a7 storage: rename getBackupPath to getBasePath 2022-04-04 14:08:24 -07:00
Johannes Zellner d1a7172895 Add remount route for mountlike backup storages 2021-10-11 18:12:11 +02:00
Girish Ramakrishnan c4dffa393b backups: remove entries from database that don't exist in storage
fixes #772
2021-02-19 11:34:22 -08:00
Johannes Zellner 8624e2260d add storage api to make preflight checks
Currently there is only disk space checking but sshfs and cifs need
mount point checking as well
2020-06-08 16:25:05 +02:00
Johannes Zellner 330b4a613c Retrieve the backupPath from the storage provider itself 2020-06-05 13:27:18 +02:00
Johannes Zellner 9bdeff0a39 Always use constants.SECRET_PLACEHOLDER 2020-05-14 23:02:02 +02:00
Girish Ramakrishnan d0a3d00492 Use NOT_IMPLEMENTED error code 2019-12-04 10:22:22 -08:00
Girish Ramakrishnan a88893b10a remove/inject backups secret fields
follow same pattern as dns code

fixes #615
2019-02-09 20:44:05 -08:00
Girish Ramakrishnan 737541f707 storage: standardize the listDir API 2018-07-27 15:23:26 -07:00
Girish Ramakrishnan df98847535 Remove backupDone from storage interface
This is only used by caas
2018-07-27 12:08:19 -07:00
Girish Ramakrishnan 98e19e6df5 fix upload errors causing double callback 2018-02-27 19:16:03 -08:00
Girish Ramakrishnan 0bba985ff1 storage: Add implementation note 2018-02-22 12:30:55 -08:00
Girish Ramakrishnan e022dbf8a6 Revert "merge the done callback into the main code"
This reverts commit c39bec8cc1.

This was committed with extra stuff by mistake
2018-02-22 10:58:56 -08:00
Girish Ramakrishnan c39bec8cc1 merge the done callback into the main code 2018-02-21 20:17:58 -08:00
Girish Ramakrishnan ac9e421ecf improved backup progress and logging 2017-10-10 22:49:38 -07:00
Girish Ramakrishnan d70ff7cd5b Make copy() return event emitter
This way the storage logic does not need to rely on progress
2017-10-04 11:02:50 -07:00
Girish Ramakrishnan a34180c27b Add format to backupsdb
Call remove/removeDir based on the format
2017-09-27 18:02:30 -07:00
Girish Ramakrishnan a9e89b57d9 merge caas storage into s3 backend 2017-09-26 12:28:33 -07:00
Girish Ramakrishnan b0f85678d4 Implement downloadDir for flat-file format 2017-09-23 18:07:26 -07:00
Girish Ramakrishnan e43413e063 implement remove dir in storage backends 2017-09-23 12:34:51 -07:00
Girish Ramakrishnan 97da8717ca Refactor backup strategy logic into backups.js 2017-09-20 14:09:55 -07:00
Girish Ramakrishnan 26f33a8e9b Send resolved path to the storage APIs 2017-09-19 21:58:35 -07:00
Girish Ramakrishnan 781f543e87 Rename API calls in the storage backend 2017-09-17 18:50:29 -07:00
Girish Ramakrishnan e78d976c8f Fix backup mapping (mail dir has moved) 2017-09-13 09:51:20 -07:00
Girish Ramakrishnan aeba8e8fd2 Use retentionSecs to cleanup backups
Part of #310
2017-04-23 19:34:00 -07:00
Johannes Zellner a7d7935451 Remove backup download code 2017-04-22 22:17:29 +02:00
Johannes Zellner 33492333c7 Fix backupDone() for caas 2017-04-21 10:31:43 +02:00