Commit Graph

430 Commits

Author SHA1 Message Date
Johannes Zellner
21191bdc50 Give sshfs identity files unique filenames across mounts
If the same host was mounted as volume and backup or as a temporary
backup import, sharing the filename of the identify file would mean it
will get removed while still in use
2025-11-06 16:25:06 +01:00
Girish Ramakrishnan
a518ee83cc backups: show same filesystem warning
fixes #867
2025-11-05 16:58:22 +01:00
Girish Ramakrishnan
de84b5113c mounts: always return message when getting status 2025-11-05 16:52:32 +01:00
Girish Ramakrishnan
d6a4dd6965 backup sites: fix listing when status call errors
* fix backend to not retry in status call
* fix frontend to continue loading view if status errors
* fix connect-lastmile to show the exact path that is timing out
2025-10-16 14:13:31 +02:00
Girish Ramakrishnan
11f2acd702 init data with initial config and only mutate what we allow editing 2025-10-09 12:20:59 +02:00
Girish Ramakrishnan
43e426ab9f Revert "Add no-use-before-define linter rule"
This reverts commit fdcc5d68a2.

Unfortunately, this requires us to move exports to the bottom.
This in turn causes circular dep issues and also access of
exports.GLOBAL_VAR in the global context
2025-10-08 21:17:52 +02:00
Girish Ramakrishnan
f2316ec84e storage: remove usage of secret placeholder 2025-10-08 15:49:56 +02:00
Girish Ramakrishnan
f92cdf36f5 system: return the mounted and unmounted block devices (as long as xfs or ext4) 2025-09-23 18:51:23 +02:00
Girish Ramakrishnan
305441ea28 backups: remove noop backend
the noop backend is migrated into 0 sites config.

when the updater code sees that there is no site to backup, it will
just fail. user has to manually update with skipBackup flag.
2025-09-22 16:44:12 +02:00
Girish Ramakrishnan
43cc91aca2 fix tests 2025-09-13 15:22:01 +02:00
Girish Ramakrishnan
31df40a841 storage: add copyDir
we changed listDir in c44863a9bb to list
a directory . this broke copy for files since a '/' is added when listing
the file.
2025-08-26 00:01:14 +02:00
Girish Ramakrishnan
c44863a9bb s3: ensure we list directory
<timestamp>/box_v9.0.0.backupinfo ends up getting listed
when trying to list <timestamp>/box_v9.0.0 !
2025-08-25 19:15:21 +02:00
Girish Ramakrishnan
832a25601d storage: make listDir return paths relative to the root
this seems more natural to work with
2025-08-16 18:13:02 +05:30
Girish Ramakrishnan
f2225ff6b6 fs.rmdirSync { recursive: true } is deprecated 2025-08-14 13:43:41 +05:30
Girish Ramakrishnan
12e073e8cf use node: prefix for requires
mostly because code is being autogenerated by all the AI stuff using
this prefix. it's also used in the stack trace.
2025-08-14 12:55:35 +05:30
Girish Ramakrishnan
fddbe6a574 Fixup tests 2025-08-04 23:11:50 +02:00
Johannes Zellner
a887be432b Create correct mount objects for mountpoint validation 2025-08-04 15:25:27 +02:00
Johannes Zellner
705311f01f Fix typo 2025-08-04 15:17:27 +02:00
Johannes Zellner
1016d41d7a Rename backup label to name and separate backup listing into new view 2025-08-04 15:00:24 +02:00
Johannes Zellner
90c82ab1e7 make sure we only use mount objects with the mounts api 2025-08-04 14:40:35 +02:00
Johannes Zellner
cf1e1aac77 Fixup some refactoring issues in mounting ext4 2025-08-04 13:32:12 +02:00
Girish Ramakrishnan
7ef9e7eb51 storage: add getStatus hook 2025-08-04 11:01:20 +02:00
Girish Ramakrishnan
547d8ae113 filesystem: rename backupFolder to backupDir 2025-08-02 10:37:37 +02:00
Girish Ramakrishnan
4fcaae8053 backup: fix listDir to return path instead of fullPath 2025-08-02 10:24:51 +02:00
Girish Ramakrishnan
c935744f4c backups: root ~~canal~~ path surgery
remove rootPath and getBackupFilePath from the backup target and
make this backend specific.
2025-08-02 10:15:34 +02:00
Girish Ramakrishnan
a01e1bad0f backuptarget: pseudo target for import and restore 2025-08-02 00:06:47 +02:00
Girish Ramakrishnan
53e9925880 backups: make id, provider a backend specific setting
the backend can stash whatever values it wants in the config.
just like the DNS backends, we make verifyConfig return a sanitized config.
added benefit is that extra user fields (via API) are also not dumped into the db.
2025-08-01 18:55:04 +02:00
Girish Ramakrishnan
9dfe6242b9 mounts: rename name to description
this makes it clear that the field is not important
2025-08-01 15:48:11 +02:00
Girish Ramakrishnan
ea419509f1 backups: add setup/teardown
1. add setup, teardown hooks
2. move the managed mount setup and teardown to filesystem backend
3. remove this vague storage.js

we should convert storageApi into a real object, so we don't have to
keep passing apiConfig around
2025-08-01 15:36:25 +02:00
Johannes Zellner
2386124089 Upcloud S3 does not return a Contents property if no keys are found 2025-07-22 18:55:44 +02:00
Girish Ramakrishnan
5548ed1b57 s3: fix malformed endpoint URL from older versions
this is specifically for linode, but may apply for other storages too
2025-07-16 09:12:57 +02:00
Johannes Zellner
4eed77e123 Do not crash in gcs if remote path has zero objects 2025-07-15 20:39:38 +02:00
Girish Ramakrishnan
bb257f2bf2 s3: make delete objects work again again 2025-07-15 01:11:23 +02:00
Girish Ramakrishnan
76f365f7e8 s3: make delete objects work again 2025-07-15 00:13:40 +02:00
Girish Ramakrishnan
499336e752 s3: fix debug 2025-07-14 22:24:23 +02:00
Girish Ramakrishnan
e53ec3e43c s3: add debugging code 2025-07-14 21:59:56 +02:00
Girish Ramakrishnan
15ac81c1cd s3: format errors with .Code 2025-07-14 15:54:31 +02:00
Girish Ramakrishnan
241053e1a8 s3: error handling has changed in v3 api 2025-06-20 16:07:21 +02:00
Girish Ramakrishnan
48bf21a537 s3: use listObjectsV2 API
in some minio instances, listObjectV1 is not returning all the objects
2025-05-26 14:17:13 +02:00
Girish Ramakrishnan
65f066d391 sshfs: implement rm via ssh
this is similar to change we did for cp -r . sshfs is often flaky with lots
of concurrent operations
2025-04-09 15:48:53 +02:00
Girish Ramakrishnan
20a30b303b s3: fix encoding of downloaded stream 2025-02-13 19:29:07 +01:00
Girish Ramakrishnan
59721a3f1a s3: in v3, Body is a stream
https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/migrate-s3.html
2025-02-13 15:17:48 +01:00
Girish Ramakrishnan
cb6d531300 storage: automatically abort old multipart uploads in s3 2025-02-13 11:09:15 +01:00
Girish Ramakrishnan
a138425298 storage: start migration of s3 api 2025-02-12 23:04:37 +01:00
Girish Ramakrishnan
9888aa8c08 storage: rework listDir api to be a generator (like) function 2025-02-12 19:34:57 +01:00
Girish Ramakrishnan
23d91c7b37 upgrade readdirp 2025-02-05 15:50:10 +01:00
Girish Ramakrishnan
837d5803c8 backups: add preserve attributes checkbox 2025-01-24 13:59:01 +01:00
Girish Ramakrishnan
b4d58f0609 aws: add a 20min timeout
in some services like b2, the multi-part copy just hangs. this allows
us to retry
2024-11-20 07:13:43 +05:30
Girish Ramakrishnan
6407d795ed du: better error handling of du
du can fail when files and directories go missing. luckily, when du fails,
it still provides the best effort output
2024-11-06 14:54:52 +01:00
Girish Ramakrishnan
6c3ca9c364 shell: rework code to use shell.spawn
spawn gives out streams and we have more control over the stdout/stderr
buffers. otherwise, we have to provide a max buffer capture size to exec
2024-10-15 12:13:46 +02:00