Commit Graph

219 Commits

Author SHA1 Message Date
Johannes Zellner 1ef252fbc2 Revert "Rely on single private key file for optimized ssh remote fs operations"
This reverts commit aaebe01892.
2025-12-12 15:26:56 +01:00
Johannes Zellner aaebe01892 Rely on single private key file for optimized ssh remote fs operations 2025-12-12 14:44:19 +01:00
Johannes Zellner 043d89c03b Ensure we purge the ssh backup key file in case it was left over by a
previous failed backup run

fs.writeFileSync() would fail to overwrite due to restricted file mode
for ssh
2025-12-02 12:14:33 +01:00
Johannes Zellner c2378d33b4 Also use a temporary SSH identity file for optimized ssh remote rm -rf 2025-11-27 10:04:06 +01:00
Girish Ramakrishnan 4ef1339ba2 filesystem: handle non-existent prefix 2025-11-26 11:25:35 +01:00
Girish Ramakrishnan e106dcd76a storage: pass limits object to backend 2025-11-14 13:18:21 +01:00
Johannes Zellner afde81ef3e Use a temporary identity file for remote ssh copy 2025-11-13 10:27:33 +01:00
Johannes Zellner 21191bdc50 Give sshfs identity files unique filenames across mounts
If the same host was mounted as volume and backup or as a temporary
backup import, sharing the filename of the identify file would mean it
will get removed while still in use
2025-11-06 16:25:06 +01:00
Girish Ramakrishnan a518ee83cc backups: show same filesystem warning
fixes #867
2025-11-05 16:58:22 +01:00
Girish Ramakrishnan de84b5113c mounts: always return message when getting status 2025-11-05 16:52:32 +01:00
Girish Ramakrishnan 43e426ab9f Revert "Add no-use-before-define linter rule"
This reverts commit fdcc5d68a2.

Unfortunately, this requires us to move exports to the bottom.
This in turn causes circular dep issues and also access of
exports.GLOBAL_VAR in the global context
2025-10-08 21:17:52 +02:00
Girish Ramakrishnan f2316ec84e storage: remove usage of secret placeholder 2025-10-08 15:49:56 +02:00
Girish Ramakrishnan f92cdf36f5 system: return the mounted and unmounted block devices (as long as xfs or ext4) 2025-09-23 18:51:23 +02:00
Girish Ramakrishnan 31df40a841 storage: add copyDir
we changed listDir in c44863a9bb to list
a directory . this broke copy for files since a '/' is added when listing
the file.
2025-08-26 00:01:14 +02:00
Girish Ramakrishnan 832a25601d storage: make listDir return paths relative to the root
this seems more natural to work with
2025-08-16 18:13:02 +05:30
Girish Ramakrishnan f2225ff6b6 fs.rmdirSync { recursive: true } is deprecated 2025-08-14 13:43:41 +05:30
Girish Ramakrishnan 12e073e8cf use node: prefix for requires
mostly because code is being autogenerated by all the AI stuff using
this prefix. it's also used in the stack trace.
2025-08-14 12:55:35 +05:30
Girish Ramakrishnan fddbe6a574 Fixup tests 2025-08-04 23:11:50 +02:00
Johannes Zellner a887be432b Create correct mount objects for mountpoint validation 2025-08-04 15:25:27 +02:00
Johannes Zellner 705311f01f Fix typo 2025-08-04 15:17:27 +02:00
Johannes Zellner 1016d41d7a Rename backup label to name and separate backup listing into new view 2025-08-04 15:00:24 +02:00
Johannes Zellner 90c82ab1e7 make sure we only use mount objects with the mounts api 2025-08-04 14:40:35 +02:00
Johannes Zellner cf1e1aac77 Fixup some refactoring issues in mounting ext4 2025-08-04 13:32:12 +02:00
Girish Ramakrishnan 7ef9e7eb51 storage: add getStatus hook 2025-08-04 11:01:20 +02:00
Girish Ramakrishnan 547d8ae113 filesystem: rename backupFolder to backupDir 2025-08-02 10:37:37 +02:00
Girish Ramakrishnan 4fcaae8053 backup: fix listDir to return path instead of fullPath 2025-08-02 10:24:51 +02:00
Girish Ramakrishnan c935744f4c backups: root ~~canal~~ path surgery
remove rootPath and getBackupFilePath from the backup target and
make this backend specific.
2025-08-02 10:15:34 +02:00
Girish Ramakrishnan a01e1bad0f backuptarget: pseudo target for import and restore 2025-08-02 00:06:47 +02:00
Girish Ramakrishnan 53e9925880 backups: make id, provider a backend specific setting
the backend can stash whatever values it wants in the config.
just like the DNS backends, we make verifyConfig return a sanitized config.
added benefit is that extra user fields (via API) are also not dumped into the db.
2025-08-01 18:55:04 +02:00
Girish Ramakrishnan 9dfe6242b9 mounts: rename name to description
this makes it clear that the field is not important
2025-08-01 15:48:11 +02:00
Girish Ramakrishnan ea419509f1 backups: add setup/teardown
1. add setup, teardown hooks
2. move the managed mount setup and teardown to filesystem backend
3. remove this vague storage.js

we should convert storageApi into a real object, so we don't have to
keep passing apiConfig around
2025-08-01 15:36:25 +02:00
Girish Ramakrishnan 65f066d391 sshfs: implement rm via ssh
this is similar to change we did for cp -r . sshfs is often flaky with lots
of concurrent operations
2025-04-09 15:48:53 +02:00
Girish Ramakrishnan cb6d531300 storage: automatically abort old multipart uploads in s3 2025-02-13 11:09:15 +01:00
Girish Ramakrishnan 9888aa8c08 storage: rework listDir api to be a generator (like) function 2025-02-12 19:34:57 +01:00
Girish Ramakrishnan 23d91c7b37 upgrade readdirp 2025-02-05 15:50:10 +01:00
Girish Ramakrishnan 837d5803c8 backups: add preserve attributes checkbox 2025-01-24 13:59:01 +01:00
Girish Ramakrishnan 6407d795ed du: better error handling of du
du can fail when files and directories go missing. luckily, when du fails,
it still provides the best effort output
2024-11-06 14:54:52 +01:00
Girish Ramakrishnan 6c3ca9c364 shell: rework code to use shell.spawn
spawn gives out streams and we have more control over the stdout/stderr
buffers. otherwise, we have to provide a max buffer capture size to exec
2024-10-15 12:13:46 +02:00
Girish Ramakrishnan a9e1d7641d shell: make require take a tag 2024-10-14 21:08:32 +02:00
Girish Ramakrishnan ea72cef7f9 storage: remove getProviderStatus 2024-09-09 17:36:51 +02:00
Girish Ramakrishnan 1cd577cc65 filesystem: remove debug warning 2024-09-08 15:25:49 +02:00
Girish Ramakrishnan 247309e11b use constant 2024-07-30 11:00:50 +02:00
Girish Ramakrishnan b26ff08a3c shell: copy over code and signal values from cp object 2024-07-22 21:24:27 +02:00
Girish Ramakrishnan 44678cf5f1 sshfs: if remote copy fails, fallback to sshfs based copy
remote copy can file if there is no cp in the remote . for example,
if it was a windows server.
2024-07-22 20:53:19 +02:00
Girish Ramakrishnan 1dc6b40a68 tgz: extract using tar-stream directly
we used have a fork of tar-fs. using tar-stream directly gives us
more control
2024-07-08 13:06:56 +02:00
Johannes Zellner d122ece8e9 Attempt server side copy on sshfs via ssh exec
only so far tested agains hetzner storage boxes which apparently run BSD
unix tools
2024-07-02 19:51:34 +02:00
Girish Ramakrishnan 2b1b304c6e backup/import/restore: fix crash with root path calcuation
rootPath was calculated before the arguments were validated
2024-04-09 13:53:48 +02:00
Girish Ramakrishnan a6f078330f shell: no need to promise scoping 2024-02-21 19:40:27 +01:00
Girish Ramakrishnan cfd5c0f82b shell: rewrite exec to use execFile
this also renames execFile to execArgs
2024-02-21 18:54:43 +01:00
Girish Ramakrishnan 2237d2bbb7 shell: remove usage of .spawn 2024-02-21 13:27:04 +01:00