Commit Graph

369 Commits

Author SHA1 Message Date
Johannes Zellner d122ece8e9 Attempt server side copy on sshfs via ssh exec
only so far tested agains hetzner storage boxes which apparently run BSD
unix tools
2024-07-02 19:51:34 +02:00
Girish Ramakrishnan e481606d0e lint 2024-07-02 16:09:36 +02:00
Girish Ramakrishnan 2b1b304c6e backup/import/restore: fix crash with root path calcuation
rootPath was calculated before the arguments were validated
2024-04-09 13:53:48 +02:00
Girish Ramakrishnan a6f078330f shell: no need to promise scoping 2024-02-21 19:40:27 +01:00
Girish Ramakrishnan cfd5c0f82b shell: rewrite exec to use execFile
this also renames execFile to execArgs
2024-02-21 18:54:43 +01:00
Girish Ramakrishnan 2237d2bbb7 shell: remove usage of .spawn 2024-02-21 13:27:04 +01:00
Girish Ramakrishnan 54c7757e38 Fix crash 2024-02-20 21:53:52 +01:00
Girish Ramakrishnan 3da3ccedcb volumes: only wait for 5 seconds for mount status
mountpoint -q can never exit if the nfs mount disappears, for example
2024-02-20 21:38:57 +01:00
Girish Ramakrishnan 243a254f3e filesystem: remove hook should not rm recursively
this causes a bug in the backupcleaner when it tries to prune
empty directories when using the filesystem backend.

the bug is hit when a box backup is getting cleaned up but
one or more app backups are preserved.
2024-01-25 11:50:48 +01:00
Girish Ramakrishnan 934c701be2 vultr: fix copy of large objects
https://forum.cloudron.io/topic/10266/backups-are-failing
2023-10-26 09:51:07 +02:00
Girish Ramakrishnan aa8c23c8b3 rework backup root
notes:
* backup root cannot come from backend. for dynamic mounts backend cannot know where it is mounted
* backupConfig is 3 parts - format / mount / password . there is also this rootPath (which should not be in db)
* password should be stored separately in settings at some point
* format has to be passed along everywhere because we allow restore from  same backupConfig but different format. we do this by saving the format in the backups table

fixes #819
2023-08-15 22:51:45 +05:30
Girish Ramakrishnan e6b85c2df7 remount does not need a backend hook 2023-08-15 08:55:38 +05:30
Johannes Zellner 10646e9e04 Add generic disk (partition) backup provider to replace ext4 and xfs 2023-08-08 15:11:22 +02:00
Girish Ramakrishnan 3d5c21d9ca backups: encrypted backups must have .enc extension 2023-07-24 22:25:06 +05:30
Girish Ramakrishnan 050a82039a getBackupProviderStatus -> getProviderStatus 2023-07-15 11:00:45 +05:30
Girish Ramakrishnan c86059e070 backups: move limits into a sub object
fixes #817
2023-07-13 12:17:57 +05:30
Girish Ramakrishnan 4a1cdd4ef1 Update aws-sdk and suppress maintenance mode message
https://github.com/aws/aws-sdk-js/issues/4354
2023-05-11 22:18:00 +02:00
Girish Ramakrishnan c4f4f3e914 logs: use %o to format error
otherwise, they are printed as multi-line and this messes up tail+date formatting
2023-04-16 10:49:59 +02:00
Girish Ramakrishnan 093fc98ae5 s3: add listing check
This is needed for situations like in cloudflare where the endpoint can
be mistakenly configured with the bucket name like https://xx.r2.cloudflarestorage.com/cloudron-backups .
The upload and del calls work but list and copy does not.
2023-01-17 11:13:03 +01:00
Girish Ramakrishnan 40bcfdba76 s3: throw any copy errors 2023-01-17 11:00:03 +01:00
Girish Ramakrishnan 8a5d4e2fb0 better debugs 2022-11-05 08:43:02 +01:00
Girish Ramakrishnan 17c7cc5ec7 Remove external df module
It has some parsing issues with locale
2022-10-18 19:56:18 +02:00
Girish Ramakrishnan c3793da5bb split checkPrecondition so it can be used in cleaner as well 2022-10-02 17:41:21 +02:00
Girish Ramakrishnan 4f4a0ec289 use mount code to check mount status 2022-10-02 16:51:03 +02:00
Girish Ramakrishnan 56b981a52b backups: when checking mount status, ignore the prefix 2022-10-02 16:33:04 +02:00
Girish Ramakrishnan 074e9cfd93 rename getRootPath to getBackupRootPath 2022-10-02 16:26:27 +02:00
Girish Ramakrishnan 9d17c6606b rename to checkBackupPreconditions
since this is called only by the backup logic
2022-10-02 16:20:14 +02:00
Girish Ramakrishnan b32288050e backups: check mount status before checking available size 2022-10-02 16:16:30 +02:00
Girish Ramakrishnan 544ca6e1f4 initial xfs support 2022-06-08 10:58:00 -07:00
Girish Ramakrishnan a098c6da34 noop: removeDir is async 2022-04-30 16:35:39 -07:00
Girish Ramakrishnan 94e70aca33 storage: downloadDir is not part of interface 2022-04-30 16:24:49 -07:00
Girish Ramakrishnan ea01586b52 storage: make copy async 2022-04-30 16:24:45 -07:00
Girish Ramakrishnan 2280b7eaf5 Add S3MultipartDownloadStream
This extends the modern Readable class
2022-04-29 18:23:56 -07:00
Girish Ramakrishnan 64a2493ca2 Fixup prettyBytes 2022-04-15 17:56:24 -05:00
Girish Ramakrishnan 801dddc269 inline S3ReadStream 2022-04-15 11:25:15 -05:00
Girish Ramakrishnan 9a886111ad inline chunk.js 2022-04-15 09:25:54 -05:00
Girish Ramakrishnan bdc9a0cbe3 inline prettyBytes 2022-04-15 09:18:07 -05:00
Girish Ramakrishnan 555f914537 remove lodash.chunk 2022-04-15 08:07:46 -05:00
Girish Ramakrishnan f7ed044a40 fix storage test 2022-04-15 07:49:38 -05:00
Girish Ramakrishnan 0abc6c8844 replace pretty-bytes module 2022-04-15 07:34:16 -05:00
Girish Ramakrishnan 185d5d66ad even more constness 2022-04-14 20:30:00 -05:00
Girish Ramakrishnan 685bda35b9 storage: make remove and removeDir async 2022-04-14 16:16:20 -05:00
Girish Ramakrishnan d54c03f0a0 storage: make exists async 2022-04-14 12:24:34 -05:00
Girish Ramakrishnan 11f7be2065 storage: verifyConfig is now async 2022-04-14 12:24:30 -05:00
Girish Ramakrishnan a39e0ab934 storage: make remount async 2022-04-14 09:57:31 -05:00
Girish Ramakrishnan b51082f7e4 storage: checkPreconditions is now async 2022-04-14 07:40:19 -05:00
Girish Ramakrishnan 9ec76c69ec s3: make callback of getS3Config 2022-04-14 07:35:41 -05:00
Girish Ramakrishnan 452a4d9a75 backups: add remotePath
the main motivation is that id can be used in REST API routes. previously,
the id was a path and this had a "/" in it. This made /api/v1/backups/:backupId
not work.
2022-04-04 20:40:40 -07:00
Girish Ramakrishnan 54934c41a7 storage: rename getBackupPath to getBasePath 2022-04-04 14:08:24 -07:00
Girish Ramakrishnan 2434e81383 backups: fix incorrect mountpoint check with managed mounts 2022-02-25 12:53:05 -08:00