Commit Graph

202 Commits

Author SHA1 Message Date
Girish Ramakrishnan
c4f4f3e914 logs: use %o to format error
otherwise, they are printed as multi-line and this messes up tail+date formatting
2023-04-16 10:49:59 +02:00
Girish Ramakrishnan
582994b9d6 addons: stable IPv4 addresses
give addons static IPv4 so one can reliably connect from outside via
SSH tunnel
2023-02-21 12:20:44 +01:00
Girish Ramakrishnan
36ddb8c7c2 prune: normalize the tag 2022-11-30 21:12:00 +01:00
Girish Ramakrishnan
6c9aa1a77f Revert "prune all images instead of parsing output"
This reverts commit d42c524a46.

This caused a bug that all app images are getting removed since we remove
all containers on infra update!
2022-11-30 20:00:51 +01:00
Girish Ramakrishnan
79cb8ef251 add route to get platform status 2022-11-30 19:54:32 +01:00
Girish Ramakrishnan
d42c524a46 prune all images instead of parsing output
nothing is really lost since these are just unused images
2022-11-30 10:01:50 +01:00
Girish Ramakrishnan
43f86674b4 Remove delay module 2022-04-15 07:52:35 -05:00
Girish Ramakrishnan
6aa034ea41 platform: Only re-create docker network on version change 2022-02-10 09:32:22 -08:00
Girish Ramakrishnan
ca83deb761 Docker IPv6 support
Docker's initial IPv6 support is based on allocating public IPv6 to containers.
This approach has many issues:
* The server may not get a block of IPv6 assigned to it
* It's complicated to allocate a block of IPv6 to cloudron server on home setups
* It's unclear how dynamic IPv6 is. If it's dynamic, then should containers be recreated?
* DNS setup is complicated
* Not a issue for Cloudron itself, but with -P, it just exposed the full container into the world

Given these issues, IPv6 NAT is being considered. Even though NAT is not a security mechanism as such,
it does offer benefits that we care about:
* We can allocate some private IPv6 to containers
* Have docker NAT66 the exposed ports
* Works similar to IPv4

Currently, the IPv6 ports are always mapped and exposed. The "Enable IPv6" config option is only whether
to automate AAAA records or not. This way, user can enable it and 'sync' dns and we don't need to
re-create containers etc. There is no inherent benefit is not exposing IPv6 at all everywhere unless we find
it unstable.

Fixes #264
2022-02-09 23:54:53 -08:00
Girish Ramakrishnan
ff664486ff do not start if platform.start does not work 2022-02-09 23:15:37 -08:00
Girish Ramakrishnan
852eebac4d move cloudron network creation to platform code
this gives us more control on re-creating the network with different
arguments/options when needed.
2022-02-09 23:15:37 -08:00
Girish Ramakrishnan
7ffcfc5206 auditSource: add PLATFORM 2021-11-17 10:33:28 -08:00
Girish Ramakrishnan
af2a8ba07f add retry to platform.start instead
this is because it holds a lock and cannot be re-tried

See also 0c0aeeae4c which tried to
make it for all startup tasks
2021-11-02 23:35:53 -07:00
Girish Ramakrishnan
3bcd32c56d restore: mount all volumes before restoring apps
fixes #786
2021-09-28 11:51:01 -07:00
Girish Ramakrishnan
7709e155e0 more async'ification 2021-09-07 11:21:06 -07:00
Girish Ramakrishnan
77f5cb183b merge appdb.js into apps.js 2021-08-23 15:35:38 -07:00
Girish Ramakrishnan
e59d0e878d merge taskdb into tasks.js 2021-07-14 10:37:12 -07:00
Girish Ramakrishnan
5b51f73be4 restore: add skipDnsSetup flag
part of #737
2021-02-24 14:56:09 -08:00
Girish Ramakrishnan
9f9575f46a Fixes to service configuration
restart service does not rebuild automatically, we should add a route
for that. we need to figure where to scale services etc if we randomly
create containers like that.
2021-01-21 17:41:22 -08:00
Girish Ramakrishnan
0edb673dc6 rename platform config to services config 2021-01-21 12:19:57 -08:00
Girish Ramakrishnan
cd1b46848e Fix bug where graphite and sftp are not incrementally upgraded 2021-01-21 12:00:23 -08:00
Girish Ramakrishnan
6bd87485c6 rename addons.js to services.js
services is the named container (services view)
addons is more like a heroku concept
2021-01-21 11:31:35 -08:00
Girish Ramakrishnan
6ae1de6989 test: make apps test work 2020-11-21 23:25:28 -08:00
Girish Ramakrishnan
09c8248e31 move back docker network creation to start.sh
dockerproxy and unbound rely on it.
2020-11-20 17:22:57 -08:00
Girish Ramakrishnan
c0b0029935 statically allocate app container IPs
We removed httpPort with the assumption that docker allocated IPs
and kept them as long as the container is around. This turned out
to be not true because the IP changes on even container restart.

So we now allocate IPs statically. The iprange makes sure we don't
overlap with addons and other CI app or JupyterHub apps.

https://github.com/moby/moby/issues/6743
https://github.com/moby/moby/pull/19001
2020-11-20 16:19:59 -08:00
Girish Ramakrishnan
4da8c8d6db updateServiceConfig: remove retry from platform code 2020-09-22 21:46:11 -07:00
Girish Ramakrishnan
316a1ae2c5 only scale back containers on infra change 2020-09-02 18:13:08 -07:00
Girish Ramakrishnan
b1b6f70118 Kill all tasks on shutdown and startup
BindsTo will kill all the tasks when systemctl stop box is executed.
But when restarted, it keeps the tasks running. Because of this behavior,
we kill the tasks on startup and stop of the box code.
2020-08-06 23:47:40 -07:00
Girish Ramakrishnan
d089444441 db upgrade: stop containers only after exporting
we cannot export if the containers were nuked in the platform logic.
for this reason, move the removal near the place where they get started.
2020-07-30 15:28:53 -07:00
Girish Ramakrishnan
b0d65a1bae rename startApps to markApps 2020-07-30 15:28:50 -07:00
Girish Ramakrishnan
7efb57c8da restart apps on addon container change
when the IP changes on addon container re-create, the apps don't
detect this (maybe there is some large DNS cache timeout in docker)
2020-05-22 16:45:03 -07:00
Johannes Zellner
b366b0fa6a Stop container with isCloudronManged labels instead of by network 2020-03-28 22:46:32 +01:00
Girish Ramakrishnan
85c13cae58 Fix platform update logic 2019-09-24 21:21:49 -07:00
Girish Ramakrishnan
1a061e4446 Only check installationState to resume tasks
also, make resumeTasks go via app logic to capture end of task
2019-09-24 00:37:29 -07:00
Girish Ramakrishnan
9f1210202a port taskmanager to use tasks 2019-08-28 15:17:53 -07:00
Girish Ramakrishnan
9c2ff2f862 fix image prune logic 2019-05-29 12:15:13 -07:00
Johannes Zellner
79ccfa087d Only delete docker container within the cloudron network
When running tests they would randomly delete all local docker container
2019-04-17 12:44:03 +02:00
Girish Ramakrishnan
a0d1016c01 containerize sftp 2019-04-04 22:43:02 -07:00
Johannes Zellner
ef0403af0e Make proftpd conf static 2019-03-19 15:24:09 -07:00
Johannes Zellner
044b27967e Make initial sftp connection work 2019-03-19 15:24:09 -07:00
Girish Ramakrishnan
da2b00c9cf Move cert change notification into ensureCertificate()
When ensureCertificate renews the cert, the filename will match the
nginx config cert file. The current code detects that this implies
that the cert has not changed and thus does not update mail container.

Move the notification into ensureCertificate() itself. If we have a wildcard
cert and it gets renewed when installing a new app, then mail container will
still get it.
2019-03-04 15:24:09 -08:00
Girish Ramakrishnan
f3189f72fd Use mail.restartMail alias 2019-03-04 10:42:42 -08:00
Girish Ramakrishnan
e7294f2950 Make handleCertChanged take a callback 2018-12-11 11:02:32 -08:00
Girish Ramakrishnan
6d128595e7 all container related functionality is 'service' 2018-12-02 19:12:43 -08:00
Girish Ramakrishnan
4142d7a050 Fix error handling of all the execSync usage 2018-11-23 13:11:15 -08:00
Girish Ramakrishnan
9b4fffde29 Use shell.exec instead of shell.execSync 2018-11-23 11:18:45 -08:00
Girish Ramakrishnan
5c9b6736f0 Remove pruneInfraImages out of platform startup
If it fails, it's really OK. Maybe we will have a button to remove
images.

This goes further into step of webadmin always staying up and not
crashing/erroring for cosmetic issues.
2018-11-17 19:35:39 -08:00
Girish Ramakrishnan
82037b70e4 retry apply of platform config
it seems that scaling down addons can fail at times. this can happen
if the kernel is busy swapping things etc. so, we do not block the
platform startup for this operation.

Nov 13 05:20:00 localhost dockerd[28831]: time="2018-11-13T05:20:00.365056059Z" level=error msg="Handler for POST /v1.37/containers/mongodb/update returned error: Cannot update container 6532d4a923ce9f10303f2e2aa7f03c35383864f44f3db6abd5c58da3c1a7702f: docker-runc did not terminate sucessfully: failed to write 419430400 to memory.memsw.limit_in_bytes: write /sys/fs/cgroup/memory/docker/6532d4a923ce9f10303f2e2aa7f03c35383864f44f3db6abd5c58da3c1a7702f/memory.memsw.limit_in_bytes: device or resource busy\n: unknown"
Nov 13 05:20:00 localhost node[5486]: box:shell updatemongodb (stderr): Error response from daemon: Cannot update container 6532d4a923ce9f10303f2e2aa7f03c35383864f44f3db6abd5c58da3c1a7702f: docker-runc did not terminate sucessfully: failed to write 419430400 to memory.memsw.limit_in_bytes: write /sys/fs/cgroup/memory/docker/6532d4a923ce9f10303f2e2aa7f03c35383864f44f3db6abd5c58da3c1a7702f/memory.memsw.limit_in_bytes: device or resource busy
2018-11-12 21:58:57 -08:00
Girish Ramakrishnan
a3a22604ba Mark import errors as app errors 2018-11-11 10:36:11 -08:00
Girish Ramakrishnan
78ac1d2a12 Add isCloudronManaged label to containers managed by cloudron 2018-11-10 19:00:03 -08:00