Commit Graph

340 Commits

Author SHA1 Message Date
Girish Ramakrishnan 0d535d2d5c allocate swap size for containers based on system ratio 2021-01-20 18:41:51 -08:00
Girish Ramakrishnan 7b24239d38 update the service config in addons code 2021-01-20 11:10:50 -08:00
Girish Ramakrishnan 025eb18411 Use a single memoryLimit instead of memory and memorySwap
We will make the percent allocation dynamic depending on the system.

When we have servers with a large amount of RAM but little swap, we
seem to use a lot of swap because of 50% allocation strategy. In such
systems, we run out of swap and thus have OOM errors even though there
is a lot of RAM available!
2021-01-19 19:43:41 -08:00
Girish Ramakrishnan 3b8bc47ee5 Set and clear timeout for external requests
otherwise, the server crashes for a write after timeout
2020-12-06 23:31:57 -08:00
Girish Ramakrishnan 258d81d7e9 mongo: bring mem limit in-line with others 2020-12-04 11:04:21 -08:00
Girish Ramakrishnan 89c82fb001 send the raw healthcheck as part of status call 2020-12-02 17:07:33 -08:00
Girish Ramakrishnan 6218ee30a7 proxyAuth: inject CLOUDRON_PROXY_AUTH 2020-11-26 15:04:25 -08:00
Girish Ramakrishnan 3a0b9d7b3b turn: add note 2020-11-25 10:19:01 -08:00
Girish Ramakrishnan e511b70d8f bring back resolvconf and unbound DNS
bd9c664b1a tried to remove it and use
the system resolver. However, we found that debian has a quirk that it adds
it adds the fqdn as 127.0.1.1. This means that the docker containers
resolve the my.example.com domain to that and can't connect.

This affects any apps doing a turn test (CLOUDRON_TURN/STUN_SERVER)
and also apps like SOGo which use the mail server hostname directly (since
they require proper certs).

https://www.debian.org/doc/manuals/debian-reference/ch05.en.html#_the_hostname_resolution

So, the solution is to go back to unbound, now that port 53 binding is specially
handled anyway in docker.js
2020-11-25 10:02:43 -08:00
Girish Ramakrishnan bd9c664b1a Free up port 53
It's all very complicated.

Approach 1: Simple move unbound to not listen on 0.0.0.0 and only the internal
ones. However, docker has no way to bind only to the "public" interface.

Approach 2: Move the internal unbound to some other port. This required a PR
for haraka - https://github.com/haraka/Haraka/pull/2863 . This works and we use
systemd-resolved by default. However, it turns out systemd-resolved with hog the
lo and thus docker cannot bind again to port 53.

Approach 3: Get rid of systemd-resolved and try to put the dns server list in
/etc/resolv.conf. This is surprisingly hard because the DNS listing can come from
DHCP or netplan or wherever. We can hardcode some public DNS servers but this seems
not a good idea for privacy.

Approach 4: So maybe we don't move the unbound away to different port after all.
However, all the work for approach 2 is done and it's quite nice that the default
resolver is used with the default dns server of the network (probably a caching
server + also maybe has some home network firewalled dns).

So, the final solution is to bind to the make docker bind to the IP explicity.
It's unclear what will happen if the IP changes, maybe it needs a restart.
2020-11-18 23:25:56 -08:00
Girish Ramakrishnan 625dc7c49b Add proxyAuth as an addon 2020-11-10 16:50:36 -08:00
Girish Ramakrishnan 5541b89cf7 Revert "redis: add optional flag"
This reverts commit 0cac5610c8.
2020-10-27 08:48:45 -07:00
Girish Ramakrishnan aaeed5d18b Revert "Another check for redis services configs"
This reverts commit d6c3c8a294.
2020-10-27 08:48:17 -07:00
Johannes Zellner d6c3c8a294 Another check for redis services configs 2020-10-27 14:47:52 +01:00
Johannes Zellner d337fc6d47 Do not crash if an app does not have a redis service config 2020-10-27 09:32:22 +01:00
Girish Ramakrishnan 0cac5610c8 redis: add optional flag 2020-10-24 10:34:30 -07:00
Girish Ramakrishnan 4881d8e3a1 Add option to allow non-admins to access SFTP 2020-10-21 23:38:13 -07:00
Girish Ramakrishnan 4836b16030 postgresql: make the locale configurable 2020-10-12 18:57:34 -07:00
Girish Ramakrishnan e422dd1198 turn service must be rebuilt on dashboard domain change
restart only restarts the container and does not affect the env
variables.
2020-09-23 15:18:28 -07:00
Girish Ramakrishnan a75928d805 Fix coding style 2020-09-23 15:13:23 -07:00
Girish Ramakrishnan 4da8c8d6db updateServiceConfig: remove retry from platform code 2020-09-22 21:46:11 -07:00
Girish Ramakrishnan 3c565defca retry setting memory of services 2020-09-22 21:42:47 -07:00
Girish Ramakrishnan eae33161c1 Forgot the CLOUDRON_ prefix 2020-09-08 19:33:59 -07:00
Girish Ramakrishnan f14df141f7 Add MAIL_SERVER_HOST
This points to the mail fqdn
2020-09-08 19:33:23 -07:00
Girish Ramakrishnan f7a4330cd1 Add CLOUDRON_LDAP_HOST
We have MYSQL_HOST, POSTGRESQL_HOST etc. Just this LDAP has _SERVER
2020-09-08 19:32:21 -07:00
Girish Ramakrishnan 316a1ae2c5 only scale back containers on infra change 2020-09-02 18:13:08 -07:00
Girish Ramakrishnan d5ec38c4db do not restrict postgresql db memory
see also 3ea6610923
2020-08-30 21:37:57 -07:00
Girish Ramakrishnan 3ea6610923 do not restrict memory on startup of database addons
this helps the import case where we need all the memory we can get.
we scale the memory down once platform is ready in any case.
2020-08-20 11:16:35 -07:00
Girish Ramakrishnan 60a787ce3d If db name exists, re-use it (for repair mode) 2020-08-17 12:04:02 -07:00
Girish Ramakrishnan f96bc6d5f4 keep mongodb database names short 2020-08-17 10:28:49 -07:00
Girish Ramakrishnan aeee8afc02 export database: fix async logic 2020-08-09 11:14:11 -07:00
Girish Ramakrishnan 1b307632ab Use debug instead of console.* everywhere
No need to patch up console.* anymore

also removes supererror
2020-08-02 12:04:55 -07:00
Girish Ramakrishnan d089444441 db upgrade: stop containers only after exporting
we cannot export if the containers were nuked in the platform logic.
for this reason, move the removal near the place where they get started.
2020-07-30 15:28:53 -07:00
Girish Ramakrishnan 16288cf277 better debug 2020-07-30 11:42:03 -07:00
Girish Ramakrishnan fe35f4497b Fix two typos 2020-07-30 10:58:24 -07:00
Girish Ramakrishnan 625463f6ab export the database before upgrade
it's possible that
a) backups are completely disabled
b) skip backup option is selected when upgrading

in the above cases, the dump file is not generated and thus any addon
upgrade will fail. to fix, we dump the db fresh for database upgrades.
2020-07-30 10:23:08 -07:00
Johannes Zellner 0aab891980 Support nginx logs 2020-06-15 17:30:16 +02:00
Girish Ramakrishnan 4ede765e1f typo: memoryLimit -> memory 2020-05-29 13:29:01 -07:00
Girish Ramakrishnan de4b3e55fa Use apps.getAll so that app.fqdn is valid 2020-05-24 18:21:35 -07:00
Girish Ramakrishnan b8c7357fea redis: if container inactive, return stopped status 2020-05-18 14:43:23 -07:00
Girish Ramakrishnan 819f8e338f stop app now stops it's services as well 2020-05-18 14:33:07 -07:00
Girish Ramakrishnan 9569e46ff8 use docker.restart instead of start/stop since it is atomic 2020-05-18 13:35:42 -07:00
Girish Ramakrishnan dcaccc2d7a add redis status
part of #671
2020-05-03 19:46:07 -07:00
Girish Ramakrishnan 66f66fd14f docker: clean up volume API 2020-04-29 21:28:49 -07:00
Girish Ramakrishnan efb39fb24b refactor for addon/service/container consistency
addon - app manifest thing. part of app lifecycle
services - implementation of addon (may have containers assoc)
2020-04-28 15:32:02 -07:00
Girish Ramakrishnan fbcc4cfa50 Rename KNOWN_ADDONS to ADDONS 2020-04-27 22:59:35 -07:00
Girish Ramakrishnan 474a3548e0 Rename KNOWN_SERVICES to SERVICES 2020-04-27 22:59:11 -07:00
Girish Ramakrishnan 2cdf68379b Revert "add volume support"
This reverts commit b8bb69f730.

Revert this for now, we will try a simpler non-object volume first
2020-04-27 22:55:43 -07:00
Girish Ramakrishnan b8bb69f730 add volume support
part of #668, #569
2020-04-24 22:09:07 -07:00
Girish Ramakrishnan 2dcab77ed1 Fix issue where app with oauth addon will not backup or uninstall 2020-04-18 10:08:20 -07:00