Commit Graph

31 Commits

Author SHA1 Message Date
girish@cloudron.io
bcb0e61bfc Kill child processes
On Unix, child processes are not killed when parent dies.

Each process is part of a process group (pgid). When pgid == pid,
it is the process group leader.

node creates child processes with the parent as the group leader
(detached = false).

You can send a signal to entire group using kill(-pgid), as in,
negative value in argument. Systemd can be made to do this by
setting the KillMode=control-group.

Unrelated: Process groups reside inside session groups. Each session
group has a controlling terminal. Only one process in the session
group has access to the terminal. Process group is basically like
a bash pipeline. A session group is the entire login session with only
one process having terminal access at a time.

Fixes #543
2016-01-21 17:44:17 -08:00
Girish Ramakrishnan
87f5e3f102 workaround journalctl logging bug 2015-12-22 13:05:00 -08:00
Girish Ramakrishnan
f32027e15b Try alternative configuration for systemd restart rate limit 2015-12-12 13:15:41 -08:00
Girish Ramakrishnan
4b6a92955b configure to get only 1 email every 10 minutes 2015-12-12 11:47:32 -08:00
Girish Ramakrishnan
5d389337cd make /var/log readonly
Expect apps to redirect logs of stdout/stderr

Part of #503
2015-10-15 00:46:50 -07:00
Girish Ramakrishnan
a977597217 cleanup tmpdir in janitor 2015-10-14 23:21:03 -07:00
Girish Ramakrishnan
7f29eed326 fold janitor into main box code cron job
the volume cleaner will now also come into janitor
2015-10-14 22:39:34 -07:00
Girish Ramakrishnan
ec895a4f31 do not use -f to logrotate
Normally, logrotate is run as a daily cron job. It will not modify a log
multiple times in one day unless the criterion for that log is based on
the log's size and logrotate is being run multiple times each day, or
unless the -f or --force option is used.
2015-10-14 15:10:53 -07:00
Girish Ramakrishnan
3fc0a96bb0 Add docker volumes janitor
This cleans up tmp and logrotates /var/log every 12 hours.

Note that this janitor is separate from the box janitor because they
run as different users.

Fixes #503
2015-10-14 14:18:36 -07:00
Girish Ramakrishnan
185b574bdc Add custom apparmor profile for cloudron apps
Docker generates an apparmor profile on the fly under /etc/apparmor.d/docker.
This profile gets overwritten on every docker daemon start.

This profile allows processes to ptrace themselves. This is required by
circus (python process manager) for reasons unknown to me. It floods the logs
with
    audit[7623]: <audit-1400> apparmor="DENIED" operation="ptrace" profile="docker-default" pid=7623 comm="python3.4" requested_mask="trace" denied_mask="trace" peer="docker-default"

This is easily tested using:
    docker run -it cloudron/base:0.3.3 /bin/bash
        a) now do ps
        b) journalctl should show error log as above

    docker run --security-opt=apparmor:docker-cloudron-app -it cloudron/base:0.3.3 /bin/bash
        a) now do ps
        b) no error!

Note that despite this, the process may not have ability to ptrace since it does not
have CAP_PTRACE. Also, security-opt is the profile name (inside the apparmor config file)
and not the filename.

References:
    https://groups.google.com/forum/#!topic/docker-user/xvxpaceTCyw
    https://github.com/docker/docker/issues/7276
    https://bugs.launchpad.net/ubuntu/+source/docker.io/+bug/1320869

This is an infra update because we need to recreate containers to get the right profile.

Fixes #492
2015-09-21 11:01:44 -07:00
Girish Ramakrishnan
8aff2b9e74 remove oauthproxy systemd configs 2015-09-14 12:02:38 -07:00
Girish Ramakrishnan
1cd9d07d8c Merge apphealthtask into box server
We used to run this as a separate process but no amount of node/v8 tweaking
makes them run as standalone with 50M RSS.

Three solutions were considered for the memory issue:
1. Use systemd timer. apphealthtask needs to run quiet frequently (10 sec)
   for the ui to get the app health update immediately after install.

2. Merge into box server (this commit)

3. Increase memory to 80M. This seems to make apphealthtask run as-is.
2015-09-14 10:52:11 -07:00
Girish Ramakrishnan
f028649582 Rename app.js to box.js 2015-09-14 10:43:47 -07:00
Girish Ramakrishnan
29e05b1caa make janitor a systemd timer
one process lesser
2015-09-11 18:43:51 -07:00
Girish Ramakrishnan
6945a712df limit node memory usage
node needs to be told how much space it can usage, otherwise it keeps
allocating and we cannot keep it under 50M. keeping old space to 30M,
lets the memory hover around 40M

there are many options to v8 but I haven't explored them all:
--expose_gc - allows scripts to call gc()
--max_old_space_size=30 --max_semi_space_size=2048 (old/new space)
    node first allocates new objects in new space. if these objects are in use
    around for some time, it moves them to old space. the idea here is that it
    runs gc aggressively on new space since new objects die more than old ones.

    the new space is split into two halves of equal size called semi spaces.

--gc_interval=100 --optimize_for_size --max_executable_size=5 --gc_global --stack_size=1024

http://erikcorry.blogspot.com/2012/11/memory-management-flags-in-v8.html
http://jayconrod.com/posts/55/a-tour-of-v8-garbage-collection
https://code.google.com/p/chromium/issues/detail?id=280984
http://stackoverflow.com/questions/30252905/nodejs-decrease-v8-garbage-collector-memory-usage
http://www.appfruits.com/2014/08/running-node-js-on-arduino-yun/

note: this is not part of shebang because linux shebang does not support args! so we cannot
pass node args as part of shebang.
2015-09-10 21:24:36 -07:00
Girish Ramakrishnan
03048d7d2f set memorylimit for crashnotifier as well 2015-09-10 14:19:44 -07:00
Girish Ramakrishnan
26aefadfba systemd: fix crashnotifier 2015-09-07 21:40:01 -07:00
Girish Ramakrishnan
51a28842cf systemd: pass the instance name as argument 2015-09-07 21:16:22 -07:00
Girish Ramakrishnan
773c326eb7 systemd: just wait for 5 seconds for box to die 2015-09-07 20:58:14 -07:00
Girish Ramakrishnan
cb2fb026c5 systemd: do not restart crashnotifier 2015-09-07 20:54:58 -07:00
Girish Ramakrishnan
a4731ad054 200m is a more sane memory limit 2015-09-07 20:48:29 -07:00
Girish Ramakrishnan
aa33938fb5 systemd: fix config files 2015-09-07 20:46:32 -07:00
Girish Ramakrishnan
2a4c467ab8 systemd: Fix crashnotifier 2015-09-07 20:14:37 -07:00
Girish Ramakrishnan
6be6092c0e Add memory limits on services 2015-09-07 19:16:34 -07:00
Girish Ramakrishnan
e76584b0da Move from supervisor to systemd
This removes logrotate as well since we use systemd logging
2015-09-07 14:31:25 -07:00
Girish Ramakrishnan
10df0a527f Fix typo
remove thead_cache_size. it's dynamic anyways
2015-08-13 14:53:05 -07:00
Girish Ramakrishnan
e78dbcb5d4 limit threads and max connections 2015-08-13 14:42:36 -07:00
Girish Ramakrishnan
81b7432044 Turn off performance_schema in mysql 5.6 2015-08-13 13:47:44 -07:00
Girish Ramakrishnan
c4c463cbc2 collect logs using a sudo script
docker logs can only be read by root
2015-08-08 19:04:59 -07:00
Girish Ramakrishnan
ef9508ccc5 Use BOX_ENV instead of NODE_ENV
Let NODE_ENV be used by node modules and always be set to production

Fixes #453
2015-07-24 01:42:28 -07:00
Girish Ramakrishnan
df9d321ac3 app.portBindings and newManifest.tcpPorts may be null 2015-07-20 00:10:36 -07:00