diff --git a/.gitignore b/.gitignore index 3556d45db..99a44158f 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,5 @@ node_modules/ coverage/ -docs/ webadmin/dist/ setup/splash/website/ installer/src/certs/server.key diff --git a/docs/img/activity.png b/docs/img/activity.png new file mode 100644 index 000000000..e49823284 Binary files /dev/null and b/docs/img/activity.png differ diff --git a/docs/img/app_access_control.png b/docs/img/app_access_control.png new file mode 100644 index 000000000..dfb9f0517 Binary files /dev/null and b/docs/img/app_access_control.png differ diff --git a/docs/img/app_configure.png b/docs/img/app_configure.png new file mode 100644 index 000000000..55363edc8 Binary files /dev/null and b/docs/img/app_configure.png differ diff --git a/docs/img/app_external_domain.png b/docs/img/app_external_domain.png new file mode 100644 index 000000000..67b84346a Binary files /dev/null and b/docs/img/app_external_domain.png differ diff --git a/docs/img/app_info.png b/docs/img/app_info.png new file mode 100644 index 000000000..773dad4a5 Binary files /dev/null and b/docs/img/app_info.png differ diff --git a/docs/img/app_install.png b/docs/img/app_install.png new file mode 100644 index 000000000..b4c6675ac Binary files /dev/null and b/docs/img/app_install.png differ diff --git a/docs/img/app_single_user.png b/docs/img/app_single_user.png new file mode 100644 index 000000000..30b70ab72 Binary files /dev/null and b/docs/img/app_single_user.png differ diff --git a/docs/img/custom_domain_change.png b/docs/img/custom_domain_change.png new file mode 100644 index 000000000..b4865cce1 Binary files /dev/null and b/docs/img/custom_domain_change.png differ diff --git a/docs/img/custom_domain_menu.png b/docs/img/custom_domain_menu.png new file mode 100644 index 000000000..9320c6672 Binary files /dev/null and b/docs/img/custom_domain_menu.png differ diff --git a/docs/img/email_alias.png b/docs/img/email_alias.png new file mode 100644 index 000000000..20bb17bd1 Binary files /dev/null and b/docs/img/email_alias.png differ diff --git a/docs/img/graphs.png b/docs/img/graphs.png new file mode 100644 index 000000000..27c267f70 Binary files /dev/null and b/docs/img/graphs.png differ diff --git a/docs/img/groups.png b/docs/img/groups.png new file mode 100644 index 000000000..aeccf11d8 Binary files /dev/null and b/docs/img/groups.png differ diff --git a/docs/img/users.png b/docs/img/users.png new file mode 100644 index 000000000..403298302 Binary files /dev/null and b/docs/img/users.png differ diff --git a/docs/img/webadmin_domain.png b/docs/img/webadmin_domain.png new file mode 100644 index 000000000..13e93e582 Binary files /dev/null and b/docs/img/webadmin_domain.png differ diff --git a/docs/makeDocs b/docs/makeDocs deleted file mode 100755 index 3326ee29a..000000000 --- a/docs/makeDocs +++ /dev/null @@ -1,5 +0,0 @@ -#!/bin/sh - -set -eu - -./node_modules/.bin/apidoc -i src/routes -o docs diff --git a/docs/references/addons.md b/docs/references/addons.md new file mode 100644 index 000000000..57ea56157 --- /dev/null +++ b/docs/references/addons.md @@ -0,0 +1,386 @@ +# Addons + +## Overview + +Addons are services like database, authentication, email, caching that are part of the +Cloudron runtime. Setup, provisioning, scaling and maintanence of addons is taken care of +by the runtime. + +The fundamental idea behind addons is to allow sharing of Cloudron resources across applications. +For example, a single MySQL server instance can be used across multiple apps. The Cloudron +runtime sets up addons in such a way that apps are isolated from each other. + +## Using Addons + +Addons are opt-in and must be specified in the [Cloudron Manifest](/references/manifest.html). +When the app runs, environment variables contain the necessary information to access the addon. +For example, the mysql addon sets the `MYSQL_URL` environment variable which is the +connection string that can be used to connect to the database. + +When working with addons, developers need to remember the following: +* Environment variables are subject to change every time the app restarts. This can happen if the +Cloudron is rebooted or restored or the app crashes or an addon is re-provisioned. For this reason, +applications must never cache the value of environment variables across restarts. + +* Addons must be setup or updated on each application start up. Most applications use DB migration frameworks +for this purpose to setup and update the DB schema. + +* Addons are configured in the [addons section](/references/manifest.html#addons) of the manifest as below: +``` + { + ... + "addons": { + "oauth": { }, + "redis" : { } + } + } +``` + +## All addons + +### email + +This addon allows an app to send and recieve emails on behalf of the user. The intended use case is webmail applications. + +If an app wants to send mail (e.g notifications), it must use the [sendmail](/references/addons#sendmail) +addon. If the app wants to receive email (e.g user replying to notification), it must use the +[recvmail](/references/addons#recvmail) addon instead. + +Apps using the IMAP and ManageSieve services below must be prepared to accept self-signed certificates (this is not a problem +because these are addresses internal to the Cloudron). + +Exported environment variables: +``` +MAIL_SMTP_SERVER= # SMTP server IP or hostname. Supports STARTTLS (TLS upgrade is enforced). +MAIL_SMTP_PORT= # SMTP server port +MAIL_IMAP_SERVER= # IMAP server IP or hostname. TLS required. +MAIL_IMAP_PORT= # IMAP server port +MAIL_SIEVE_SERVER= # ManageSieve server IP or hostname. TLS required. +MAIL_SIEVE_PORT= # ManageSieve server port +MAIL_DOMAIN= # Domain of the mail server +``` + +### ldap + +This addon provides LDAP based authentication via LDAP version 3. + +Exported environment variables: +``` +LDAP_SERVER= # ldap server IP +LDAP_PORT= # ldap server port +LDAP_URL= # ldap url of the form ldap://ip:port +LDAP_USERS_BASE_DN= # ldap users base dn of the form ou=users,dc=cloudron +LDAP_GROUPS_BASE_DN= # ldap groups base dn of the form ou=groups,dc=cloudron +LDAP_BIND_DN= # DN to perform LDAP requests +LDAP_BIND_PASSWORD= # Password to perform LDAP requests +``` + +For debugging, [cloudron exec](https://www.npmjs.com/package/cloudron) can be used to run the `ldapsearch` client within the context of the app: +``` +cloudron exec + +# list users +> ldapsearch -x -h "${LDAP_SERVER}" -p "${LDAP_PORT}" -b "${LDAP_USERS_BASE_DN}" + +# list users with authentication (Substitute username and password below) +> ldapsearch -x -D cn=,${LDAP_USERS_BASE_DN} -w -h "${LDAP_SERVER}" -p "${LDAP_PORT}" -b "${LDAP_USERS_BASE_DN}" + +# list admins +> ldapsearch -x -h "${LDAP_SERVER}" -p "${LDAP_PORT}" -b "${LDAP_USERS_BASE_DN}" "memberof=cn=admins,${LDAP_GROUPS_BASE_DN}" + +# list groups +> ldapsearch -x -h "${LDAP_SERVER}" -p "${LDAP_PORT}" -b "${LDAP_GROUPS_BASE_DN}" +``` + +### localstorage + +Since all Cloudron apps run within a read-only filesystem, this addon provides a writeable folder under `/app/data/`. +All contents in that folder are included in the backup. On first run, this folder will be empty. File added in this path +as part of the app's image (Dockerfile) won't be present. A common pattern is to create the directory structure required +the app as part of the app's startup script. + +The permissions and ownership of data within that directory are not guranteed to be preserved. For this reason, each app +has to restore permissions as required by the app as part of the app's startup script. + +If the app is running under the recommeneded `cloudron` user, this can be achieved with: +``` +chown -R cloudron:cloudron /app/data +``` + +### mongodb + +By default, this addon provide mongodb 2.6.3. + +Exported environment variables: +``` +MONGODB_URL= # mongodb url +MONGODB_USERNAME= # username +MONGODB_PASSWORD= # password +MONGODB_HOST= # server IP/hostname +MONGODB_PORT= # server port +MONGODB_DATABASE= # database name +``` + +For debugging, [cloudron exec](https://www.npmjs.com/package/cloudron) can be used to run the `mongo` shell within the context of the app: +``` +cloudron exec + +# mongo -u "${MONGODB_USERNAME}" -p "${MONGODB_PASSWORD}" ${MONGODB_HOST}:${MONGODB_PORT}/${MONGODB_DATABASE} + +``` +### mysql + +By default, this addon provides a single database on MySQL 5.6.19. The database is already created and the application +only needs to create the tables. + +Exported environment variables: +``` +MYSQL_URL= # the mysql url (only set when using a single database, see below) +MYSQL_USERNAME= # username +MYSQL_PASSWORD= # password +MYSQL_HOST= # server IP/hostname +MYSQL_PORT= # server port +MYSQL_DATABASE= # database name (only set when using a single database, see below) +``` + +For debugging, [cloudron exec](https://www.npmjs.com/package/cloudron) can be used to run the `mysql` client within the context of the app: +``` +cloudron exec + +> mysql --user=${MYSQL_USERNAME} --password=${MYSQL_PASSWORD} --host=${MYSQL_HOST} ${MYSQL_DATABASE} + +``` + +The `multipleDatabases` option can be set to `true` if the app requires more than one database. When enabled, +the following environment variables are injected: + +``` +MYSQL_DATABASE_PREFIX= # prefix to use to create databases +``` + +### oauth + +The Cloudron OAuth 2.0 provider can be used in an app to implement Single Sign-On. + +Exported environment variables: +``` +OAUTH_CLIENT_ID= # client id +OAUTH_CLIENT_SECRET= # client secret +``` + +The callback url required for the OAuth transaction can be contructed from the environment variables below: + +``` +APP_DOMAIN= # hostname of the app +APP_ORIGIN= # origin of the app of the form https://domain +API_ORIGIN= # origin of the OAuth provider of the form https://my-cloudrondomain +``` + +OAuth2 URLs can be constructed as follows: + +``` +AuthorizationURL = ${API_ORIGIN}/api/v1/oauth/dialog/authorize # see above for API_ORIGIN +TokenURL = ${API_ORIGIN}/api/v1/oauth/token +``` + +The token obtained via OAuth has a restricted scope wherein they can only access the [profile API](/references/api.html#profile). This restriction +is so that apps cannot make undesired changes to the user's Cloudron. + +We currently provide OAuth2 integration for Ruby [omniauth](https://github.com/cloudron-io/omniauth-cloudron) and Node.js [passport](https://github.com/cloudron-io/passport-cloudron). + +### postgresql + +By default, this addon provides PostgreSQL 9.4.4. + +Exported environment variables: +``` +POSTGRESQL_URL= # the postgresql url +POSTGRESQL_USERNAME= # username +POSTGRESQL_PASSWORD= # password +POSTGRESQL_HOST= # server name +POSTGRESQL_PORT= # server port +POSTGRESQL_DATABASE= # database name +``` + +The postgresql addon whitelists the hstore and pg_trgm extensions to be installable by the database owner. + +For debugging, [cloudron exec](https://www.npmjs.com/package/cloudron) can be used to run the `psql` client within the context of the app: +``` +cloudron exec + +> PGPASSWORD=${POSTGRESQL_PASSWORD} psql -h ${POSTGRESQL_HOST} -p ${POSTGRESQL_PORT} -U ${POSTGRESQL_USERNAME} -d ${POSTGRESQL_DATABASE} +``` + +### recvmail + +The recvmail addon can be used to receive email for the application. + +Exported environment variables: +``` +MAIL_IMAP_SERVER= # the IMAP server. this can be an IP or DNS name +MAIL_IMAP_PORT= # the IMAP server port +MAIL_IMAP_USERNAME= # the username to use for authentication +MAIL_IMAP_PASSWORD= # the password to use for authentication +MAIL_TO= # the to address to use +MAIL_DOMAIN= # the mail for which email will be received +``` + +The IMAP server only accepts TLS connections. The app must be prepared to accept self-signed certs (this is not a problem because the +imap address is internal to the Cloudron). + +For debugging, [cloudron exec](https://www.npmjs.com/package/cloudron) can be used to run the `openssl` tool within the context of the app: +``` +cloudron exec + +> openssl s_client -connect "${MAIL_IMAP_SERVER}:${MAIL_IMAP_PORT}" -crlf +``` + +The IMAP command `? LOGIN username password` can then be used to test the authentication. + +### redis + +By default, this addon provides redis 2.8.13. The redis is configured to be persistent and data is preserved across updates +and restarts. + +Exported environment variables: +``` +REDIS_URL= # the redis url +REDIS_HOST= # server name +REDIS_PORT= # server port +REDIS_PASSWORD= # password +``` + +For debugging, [cloudron exec](https://www.npmjs.com/package/cloudron) can be used to run the `redis-cli` client within the context of the app: +``` +cloudron exec + +> redis-cli -h "${REDIS_HOST}" -p "${REDIS_PORT}" -a "${REDIS_PASSWORD}" +``` + +### scheduler + +The scheduler addon can be used to run tasks at periodic intervals (cron). + +Scheduler can be configured as below: +``` + "scheduler": { + "update_feeds": { + "schedule": "*/5 * * * *", + "command": "/app/code/update_feed.sh" + } + } +``` + +In the above example, `update_feeds` is the name of the task and is an arbitrary string. + +`schedule` values must fall within the following ranges: + + * Minutes: 0-59 + * Hours: 0-23 + * Day of Month: 1-31 + * Months: 0-11 + * Day of Week: 0-6 + +_NOTE_: scheduler does not support seconds + +`schedule` supports ranges (like standard cron): + + * Asterisk. E.g. * + * Ranges. E.g. 1-3,5 + * Steps. E.g. */2 + +`command` is executed through a shell (sh -c). The command runs in the same launch environment +as the application. Environment variables, volumes (`/tmp` and `/run`) are all +shared with the main application. + +If a task is still running when a new instance of the task is scheduled to be started, the previous +task instance is killed. + + +### sendmail + +The sendmail addon can be used to send email from the application. + +Exported environment variables: +``` +MAIL_SMTP_SERVER= # the mail server (relay) that apps can use. this can be an IP or DNS name +MAIL_SMTP_PORT= # the mail server port +MAIL_SMTP_USERNAME= # the username to use for authentication as well as the `from` username when sending emails +MAIL_SMTP_PASSWORD= # the password to use for authentication +MAIL_FROM= # the from address to use +MAIL_DOMAIN= # the domain name to use for email sending (i.e username@domain) +``` + +The SMTP server does not require STARTTLS. If STARTTLS is used, the app must be prepared to accept self-signed certs. + +For debugging, [cloudron exec](https://www.npmjs.com/package/cloudron) can be used to run the `swaks` tool within the context of the app: +``` +cloudron exec + +> swaks --server "${MAIL_SMTP_SERVER}" -p "${MAIL_SMTP_PORT}" --from "${MAIL_SMTP_USERNAME}@${MAIL_DOMAIN}" --body "Test mail from cloudron app at $(hostname -f)" --auth-user "${MAIL_SMTP_USERNAME}" --auth-password "${MAIL_SMTP_PASSWORD}" +``` + +### simpleauth + +Simple Auth can be used for authenticating users with a HTTP request. This method of authentication is targeted +at applications, which for whatever reason can't use the ldap addon. +The response contains an `accessToken` which can then be used to access the [Cloudron API](/references/api.html). + +Exported environment variables: +``` +SIMPLE_AUTH_SERVER= # the simple auth HTTP server +SIMPLE_AUTH_PORT= # the simple auth server port +SIMPLE_AUTH_URL= # the simple auth server URL. same as "http://SIMPLE_AUTH_SERVER:SIMPLE_AUTH_PORT +SIMPLE_AUTH_CLIENT_ID # a client id for identifying the request originator with the auth server +``` + +This addons provides two REST APIs: + +**POST /api/v1/login** + +Request JSON body: +``` +{ + "username": " or ", + "password": "" +} +``` + +Response 200 with JSON body: +``` +{ + "accessToken": "", + "user": { + "id": "", + "username": "", + "email": "", + "admin": , + "displayName": "" + } +} +``` + +**GET /api/v1/logout** + +Request params: +``` +?access_token= +``` + +Response 200 with JSON body: +``` +{} +``` + +For debugging, [cloudron exec](https://www.npmjs.com/package/cloudron) can be used to run the `curl` tool within the context of the app: +``` +cloudron exec + +> USERNAME= + +> PASSWORD= + +> PAYLOAD="{\"clientId\":\"${SIMPLE_AUTH_CLIENT_ID}\", \"username\":\"${USERNAME}\", \"password\":\"${PASSWORD}\"}" + +> curl -H "Content-Type: application/json" -X POST -d "${PAYLOAD}" "${SIMPLE_AUTH_ORIGIN}/api/v1/login" +``` diff --git a/docs/references/api.md b/docs/references/api.md new file mode 100644 index 000000000..6d901624d --- /dev/null +++ b/docs/references/api.md @@ -0,0 +1,1458 @@ +# Overview + +The Cloudron provides a RESTful API to manage all aspects of the Cloudron like +adding users, configuring groups and installing apps. + +If you are an app developer, the [Cloudron CLI tool](https://www.npmjs.com/package/cloudron) implements a workflow that allows +you to develop apps on your Cloudron. The CLI tool uses the REST API documented here. + +## API Origin + +The Cloudron API is available at the `my` subdomain of your Cloudron. + +For example, if the Cloudron is on a custom domain, then this would be `https://my.customdomain.com`. If the +Cloudron is on a `cloudron.me` subdomain, then this would be `https://my-demo.cloudron.me`. + +# Authentication + +## Getting API Tokens + +POST `/api/v1/developer/login` admin + +Creates a token given user credentials. + +Request: +``` +{ + username: , + password: +} +``` + +Set `username` to your username and `password` to your password. Currently, only Cloudron administrators can +create API tokens. + +Response (200): +``` +{ + token: , // Token used for accessing the API + expiresAt: // ISO-8601 UTC date the token expires +} +``` + +Curl example: +``` +curl -X POST -H "Content-Type: application/json" -d '{"username": "cloudron", "password":"cloudron"}' https://my-demo.cloudron.me/api/v1/developer/login +``` + +## Using API tokens + +The access token can either be provided via the request query `?access_token=`. + +``` +curl -H "Content-Type: application/json" https://$CLOUDRON_ORIGIN/api/v1/users?access_token=$TOKEN +``` + +Alternately, the token can be provided via the `Authorization` header using `Bearer `. + +``` +curl -H "Content-Type: application/json" -H "Authorization: Bearer " https://$CLOUDRON_ORIGIN/api/v1/users +``` + +## OAuth + +OAuth authentication is meant to be used by apps. An app can get an OAuth token using the +[oauth](addons.html#oauth) or [simpleauth](addons.html#simpleauth) addon. + +Tokens obtained via OAuth have a restricted scope wherein they can only access the user's profile. +This restriction is so that apps cannot make undesired changes to the user's Cloudron. + +The access token can be provided either via the request query `?access_token=` or in the +`Authorization` header using `Bearer `. + +# REST + +## Requests + +All requests must be made via `https` protocol to ensure that the connection is encrypted. + +The general idea behind HTTP methods is: +* Use `GET` to retrieve resource information +* Use `DELETE` to destroy a resource +* Use `PUT` to update an existing resource +* Use `POST` to create a new resource + +## Error + +Error response objects have a `status` field indicating the HTTP error and a `message` field containing +a detailed description of the error. + +## Pagination + +APIs that list objects take a `page` query parameters to indicate the page number starting from index 1. +The `per_page` query parameter can be used to specify the number of items to be returned. + +# Endpoints + +## Apps + +### Install app + +POST `/api/v1/apps/install` admin + +Request: + +``` +{ + location: , // required: the subdomain on which to install the app + appStoreId: [@], // required: Cloudron Store Id of the app. Alternately, provide a manifest + manifest: , // manifest of the app to install. required if appStoreId is unset. + portBindings: null || { // mapping from application ports to public ports + }, + accessRestriction: null || { // required. list of users and groups who can access this application + users: [ ], + groups: [ ] + }, + icon: , // icon as base64 encoded string + cert: , // pem encoded TLS cert + key: , // pem encoded TLS key + memoryLimit: , // memory constraint in bytes + altDomain: // alternate domain from which this app can be reached +} +``` + +`appStoreId` is the [Cloudron Store](https://cloudron.io/appstore.html) Id of this application. For example, +`io.gogs.cloudronapp` is the id of Gogs app. A specific version can be specified using the '@' suffix. For +apps that are not published on the Cloudron Store, skip this field and provide a `manifest` instead. Apps +with an `appStoreId` will automatically be kept up-to-date as newer version of the app are published on the +store. + +`manifest` is the [manifest](https://cloudron.io/references/manifest.html) of the app to be installed. This +is only required if `appStoreId` is not provided. Apps with a manifest won't receive automatic updates. + +`location` is the subdomain on which the app is installed. This can be empty if the app was installed on the naked domain. +If another app already exists in the same location, 409 is returned. + +For apps that require login, `accessRestriction` is the *restricted* list of users and groups that can access this app. If null, all of this Cloudron users can access this app. + +`icon` is an application icon that is displayed in the web ui. If not provided, this is automatically downloaded +from the Cloudron Store (or uses a fallback icon). + +`cert` and `key` provide the TLS certificates. If the domain name of the app does not must match with the certificate +provided, a 400 will be returned. + +If `altDomain` is set, the app can be accessed from `https://`. + +Response (200): + +``` +{ + id: , // a unique id for the app +} +``` + +On success, the installation progress can be tracked by polling [installationProgress](/references/api.html#get-app). + +Curl example to install Gogs app at subdomain git-demo.cloudron.me: +``` +curl -X POST -d '{ "appStoreId": "io.gogs.cloudronapp", "location": "git", "accessRestriction": null }' -H 'Authorization: Bearer f34eb4d0d942c8f8b3c060f356f1bb6961bc07bfb3fa2b24188a240f3de975f5' https://my-demo.cloudron.me/api/v1/apps/install +``` + +Curl example to install specific version of Gogs app with SSH Port exposed at 6000: +``` +curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer 27ab70cfd10e615ec29f6d890947e2b72db47522754bfafcad6f9c0e6c9e84e9" -d '{ "appStoreId": "io.gogs.cloudronapp@0.12.6", "portBindings": { "SSH_PORT": 6000 }, "location": "git2", "accessRestriction": null }' https://my-donut.cloudron.eu/api/v1/apps/install +``` + +To restrict access to Gogs app to the *developers* group: +``` +curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer 27ab70cfd10e615ec29f6d890947e2b72db47522754bfafcad6f9c0e6c9e84e9" -d '{ "appStoreId": "io.gogs.cloudronapp@0.12.6", "location": "git3", "accessRestriction": { "groups": [ "developers" ] } }' https://my-demo.cloudron.me/api/v1/apps/install +``` + +### Get app + +GET `/api/v1/apps/:appId` admin + +Gets information about the app with id `appId`. + +Response (200): + +``` +{ + id: , // a unique id for the app + appStoreId: , // Cloudron Store Id for updates + manifest: , // current manifest of the app + installationState: , // See below + installationProgress: , // friendly string indicating installation progress + runState: , // see below + health: , // health of the application + location: , // subdomain on which app is installed + fqdn: , // the FQDN of this app + altDomain: // alternate domain from which this app can be reached + accessRestriction: null || { // list of users and groups who can access this application + users: [ ], + groups: [ ] + }, + lastBackupId: , // last known backup of this app + manifest: , // the application manifest + portBindings: { // mapping from application ports to public ports + }, + iconUrl: , // a relative url providing the icon + memoryLimit: // memory constraint in bytes +} +``` + +`id` is an unique id for this application. + +`appStoreId` is the Cloudron Store id of this application. Cloudron will use this id to look for updates to this application. This can be null if none was provided at installation time. + +`manifest` is the [Cloudron Manifest](/references/manifest.html) of the app. + +`installationState` is one of the values below: +* `pending_install` - The app is being installed. Use the `installationProgress` field to track the progress. +* `pending_configure` - The app is being re-configured. For example, if the app was moved to a new location or the port bindings was changed. +* `pending_uinstall` - The app is being uninstalled. +* `pending_restore` - The app is being restored from a previous backup. +* `pending_update` - The app is being updated. +* `pending_force_update` - The app is being force-updated. +* `pending_backup` - The app is being backed up. +* `error` - There was an error executing one of the above pending commands. +* `installed` - The app is installed. Use the `runState` and `health` to determine if the app is running and healthy. + +`installationProgress` is a string indicating the progress when the app's `installationState` is one of the `pending_*` states. It is +of the format `, `. + +`runState` is one of the values below: +* `pending_start` - The app is being started. +* `pending_stop` - The app is being stopped. +* `stopped` - The app was stopped. +* `running` - The app is running. + +`health` is one of the values below: +* `healthy` - The app is responding to health checks and is healthy. +* `unhealthy` - The app is not responding to health checks. +* `error` - There was an error checking the health of the app. +* `dead` - The app is dead. Most likely it was stopped or being uninstalled. + +`location` is the subdomain on which the app is installed. This can be empty if the app was installed on the naked domain. The app can be +accessed from `fqdn` i.e `https//`. If `altDomain` is set, the app should be accessed from `https://`. + +For apps that require login, `accessRestriction` is the *restricted* list of users and groups that can access this app. If null, all of this Cloudron users can access this app. + +`lastBackupId` is the last valid backup id. The [restore API](/references/api.html#restore-app) can be used to restore the app to this backup. + +`manifest` is the [application manifest](/references/manifest.html). + +### List apps + +GET `/api/v1/apps/:appId` admin + +Gets the list of installed apps. + +Response (200): + +``` +{ + apps: [ + { + id: , // a unique id for the app + appStoreId: , // Cloudron Store Id for updates + manifest: , // current manifest of the app + installationState: , // See below + installationProgress: , // friendly string indicating installation progress + runState: , // see below + health: , // health of the application + location: , // subdomain on which app is installed + fqdn: , // the FQDN of this app + altDomain: // alternate domain from which this app can be reached + accessRestriction: null || { // list of users and groups who can access this application + users: [ ], + groups: [ ] + }, + lastBackupId: , // last known backup of this app + manifest: , // the application manifest + portBindings: { // mapping for application ports to public ports + }, + iconUrl: , // a relative url providing the icon + memoryLimit: // memory constraint in bytes + }, + ... + ] +} +``` + +### Get icon + +GET `/api/v1/apps/:appId/icon` admin + +Gets the icon of the application with id `appId` as `image/png`. + +The icon is used in the display at Cloudron admin UI. + +### Backup app + +POST `/api/v1/apps/:appId/backup` admin + +Starts a backup of the application with id `appId`. + +The backup progress can be tracked by polling the value of [installationProgress](/references/api.html#get-app). + +### List app backups + +GET `/api/v1/apps/:appId/backups` admin + +Gets the backups of the application with id `appId`. + +Use the [Backup](/references/api.html#download-backup) API to download the backup. + +Response (200): + +``` +{ + backups: [ + { + id: , // a unique id for this backup + creationTime: , // ISO-8601 date in UTC + version: , // the app version + type: , // 'app' + dependsOn: [ , ... ], // always an empty array for app backups + state: // 'normal' + }, + ... + ] +} +``` + +### Restore app + +POST `/api/v1/apps/:appId/restore` admin + +Restores the app from a backup. + +Request: + +``` +{ + backupId: null || +} +``` + +`backupId` is an id from the [list app backups](/references/api.html#list-app-backups) API. + +Note that app backups are tied to the app's version (see the `version` field of the backup). So, restoring +an app may result in reverting the app to a previous version. + +Setting backupId to `null` has the same effect as reinstalling the application. + +### Clone app + +POST `/api/v1/apps/:appId/clone` admin + +Clones the app from a backup. + +Request: + +``` +{ + backupId: , // required. the cloned app will start with this data. + location: , // required. the subdomain for the cloned app + portBindings: null || { // required. mapping from application ports to public ports + } +} +``` + +`backupId` is an id from the [list app backups](/references/api.html#list-app-backups) API. + +`location` is a subdomain for the cloned app and will result in a 409 in case of a conflict. + +`portBindings` is a list of new tcp port mappings for the cloned app. + +Response(201): + +``` + { + id: // app id of the new app + } +``` + +The clone progress can be tracked by polling the value of [installationProgress](/references/api.html#get-app). +Be sure to use the `id` of the new app returned above and not the original app's id. + +### Get logs + +GET `/api/v1/apps/:appId/logs` admin + +Get the logs of an application with id `appId`. + +The `lines` query parameter can be used to specify the number of log lines to download. + +The response has `Content-Type` set to 'application/x-logs' and `Content-Disposition` set to +`attachment; filename="log.txt`. + +Response(200): + +``` +Line delimited JSON. + + { + realtimeTimestamp: , // wallclock timestamp + monotonicTimestamp: , // time passed since boot + message: [ ,... ], // utf8 buffer + source: // source of this message + } +``` + +Logs are aggregated from one or more `source`s. The application logs have the `source` set to `main`. +Other sources include `scheduler`. + +### Get log stream + +GET `/api/v1/apps/:appId/logstream` admin + +Stream the logs of an application with id `appId` as a `text/event-stream`. See the [EventSource](https://developer.mozilla.org/en-US/docs/Web/API/EventSource) +interface documentation for details. + +The `lines` query parameter can be used to specify the number of log lines to prefetch. + +Response(200): + +``` +Line delimited JSON + + { + realtimeTimestamp: , // wallclock timestamp + monotonicTimestamp: , // time passed since boot + message: [ ,... ], // utf8 buffer + source: // source of this message + } +``` +Logs are aggregated from one or more `source`s. The application logs have the `source` set to `main`. +Other sources include `scheduler`. + +### Configure app + +POST `/api/v1/apps/:appId/configure` admin + +Re-configures an existing app with id `appId`. + +Configuring an app won't preserve existing data. Cloudron apps are written in a way to support reconfiguring +any of the parameters listed below without loss of data. + +Request: +``` + location: , // the subdomain on which to install the app + portBindings: null || { // mapping from application ports to public ports + }, + accessRestriction: null || { // required. list of users and groups who can access this application + users: [ ], + groups: [ ] + }, + icon: , // icon as base64 encoded string + cert: , // pem encoded TLS cert + key: , // pem encoded TLS key + memoryLimit: , // memory constraint in bytes + altDomain: // alternate domain from which this app can be reached +``` + +All values are optional. See [Install app](/references/api.html#install-app) API for field descriptions. + +### Update app + +POST `/api/v1/apps/:appId/update` admin + +Updates an app with id `appId`. + +Updating an app updrades (or downgrades) the app preserving existing data. To be safe, the update process +makes a backup of existing app data first before updating the app. This allow you to +[restore the app](/references/api.html#restore-app) should the upgrade fail. + +Only apps that are installed, running and responding to health checks can generate a consistent back up. +For this reason, it is not possible to update apps that are in any other state. To override this, use +the `force` parameter described below. + +Request: + +``` +{ + appStoreId: [@], // the new version of the app + manifest: , // the manifest of the app + portBindings: { // optional + }, + icon: , // optional + force: // optional. default: false +} +``` + +`appStoreId` is the id of the app to install from the Cloudron Store. The API does not verify that +the version provided here is greater than the existing app version allowing you to downgrade apps. +Downgrading should be used with extreme care because the older version of the app may not understand +the format of existing data (for example, new db schema may not be understod by the older version). + +`manifest` provides information of the new app version that the app needs to be updated to. This is +only required if appStoreId was not provided at installation time. + +If the new version of the app requires new ports to be allocated for the app, then mapping for the new ports +can be provided in `portBindings`. + +`icon` specifies any new icon as a base64 encoded string for the updated version of the app. + +The Cloudron will only update apps that are installed, running and responding to health checks. Before +each update, the app is backed up so that it may be restored easily in case of a bad update. + +`force` can be used to force an update even if the app is in a state which prevents an update. This is +useful during app development, where you can force a crashed app to update to the latest code. + +The update progress can be tracked by polling the value of [installationProgress](/references/api.html#get-app). + +Curl example to update Gogs to a new version 0.13.0: +``` +curl -kX POST -d '{ "appStoreId": "io.gogs.cloudronapp@0.13.0" }' -H 'Content-Type: application/json' -H 'Authorization: Bearer 256e4c6c6f783dbff95ae233c63a36e297ef70a3528171b891a399f895a8e0e0' https://my-demo.cloudron.me/api/v1/apps/aaa8ad53-301b-4a77-9551-5df261686166/update +``` + +### Exec + +POST `/api/v1/apps/:appId/exec` admin + +Executes an arbitrary command in the context of the app. + +Request: + +``` +{ + cmd: [ , ... ], // the command to execute. default: '/bin/bash' + rows: , // optional. the tty window row size + columns: , // optional. the tty window column size + tty: // set to true, if a tty should be allocated +} +``` + +In order to provide separate streams for **stdout** and **stderr**, the http connection +is upgraded to **tcp** using the following headers: +``` +Upgrade: tcp +Connection: Upgrade +``` + +Once upgraded, the connection provides a full-duplex tcp connection. Clients can write **stdin** +to the connection. If a `tty` was allocated, then the server provides stdout and stderr as a +single stream. If no `tty` was allocated, then the server writes data in the following format: + +``` + +``` + +See the [Docker docs](https://docs.docker.com/engine/reference/api/docker_remote_api_v1.19/#attach-to-a-container) +for details. + +### Start app + +POST `/api/v1/apps/:appId/start` admin + +Starts an app. + +If the app cannot be started, the response code is 409. This can happen if the +app is in a state where it cannot started (for example, it is still installing). + +### Stop app + +POST `/api/v1/apps/:appId/stop` admin + +Stops an app. + +If the app cannot be stopped, the response code is 409. This can happen if the +app is in a state where it cannot stopped (for example, it is installing). + +When an app is stopped, the app's location will show an error page indicating +that the app is not running. + +### Uninstall app + +POST `/api/v1/apps/:appId/uninstall` admin + +Uninstalls an app. + +The existing backups of the app are still preserved (as per the backup configuration) and the +backup can be used to restore the app to the same state later. + +## Backups + +### Create a backup + +POST `/api/v1/backups` admin + +Schedules a complete backup of the Cloudron. + +Use the [Progress API](/references/api.html#get-progress) to track the progress of the backup. + +### List backups + +GET `/api/v1/backups` admin + +Lists the existing `box` backups. + +The Cloudron has two types of backups: +* `app` backups - Each app is backed up individually. This approach allows one to restore each app +independently of other apps. Use the [app backup API](/references/api.html#list-app-backups), if +you want to list the backups of a specific app. +* `box` backups - The Cloudron backs up certificates, user information, settings separately. This +backup contains a (virtual) link to all the app backups . + +Response (200): +``` +{ + backups: [ + { + id: , // a unique id for this backup + creationTime: , // ISO-8601 date in UTC + version: , // the Cloudron version when this backup was created + type: , // `box` + dependsOn: [ , ... ], // list of app backups that are part of this box backup + state: // 'normal' + }, + ... + ] +} +``` + +### Download backup + +POST `/api/v1/backups/:backupId/download_url` admin + +Generates a temporary URL to download the backup with id `backupId`. + +All backups are encrypted and tar zipped. `backupKey` returned in the response can be used to +decrypt the backup. + +Response (200): +``` +{ + id: , + url: , // download backups from this url + backupKey: // key to use to decrypt the backup +} +``` + +Curl example to download, decrypt and extract backup to current directory: +``` +curl -L | openssl aes-256-cbc -d -pass "pass:$" | tar -zxf - +``` + +## Cloudron + +### Update the Cloudron + +POST `/api/v1/cloudron/update` admin + +Updates the Cloudron to the latest version. + +If no new version is available, the response code will be 422. + +Some actions and events on the Cloudron like backups, app installs may block the Cloudron from updating. +In such a case, the response code will be 409. + +If the update request was accepted, the response code will be 202. Use the [Progress API](/references/api.html#get-progress) +to track the progress of the update. + +### Reboot the Cloudron + +POST `/api/v1/cloudron/reboot` admin + +Reboots the Cloudron. + +### Get progress + +GET `/api/v1/cloudron/progress` public + +Gets information about an in-progress Cloudron update or backup. + +`update` or `backup` is `null` when there is no such activity in progress. + +``` +Response (200): +{ + update: null || { percent: , message: }, + backup: null || { percent: , message: } +} +``` + +### Get status + +GET `/api/v1/cloudron/status` public + +Gets information about the Cloudron state and how it got provisioned. + +``` +Response (200): +{ + activated: , + version: , + boxVersionsUrl: , // Location of the Cloudron versions file to check for updates + apiServerOrigin: , // Always https://api.cloudron.io + provider: , + cloudronName: +} +``` + +### Get avatar + +GET `/api/v1/cloudron/avatar` public + +Returns the Cloudron avatar image as `Content-Type: image/png`. + +``` +Response (200): + +Content-Type: image/png +``` + +### Get configuration + +GET `/api/v1/cloudron/config` admin + +Gets information on how the Cloudron is configured. This is similar to the [Status API](/references/api.html#get-status) +except this contains some sensitive information and is not public. + +``` +Response (200): +{ + apiServerOrigin: , // Always https://api.cloudron.io + webServerOrigin: , // Always https://cloudron.io + isDev: , // internal + fqdn: , // The FQDN + ip: , // The public IP + version: , // Current version + update: { + box: , // Set to the next available Cloudron version + apps: { + : . // Set to next available app version + ... + } + }, + progress: { // See progress API + }, + isCustomDomain: , // false if the cloudron is on a .cloudron.me subdomain. true otherwise + developerMode: , + region: , // the geo-region of this Cloudron + size: , // the size of this Cloudron + billing: , // true if the user has setup billing + memory: , // the physical memory + provider: , + cloudronName: // the name of this cloudron +} +``` + + +## Eventlog + +### List events + +GET `/api/v1/eventlog` admin + +Lists all the past events. + +The `action` query parameter can be used to list events of a specific action. + +The `search` query parameter can be used to do a wildcard ('*search*') on the data field. + +This API supports [pagination](/references/api.html#pagination) - use the `page` and `per_page` query parameters to get specific pages. + +Response (200): + +``` +{ + eventlogs: [ + { + id: , // unique id for the event + action: , // see below + source: , // originator of this event + data: , // value depends on action. see below + creationTime: // ISO-8601 date in UTC + }, + .... + ] +} +``` + +`action` is one of the values below: +* cloudron.activate +* app.configure +* app.install +* app.restore +* app.uninstall +* app.update +* backup.finish +* backup.start +* certificate.renew +* settings.climode +* cloudron.start +* cloudron.update +* user.add +* user.login +* user.remove +* user.update + +`source` contains information on the originator of the action. For example, for user.login, this contains the IP address, the appId and the authType (ldap or simpleauth or oauth). + +`data` contains information on the event itself. For example, for user.login, this contains the userId that logged in. For app.install, it contains the manifest and location of the app that was installed. + +To list all the app installation events: +``` +curl -X GET -H 'Authorization: Bearer cb0463455a6606482be7956fc3abd53330ae23244e3492cda3914a2c5154c47e' https://my-demo.cloudron.me/api/v1/eventlog?action=app.install +``` + +## Groups + +Cloudron groups are a mechanism to restrict application access to a subset of users. You can add one or more users +to a group and assign one or more groups to an application. Only users that are members of any of the groups can access the application. + +Group membership is dynamic. Users instantly lose or gain access to an application based on their group +membership. + +### Create group + +POST `/api/v1/groups` admin + +Creates a new group. + +Request: +``` +{ + name: +} +``` + +`name` must be atleast 2 characters. The special built-in group named `admin` has all the +Cloudron administrators. + +Response (200): +``` +{ + id: , + name: +} +``` + +### Get group + +GET `/api/v1/groups/:groupId` admin + +Gets an existing group with id `groupId`. + +Response (200): +``` +{ + id: , + name: , + userIds: [ , ... ] // list of users that are part of this group +} +``` + +### List groups + +GET `/api/v1/groups` admin + +Lists all groups. + +Response (200): +``` +{ + groups: [ + { + id: , + name: , + userIds: [ , ... ] // list of users that are part of this group + }, + ... + ] +} +``` + +### Delete group + +DELETE `/api/v1/groups/:groupId` admin + +Deletes an existing group with id `groupId`. + +The special `admin` group cannot be removed. + +Response (204): +``` +{} +``` + +## Mailboxes + +Mailboxes allow users to receive mail using `IMAP`. Every user gets a mailbox with the same name as the username. +Users can receive email using IMAP (TLS) at `my.customdomain.com` and send mail using SMTP (STARTTLS) at +`my.customdomain.com`. + +Apps can also receive email using the [recvmail addon](/references/addons.html#recvmail). App mailboxes are +named as `.app`. For this reason, the `.app` suffix is reserved for apps. + +### Create mailbox + +POST `/api/v1/mailboxes` admin + +Creates a new mailbox. + +`name` specifies the address at which email can be received and does not include the domain name. `name` can contain +only alphanumeric characters and a dot ('.') and must be lower case. The Cloudron mail server support '+' based subaddressing (i.e)name+tag@domain.com will be delivered to this mailbox. + +mailboxes can only be accessed by users with the same username as the mailbox. A mailbox is automatically created +for every user on the Cloudron. + +Request: +``` +{ + name: +} +``` + +`name` must be atleast 2 characters and must be alphanumeric. + +Response (200): +``` +{ + id: , + name: +} +``` + +### Get mailbox + +GET `/api/v1/mailboxes/:mailboxName` admin + +Gets an existing mailbox with name `mailboxName`. + +Response (200): +``` +{ + id: , + name: , + aliases: [ , ...] +} +``` + +### List mailboxes + +GET `/api/v1/mailboxes` admin + +Lists all mailboxes. + +Response (200): +``` +{ + mailboxes: [ + { + id: , + name: + }, + ... + ] +} +``` + +### Set mailbox aliases + +PUT `/api/v1/mailboxes/:mailboxName/aliases` admin + +Sets aliases of an existing mailbox. + +An alias is an alternate email address for a mailbox. Mails received at the alternate address are stored +in the mailbox. The email's `to` header should indicate the address that the original sent it to. + +This also allows users to send emails with `From` address set to an alias. + +Note that this call will replace the current aliases for this mailbox, it does **not** merge the provided ones with the current aliases. + +Aliases have to be unique and cannot conflict with existing aliases or mailbox names. Any conflict with result in a 409. + +Request: +``` +{ + aliases: [ , ... ] +} + +``` + +### Get mailbox aliases + +GET `/api/v1/mailboxes/:mailboxName/aliases` admin + +Gets aliases of an existing mailbox. + +Response (200): +``` +{ + aliases: [ , ... ] +} +``` + +### Delete mailbox + +DELETE `/api/v1/mailboxes/:mailboxName` admin + +Deletes an existing with name `mailboxName`. + +Response (204): +``` +{} +``` + +## Profile + +### Get profile + +GET `/api/v1/profile` profile + +Gets the profile information of the token owner. This is useful to verify access tokens. + +Response (200): +``` +{ + id: , + username: , + email: , + admin: , + displayName: +} +``` + +### Update profile + +POST `/api/v1/profile` profile + +Updates the email or displayName of the token owner. + +Request: +``` +{ + email: , // optional + displayName: // optional +} +``` + +Response (204): +``` +{} +``` + +### Update password + +POST `/api/v1/profile/password` profile + +Updates the password of the token owner. + +Request: +``` +{ + password: , // current password. only required for OAuth tokens + newPassword: +} +``` + +Response (204): +``` +{} +``` + +### Tutorial + +POST `/api/v1/profile/tutorial` profile + +Toggles display of the tutorial when the token owner logs in. + +Request: +``` +{ + showTutorial: +} +``` + +Response (204): +``` +{} +``` + +## Settings + +### Get auto update pattern + +GET `/api/v1/settings/autoupdate_pattern` admin + +Gets the auto update pattern that the Cloudron uses to automatically update itself and installed apps. + +Patterns are matched based on the Cloudron's [timezone](references/api.html#get-timezone). + +Response (200): +``` +{ + pattern: // a cron pattern +} +``` + +### Set auto update pattern + +POST `/api/v1/settings/autoupdate_pattern` admin + +Sets the auto update pattern that the Cloudron uses to automatically update itself and installed apps. + +Request: +``` +{ + pattern: // a cron pattern +} +``` + +`pattern` has the format listed in the [node-cron](https://github.com/ncb000gt/node-cron#cron-ranges) page. +Note that unlike classic crontab format, the pattern contains seconds as the first part. + +Setting pattern to `never` disables auto update. + +Some examples of patterns are: +* `00 00 1,3,5,23 * * *` would run updates at 1am, 3am, 5am, 11pm every night. +* `0 030 4 1,15 * 5` would run updates at 4:30 am on the 1st and 15th of each month, plus every Friday. + +Patterns are matched based on the Cloudron's [timezone](references/api.html#get-timezone). + +### Get avatar +GET `/api/v1/settings/cloudron_avatar` admin + +Gets the Cloudron avatar image as `Content-Type: image/png`. + +Note that the avatar is also available without authentication using the [Get avatar](/references/api.html#get-avatar) API. + +Response (200): +``` +Content-Type: image/png +``` + +### Set avatar + +POST `/api/v1/settings/cloudron_avatar` admin + +Sets the avatar of the Cloudron. + +The request is sent as `multipart/form-data`. It must contain a field named `avatar` and must be a `png`. + +Example: +``` +curl -X POST -F "avatar=@./avatar.png" -H 'Authorization: Bearer 215841b5943f5432a26ef3a1526f548a40268a92ed9baca5db980be0545da596' https://my-demo.cloudron.me/api/v1/settings/cloudron_avatar +``` + +### Get Backup Configuration + +GET `/api/v1/settings/backup_config` admin internal + +Gets the credentials used to upload backups. + +This is currently internal API and is documented here for completeness. + +Response(200): +``` +{ + "provider": , // 'caas' + "key": , // encryption key + "region": , // s3 region + "bucket": , // s3 bucket name + "prefix": , // s3 bucket prefix + "token": // caas specific token +} +``` + +### Set Backup Configuration + +POST `/api/v1/settings/backup_config` admin internal + +Sets the credentials used to upload backups. + +This is currently internal API and is documented here for completeness. + +### Get DNS Configuration + +GET `/api/v1/settings/dns_config` admin internal + +Gets the credentials used to configure DNS. + +This is currently internal API and is documented here for completeness. + +Response(200): +``` +{ + "provider": , // 'caas' + "token": // caas specific token +} +``` + +### Set DNS Configuration + +POST `/api/v1/settings/dns_config` admin internal + +Sets the credentials used to configure DNS. + +This is currently internal API and is documented here for completeness. + +### Set fallback Certificate + +POST `/api/v1/settings/certificate` admin internal + +Sets the fallback TLS certificate to use if automatic certificate generation or renewal fails. + +The fallback certificate only applies to Cloudrons running on a custom domain. + +The Cloudron automatically installs and renews certificates for all subdomains using Let's Encrypt. +The fallback certificate is used if Let's Encrypt fails. + +Request: +``` +{ + cert: , // TLS certificate including the full chain + key: // TLS key +} +``` + +### Set `my` subdomain certification + +POST `/api/v1/settings/admin_certificate` admin internal + +Sets the certificate to use for the `my` subdomain. + +The `my` subdomain certificate only applies to Cloudrons running on a custom domain. + +Request: +``` +{ + cert: , // TLS certificate including the full chain + key: // TLS key +} +``` + +### Get name + +GET `/api/v1/settings/cloudron_name` admin + +Gets the name of the Cloudron. + +Note that the name is also available without authentication using the [Get status](/references/api.html#get-status) API. + +``` +Response (200): +{ + name: +} +``` + +### Set name + +POST `/api/v1/settings/cloudron_name` admin + +Sets the name of the Cloudron. + +The Cloudron name is shown in the title bar and all the login screens. It has a maximum length of 32 characters. + +Request: +``` +{ + name: +} +``` + +### Get timezone + +GET `/api/v1/settings/time_zone` admin + +Gets the timezone of the Cloudron. + +Timezone is automatically set based on the IP address from where the Cloudron was activated. This timezone is used with the [auto update pattern](/references/api.html#get-auto-update-pattern) to trigger updates at the +correct time. + +``` +Response (200): +{ + timeZone: // the timezone +} +``` + +### Set timezone + +POST `/api/v1/settings/time_zone` admin + +Sets the time zone of the Cloudron. + +Timezone is automatically set based on the IP address from where the Cloudron was activated. This timezone is used with the [auto update pattern](/references/api.html#get-auto-update-pattern) to trigger updates at the +correct time. + +See the [Tz database](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) for a list of valid values. + +Request: +``` +{ + timeZone: +} +``` + +## Users + +### Create user + +POST `/api/v1/users` admin + +Creates a new user. + +Request: +``` +{ + email: , // required + invite: , // required + username: , // optional + displayName: // optional +} +``` + +`invite` indicates if the user should be sent an invitation email. The invitation email allows the user +to setup a username and password. Set this to `false` to send out a custom invitation email of your own +to the user. The invitation link can be constructed based on the `resetToken` in the response. + +`username` has to be at least two characters long and must be alphanumeric. If unspecified, the new user +can pick any available name on first sign up. For security, `username` cannot be changed once set. +Some apps (incorrectly) use the `username` as their unique identifier. As a result, it might mistake a newly created user as a previous user with the same username. If you want to reset the username, delete the existing +the user and create a new one again. + +`displayName` may consist of one or more words to specify the first name and surname. + +Response (201): +``` +{ + id: , + username: , + displayName: , + email: , + groupIds: [ , ... ] + resetToken: // User can sign up at https://my-demo.cloudron.me/api/v1/session/account/setup.html?reset_token= +} +``` + +### Get user + +GET `/api/v1/users/:userId` admin + +Gets detailed information about a specific user with id `userId`. + +Response (200): +``` +{ + id: , + username: , + email: , + groupIds: [ , ... ], // list of groups this user is part of + admin: , // a boolean indicating if this user is an admin + displayName: +} +``` + +### List users + +GET `/api/v1/users` admin + +Lists all users. + +Response (200): +``` +{ + users: [ + { + id: , + username: , + email: , + displayName: , + groupIds: [ , ... ], // list of groups this user is part of + admin: // a boolean indicating if this user is an admin + }, + ... + ] +} +``` + +### Update user + +POST `/api/v1/users/:userId` admin + +Modify user's email or display name. As noted in [Create user](/references/api.html#create-user), username +cannot be changed. + +Request: +``` +{ + email: , // optional + displayName: // optional +} + +``` + +Response (204): +``` +{} +``` + +### Re-invite user + +POST `/api/v1/users/:userId/invite` admin + +Resends the invitation link to an existing user. + +A re-invite call invalidates any previous invite links. The invitation link can be constructed based on the resetToken in the response. + +Request: +``` +{} +``` + +Response (200): +``` +{ + resetToken: // // User can sign up at https://my-demo.cloudron.me/api/v1/session/account/setup.html?reset_token= +} +``` + +### Set groups + +PUT `/api/v1/users/:userId/groups` admin + +Sets the groups for the user with id `userId`. + +Groups are identified by groupIds which can be retrieved using the +[Groups API](/references/api.html#list-all-groups). + +An admin cannot remove himself from the special `admin` group. This will result in a response code of 403. + +Note that this call will replace the current groups for this user, it does **not** merge the provided ones with the current groups for that user. + +Request: +``` +{ + groupIds: [ , ... ] +} + +``` +Response (204): +``` +{} +``` + +### Delete user + +DELETE `/api/v1/users/:userId` admin + +Deletes user with id `userId`. + +A user cannot remove himself, this will result in a response code of 403. + +A deleted user will not be able to login to any app anymore. Currently, apps are not notified of a user +deletion and any user-specific data data will persist on the app. + +Note that applications may cache user credentials (for example, as a browser session cookie) and thus the +effect of a deletion may not be immediately visible until the user logs out. + +Response (204): +``` +{} +``` diff --git a/docs/references/architecture.md b/docs/references/architecture.md new file mode 100644 index 000000000..e39f669b4 --- /dev/null +++ b/docs/references/architecture.md @@ -0,0 +1,90 @@ +# Architecture + +## Introduction + +The Cloudron platform is designed to easily install and run web applications. +The application architecture is designed to let the Cloudron take care of system +operations like updates, backups, firewalls, domain management, certificate management +etc. This allows app developers to focus on their application logic instead of deployment. + +At a high level, an application provides an `image` and a `manifest`. The image is simply +a docker image that is a bundle of the application code and it's dependencies. The manifest +file specifies application runtime requirements like database type and authentication scheme. +It also provides meta information for display purposes in the [Cloudron Store](/appstore.html) +like the title, icon and pricing. + +Web applications like blogs, wikis, password managers, code hosting, document editing, +file syncers, notes, email, forums are a natural fit for the Cloudron. Decentralized "social" +networks are also good app candidates for the Cloudron. + +## Image + +Application images are created using [Docker](https://www.docker.io). Docker provides a way +to package (and containerize) the application as a filesystem which contains it's code, system libraries +and just about anything the app requires. This flexible approach allows the application to use just +about any language or framework. + +Application images are instantiated as `containers`. Cloudron can run one or more isolated instances +of the same application as one or more containers. + +Containerizing your application provides the following benefits: +* Apps run in the familiar environment that they were packaged for and can have libraries +and packages that are independent of the host OS. +* Containers isolate applications from one another. + +The [base image](/references/baseimage.html) is the parent of all app images. + +## Cloudron Manifest + +Each app provides a `CloudronManifest.json` that specifies information required for the +`Cloudron Store` and for the installation of the image in the Cloudron. + +Information required for container installation includes: +* List of `addons` like databases, caches, authentication mechanisms and file systems +* The http port on which the container is listening for incoming requests +* Additional TCP ports on which the application is listening to (for e.g., git, ssh, +irc protocols) + +Information required for the Cloudron Store includes: +* Unique App Id +* Title +* Version +* Logo + +See the [manifest reference](/references/manifest.html) for more information. + +## Addons + +Addons are services like database, authentication, email, caching that are part of the +Cloudron. Setup, provisioning, scaling and maintenance of addons is taken care of by the +Cloudron. + +The fundamental idea behind addons is to allow resource sharing across applications. +For example, a single MySQL server instance can be used across multiple apps. The Cloudron +sets up addons in such a way that apps are isolated from each other. + +Addons are opt-in and must be specified in the Cloudron Manifest. When the app runs, environment +variables contain the necessary information to access the addon. See the +[addon reference](/references/addons.html) for more information. + +## Authentication + +The Cloudron provides a centralized dashboard to manage users, roles and permissions. Applications +do not create or manage user credentials on their own and instead use one of the various +authentication strategies provided by the Cloudron. + +Authentication strategies include OAuth 2.0, LDAP or Simple Auth. See the +[Authentication Reference](/references/authentication.html) for more information. + +Authorizing users is application specific and it is only authentication that is delegated to the +Cloudron. + +## Cloudron Store + +Cloudron Store provides a market place to publish and optionally monetize your app. Submitting to the +Cloudron Store enables any Cloudron user to discover, purchase and install your application with +a few clicks. + +## What next? + +* [Package an existing app for the Cloudron](/tutorials/packaging.html) \ No newline at end of file diff --git a/docs/references/authentication.md b/docs/references/authentication.md new file mode 100644 index 000000000..5eade221a --- /dev/null +++ b/docs/references/authentication.md @@ -0,0 +1,115 @@ +# Authentication + +## Overview + +Cloudron provides a centralized dashboard to manage users, roles and permissions. Applications +do not create or manage user credentials on their own and instead use one of the various +authentication strategies provided by the Cloudron. + +Note that authentication only identifies a user and does not indicate if the user is authorized +to perform an action in the application. Authorizing users is application specific and must be +implemented by the application. + +## Users & Admins + +Cloudron user management is intentionally very simple. The owner (first user) of the +Cloudron is `admin` by default. The `admin` role allows one to install, uninstall and reconfigure +applications on the Cloudron. + +A Cloudron `admin` can create one or more users. Cloudron users can login and use any of the installed +apps in the Cloudron. In general, adding a cloudron user is akin to adding a person from one's family +or organization or team because such users gain access to all apps in the Cloudron. Removing a user +immediately revokes access from all apps. + +A Cloudron `admin` can give admin privileges to one or more Cloudron users. + +Each Cloudron user has an unique `username` and an `email`. + +## Strategies + +Cloudron provides multiple authentication strategies. + +* OAuth 2.0 provided by the [OAuth addon](/references/addons.html#oauth) +* LDAP provided by the [LDAP addon](/references/addons.html#ldap) +* Simple Auth provided by [Simple Auth addon](/references/addons.html#simpleauth) + +## Choosing a strategy + +Applications can be broadly categorized based on their user management as follows: + +* Multi-user aware + * Such apps have a full fledged user system and support multiple users and groups. + * These apps should use OAuth or LDAP. + * LDAP and OAuth APIs allow apps to detect if the user is a cloudron `admin`. Apps should use this flag + to show the application's admin panel for such users. + + +* No user + * Such apps have no concept of logged-in user. + * The Cloudron provides a `website visibility` setting that allows a Cloudron admin to optionally + install an OAuth proxy in front of such applications. In such a case, a user visiting the website first + authenticates with the OAuth proxy and once authenticated is allowed into the application. + * When an OAuth proxy is installed, such applications can use the `X-Authenticated-User` header from the + [ICAP Extensions](https://tools.ietf.org/html/draft-stecher-icap-subid-00#section-3.4) de facto standard. + This value can be used for display purposes or creating meta data for a document. + + +* Single user + * Such apps only have a single user who is usually also the `admin`. + * These apps can use Simple Auth or LDAP since they can authenticate users with a simple HTTP or LDAP request. + * Such apps _must_ set the `singleUser` property in the manifest which will restrict login to a single user + (configurable through the Cloudron's admin panel). + +## Public and Private apps + +`Private` apps display content only when they have a signed-in user. These apps can choose one of the +authentication strategies listed above. + +`Public` apps display content to any visiting user (e.g a blog). These apps have a `login` url to allow +the editors & admins to login. This path can be optionally set as the `configurePath` in the manifest for +discoverability (for example, some blogs hide the login link). + +Some apps allow the user to choose `private` or `public` mode or some other combination. Such configuration +is done at app install time and cannot be changed using a settings interface. It is tempting to show the user +a configuration dialog on first installation to switch the modes. This, however, leads the user to believe that +this configuration can be changed at any time later. In the case where this setting can be changed dynamically +from a settings ui in the app, it's better to simply put some sensible defaults and let the user discover +the settings. In the case where such settings cannot be changed dynamically, it is best to simply publish two +separate apps in the Cloudron store each with a different configuration. + +## External User Registration + +Some apps allow external users to register and create accounts. For example, a public company chat that +can invite anyone to join or a blog allowing registered commenters. + +Such applications must track Cloudron users and external registered users independently (for example, using a flag). +As a thumb rule, apps must provide separate login buttons for each of the possible user sources. Such a design prevents +external users from (inadvertently) spoofing Cloudron users. + +Naively handling user registration enables attacks of the following kind: +* An external user named `foo` registers in the app. +* A LDAP user named `foo` is later created on the Cloudron. +* When a user named `foo` logs in, the app cannot determine the correct `foo` anymore. Making separate login buttons for each +login source clears the confusion for both the user and the app. + +## Userid + +The preferred approach to track users in an application is a uuid or the Cloudron `username`. +The `username` in Cloudron is unique and cannot be changed. + +Tracking users using `email` field is error prone since that may be changed by the user anytime. + +## Single Sign-on + +Single sign-on (SSO) is a property where a user logged in one application automatically logs into +another application without having to re-enter his credentials. When applications implement the +OAuth strategy, they automatically take part in Cloudron SSO. When a user signs in one application with +OAuth, they will automatically log into any other app implementing OAuth. + +Conversely, signing off from one app, logs them off from all the apps. + +## Security + +The LDAP and Simple Auth strategies require the user to provide their plain text passwords to the +application. This might be a cause of concern and app developers are thus highly encouraged to integrate +with OAuth. OAuth also has the advantage of supporting Single Sign On. diff --git a/docs/references/baseimage.md b/docs/references/baseimage.md new file mode 100644 index 000000000..11d7abac5 --- /dev/null +++ b/docs/references/baseimage.md @@ -0,0 +1,101 @@ +# Base Image + +## Overview + +The application's Dockerfile must specify the FROM base image to be `cloudron/base:0.8.1`. + +The base image already contains most popular software packages including node, nginx, apache, +ruby, PHP. Using the base image greatly reduces the size of app images. + +The goal of the base image is simply to provide pre-downloaded software packages. The packages +are not configured in any way and it's up to the application to configure them as they choose. +For example, while `apache` is installed, there are no meaningful site configurations that the +application can use. + +## Packages + +The following packages are part of the base image. If you need another version, you will have to +install it yourself. + +* Apache 2.4.10 +* Composer 1.0 (alpha 10) +* Go 1.5.1 +* Gunicorn 19.0.0 +* Java 1.7, JRE IcedTea 2.5.5 +* Maven 3.0.5 +* MySQL Client 5.6 +* nginx 1.6.2 +* Node 0.10.40, 0.12.7, 4.2.1 (installed under `/usr/local/node-`) [more information](#node-js) +* Perl 5.20.1 +* PHP 5.5.12 +* Postgresql client 9.4 +* Python 2.7.8 +* Ruby 2.1.2 +* sqlite3 3.8.6 +* Supervisor 3.0 +* uwsgi 2.0.6 + +## Inspecting the base image + +The base image can be inspected by installing [Docker](https://docs.docker.com/installation/). + +Once installed, pull down the base image locally using the following command: +``` + docker pull cloudron/base:0.8.1 +``` + +To inspect the base image: +``` + docker run -ti cloudron/base:0.8.1 /bin/bash +``` + +*Note:* Please use `docker 1.9.0` or above to pull the base image. Doing otherwise results in a base +image with an incorrect image id. The image id of `cloudron/base:0.8.1` is `d038af182821`. + +## The `cloudron` user + +The base image contains a user named `cloudron` that apps can use to run their app. + +It is good security practice to run apps as a non-previleged user. + +## Env vars + +The following environment variables are set as part of the application runtime. + +### API_ORIGIN + +API_ORIGIN is set to the HTTP(S) origin of this Cloudron's API. For example, +`https://my-girish.cloudron.us`. + +### APP_DOMAIN + +APP_DOMAIN is set to the domain name of the application. For example, `app-girish.cloudron.us`. + +### APP_ORIGIN + +APP_ORIGIN is set to the HTTP(S) origin on the application. This is origin which the +user can use to reach the application. For example, `https://app-girish.cloudron.us`. + +### CLOUDRON + +CLOUDRON is always set to '1'. This is useful to write Cloudron specific code. + +### WEBADMIN_ORIGIN + +WEBADMIN_ORIGIN is set to the HTTP(S) origin of the Cloudron's web admin. For example, +`https://my-girish.cloudron.us`. + +## Recipes + +We have collected some recipes for configuring popular packages [here](/references/recipes.html). + +## Node.js + +The base image comes pre-installed with various node.js versions. + +They can be used by adding `ENV PATH /usr/local/node-/bin:$PATH`. + +Currently available versions are: + * 0.10.40 + * 0.12.7 + * 4.2.1 diff --git a/docs/references/bestpractices.md b/docs/references/bestpractices.md new file mode 100644 index 000000000..06098ec4e --- /dev/null +++ b/docs/references/bestpractices.md @@ -0,0 +1,93 @@ +# Best practices + +## Overview + +This document explains the spirit of what makes a Cloudron app. + +## No Setup + +Cloudron apps do not show a setup screen after installation and should choose reasonable +defaults. + +Databases, email configuration should be automatically picked up using [addons](/references/addons.html). + +Admin role for the application can be detected dynamically using one of the [authentication](/references/authentication.html) +strategies. + +## Image + +The Dockerfile contains a specification for building an application image. + + * Install any required software packages in the Dockerfile. + + * Create static configuration files in the Dockerfile. + + * Create symlinks to dynamic configuration files under `/run` in the Dockerfile. + + * Docker supports restarting processes natively. Should your application crash, it will + be restarted automatically. If your application is a single process, you do not require + any process manager. + + * The main process must handle `SIGTERM` and forward it as required to child processes. `bash` + does not automatically forward signals to child processes. For this reason, when using a startup + shell script, remember to use `exec ` as the last line. Doing so will replace bash with your + program and allows your program to handle signals as required. + + * Use `supervisor`, `pm2` or any of the other process managers if you application has more + then one component. This excludes web servers like apache, nginx which can already manage their + children by themselves. Be sure to pick a process manager that forwards signals to child processes. + + * Disable auto updates for apps. Updates must be triggered through the Cloudron Store. This allows the admin + to manage updates and downtime in a central location (the Cloudron Webadmin). + +## File system + +The Cloudron runs the application image as read-only. The app can only write to the following directories: + + * `/tmp` - use this for temporary files. + + * `/run` - use this for runtime configration and any dynamic data. + + * `/app/data` - When the `localstorage` addon is enabled, any data under this directory is automatically backed up. + +## Logging + +Cloudron applications stream their logs to stdout and stderr. In contrast to logging +to files, this approach has many advantages: + + * App does not need to rotate logs and the Cloudron takes care of managing logs + * App does not need special mechanism to release log file handles (on a log rotate) + * Integrates better with tooling like `cloudron cli` + +This document gives you some recipes for configuring popular libraries to log to stdout. See +[base image](/references/baseimage.html#configuring) on how to configure various libraries to log to stdout/stderr. + + +## Memory + +By default, applications get 200MB RAM (including swap). This can be changed using the `memoryLimit` field in the manifest. + +Design your application runtime for concurrent use by 10s of users. The Cloudron is not designed for concurrent access by +100s or 1000s of users. + +## Startup + + * Apps must not present a post-installation screen on first run. It should be already pre-configured for + a specific purpose. + + * Do not run as `root`. Apps can use the `cloudron` user which is part of the [base image](/references/baseimage.html) + for this purpose or create their own. + + * When using the `localstorage` addon, the application must change the ownership of files in `/app/data` as desired using `chown`. This + is necessary because file permissions may not be correctly preserved across backup, restore, application and base image + updates. + + * Addon information (mail, database) is exposed as environment variables. An application must use these values directly + and not cache them across restarts. If the variables are stored in a configuration file, then the configuration file + must be regenerated on every application start. This is usually done using a configuration template that is patched + on every startup. + +## Authentication + +Apps should integrate with one of the [authentication strategies](/references/authentication.html). +This saves the user from having to manage separate set of users for different apps. diff --git a/docs/references/button.md b/docs/references/button.md new file mode 100644 index 000000000..7436c2f72 --- /dev/null +++ b/docs/references/button.md @@ -0,0 +1,47 @@ +# Cloudron Button + +The `Cloudron Button` allows anyone to install an application with +the click of a button on their Cloudron. + +The button can be added to just about any website including the application's website +and README.md files in GitHub repositories. + +## Prerequisites + +The `Cloudron Button` is intended to work only for applications that have been +published on the Cloudron Store. The [basic tutorial](/tutorials/basic.html#publishing) +gives an overview of how to package and publish your application for the +Cloudron Store. + +## HTML Snippet + +``` + +``` + +_Note_: Replace `` with your application's id. + +## Markdown Snippet + +``` +[![Install](https://cloudron.io/img/button32.png)](https://cloudron.io/button.html?app=) +``` + +_Note_: Replace `` with your application's id. + + +## Button Height + +The button may be used in different heights - 32, 48 and 64 pixels. + +[![Install](/img/button32.png)](https://cloudron.io/button.html?app=io.gogs.cloudronapp) + +[![Install](/img/button48.png)](https://cloudron.io/button.html?app=io.gogs.cloudronapp) + +[![Install](/img/button64.png)](https://cloudron.io/button.html?app=io.gogs.cloudronapp) + +or as SVG + +[![Install](/img/button.svg)](https://cloudron.io/button.html?app=io.gogs.cloudronapp) + +_Note_: Clicking the buttons above will install [Gogs](http://gogs.io/) on your Cloudron. diff --git a/docs/references/manifest.md b/docs/references/manifest.md new file mode 100644 index 000000000..60e667859 --- /dev/null +++ b/docs/references/manifest.md @@ -0,0 +1,458 @@ +# CloudronManifest + +## Overview + +Every Cloudron Application contains a `CloudronManifest.json`. + +The manifest contains two categories of information: + +* Information about displaying the app on the Cloudron Store. For example, + the title, author information, description etc + +* Information for installing the app on the Cloudron. This includes fields + like httpPort, tcpPorts. + +A CloudronManifest.json can **only** contain fields that are listed as part of this +specification. The Cloudron Store and the Cloudron *may* reject applications that have +extra fields. + +Here is an example manifest: + +``` +{ + "id": "com.example.test", + "title": "Example Application", + "author": "Girish Ramakrishnan ", + "description": "This is an example app", + "tagline": "A great beginning", + "version": "0.0.1", + "healthCheckPath": "/", + "httpPort": 8000, + "addons": { + "localstorage": {} + }, + "manifestVersion": 1, + "website": "https://www.example.com", + "contactEmail": "support@clourdon.io", + "icon": "file://icon.png", + "tags": [ "test", "collaboration" ], + "mediaLinks": [ "www.youtube.com/watch?v=dQw4w9WgXcQ" ] +} +``` + +## Fields + +### addons + +Type: object + +Required: no + +Allowed keys +* [ldap](addons.html#ldap) +* [localstorage](addons.html#localstorage) +* [mongodb](addons.html#mongodb) +* [mysql](addons.html#mysql) +* [oauth](addons.html#oauth) +* [postgresql](addons.html#postgresql) +* [redis](addons.html#redis) +* [sendmail](addons.html#sendmail) + +The `addons` object lists all the [addons](addons.html) and the addon configuration used by the application. + +Example: +``` + "addons": { + "localstorage": {}, + "mongodb": {} + } +``` + +### author + +Type: string + +Required: yes + +The `author` field contains the name and email of the app developer (or company). + +Example: +``` + "author": "Cloudron Inc " +``` + +### changelog + +Type: markdown string + +Required: no + +The `changelog` field contains the changes in this version of the application. This string +can be a markdown style bulleted list. + +Example: +``` + "changelog": "* Add support for IE8 \n* New logo" +``` + +### configurePath + +Type: path string + +Required: no + +The `configurePath` can be used to specify the absolute path to the configuration / settings +page of the app. When this path is present, an absoluted URL is constructed from the app's +install location this path and presented to the user in the configuration dialog of the app. + +This is useful for apps that have a main page which does not display a configuration / settings +url (i.e) it's hidden for aesthetic reasons. For example, a blogging app like wordpress might +keep the admin page url hidden in the main page. Setting the configurationPath makes the +configuration url discoverable by the user. + +Example: +``` + "configurePath": "/wp-admin" +``` + +### contactEmail + +Type: email + +Required: yes + +The `contactEmail` field contains the email address that Cloudron users can contact for any +bug reports and suggestions. + +Example: +``` + "contactEmail": "support@testapp.com" +``` + +### description + +Type: markdown string + +Required: yes + +The `description` field contains a detailed description of the app. This information is shown +to the user when they install the app from the Cloudron Store. + +Example: +``` + "description": "This is a detailed description of this app." +``` + +A large `description` can be unweildy to manage and edit inside the CloudronManifest.json. For +this reason, the `description` can also contain a file reference. The Cloudron CLI tool fills up +the description from this file when publishing your application. + +Example: +``` + "description:": "file://DESCRIPTION.md" +``` + +### developmentMode + +Type: boolean + +Required: no + +Setting `developmentMode` to true disables readonly rootfs and the default memory limit. In addition, +the application *pauses* on start and can be started manually using `cloudron exec`. Note that you +cannot submit an app to the store with this field turned on. + +This mode can be used to identify the files being modified by your application - often required to +debug situations where your app does not run on a readonly rootfs. Run your app using `cloudron exec` +and use `find / -mmin -30` to find file that have been changed or created in the last 30 minutes. + +### healthCheckPath + +Type: url path + +Required: yes + +The `healthCheckPath` field is used by the Cloudron Runtime to determine if your app is running and +responsive. The app must return a 2xx HTTP status code as a response when this path is queried. In +most cases, the default "/" will suffice but there might be cases where periodically querying "/" +is an expensive operation. In addition, the app might want to use a specialized route should it +want to perform some specialized internal checks. + +Example: +``` + "healthCheckPath": "/" +``` +### httpPort + +Type: positive integer + +Required: yes + +The `httpPort` field contains the TCP port on which your app is listening for HTTP requests. This port +is exposed to the world via subdomain/location that the user chooses at installation time. While not +required, it is good practice to mark this port as `EXPOSE` in the Dockerfile. + +Cloudron Apps are containerized and thus two applications can listen on the same port. In reality, +they are in different network namespaces and do not conflict with each other. + +Note that this port has to be HTTP and not HTTPS or any other non-HTTP protocol. HTTPS proxying is +handled by the Cloudron platform (since it owns the certificates). + +Example: +``` + "httpPort": 8080 +``` + +### icon + +Type: local image filename + +Required: no + +The `icon` field is used to display the application icon/logo in the Cloudron Store. Icons are expected +to be square of size 256x256. + +``` + "icon": "file://icon.png" +``` + +### id + +Type: reverse domain string + +Required: yes + +The `id` is a unique human friendly Cloudron Store id. This is similar to reverse domain string names used +as java package names. The convention is to base the `id` based on a domain that you own. + +The Cloudron tooling allows you to build applications with any `id`. However, you will be unable to publish +the application if the id is already in use by another application. + +``` + "id": "io.cloudron.testapp" +``` + +### manifestVersion + +Type: integer + +Required: yes + +`manifestVersion` specifies the version of the manifest and is always set to 1. + +``` + "manifestVersion": 1 +``` + +### mediaLinks + +Type: array of urls + +Required: no + +The `mediaLinks` field contains an array of links that the Cloudron Store uses to display a slide show of pictures +and videos of the application. + +All links are preferably https. + +``` + "mediaLinks": [ + "www.youtube.com/watch?v=dQw4w9WgXcQ", + "https://images.rapgenius.com/fd0175ef780e2feefb30055be9f2e022.520x343x1.jpg" + ] +``` + +### memoryLimit + +Type: bytes (integer) + +Required: no + +The `memoryLimit` field is the maximum amount of memory (including swap) in bytes an app is allowed to consume before it +gets killed and restarted. + +By default, all apps have a memoryLimit of 200MB. For example, to have a limit of 500MB, + +``` + "memoryLimit": 524288000 +``` + +### maxBoxVersion + +Type: semver string + +Required: no + +The `maxBoxVersion` field is the maximum box version that the app can possibly run on. Attempting to install the app on +a box greater than `maxBoxVersion` will fail. + +This is useful when a new box release introduces features which are incompatible with the app. This situation is quite +unlikely and it is recommended to leave this unset. + +### minBoxVersion + +Type: semver string + +Required: no + +The `minBoxVersion` field is the minimum box version that the app can possibly run on. Attempting to install the app on +a box lesser than `minBoxVersion` will fail. + +This is useful when the app relies on features that are only available from a certain version of the box. If unset, the +default value is `0.0.1`. + +### singleUser + +Type: boolean + +Required: no + +The `singleUser` field can be set to true for apps that are meant to be used only a single user. + +When set, the Cloudron will display a user selection dialog at installation time. The selected user is the sole user +who can access the app. + +### tagline + +Type: one-line string + +Required: no + +The `tagline` is used by the Cloudron Store to display a single line short description of the application. + +``` + "tagline": "The very best note keeper" +``` + +### tags + +Type: Array of strings + +Required: no + +The `tags` are used by the Cloudron Store for filtering searches by keyword. + +``` + "tags": [ "git", "version control", "scm" ] +``` + +### targetBoxVersion + +Type: semver string + +Required: no + +The `targetBoxVersion` field is the box version that the app was tested on. By definition, this version has to be greater +than the `minBoxVersion`. + +The box uses this value to enable compatibility behavior of APIs. For example, an app sets the targetBoxVersion to 0.0.5 +and is published on the store. Later, box version 0.0.10 introduces a new feature that conflicts with how apps used +to run in 0.0.5 (say SELinux was enabled for apps). When the box runs such an app, it ensures compatible behavior +and will disable the SELinux feature for the app. + +If unspecified, this value defaults to `minBoxVersion`. + +### tcpPorts + +Type: object + +Required: no + +Syntax: Each key is the environment variable. Each value is an object containing `title`, `description` and `defaultValue`. +An optional `containerPort` may be specified. + +The `tcpPorts` field provides information on the non-http TCP ports/services that your application is listening on. During +installation, the user can decide how these ports are exposed from their Cloudron. + +For example, if the application runs an SSH server at port 29418, this information is listed here. At installation time, +the user can decide any of the following: +* Expose the port with the suggested `defaultValue` to the outside world. This will only work if no other app is being exposed at same port. +* Provide an alternate value on which the port is to be exposed to outside world. +* Disable the port/service. + +To illustrate, the application lists the ports as below: +``` + "tcpPorts": { + "SSH_PORT": { + "title": "SSH Port", + "description": "SSH Port over which repos can be pushed & pulled", + "defaultValue": 29418, + "containerPort": 22 + } + }, +``` + +In the above example: +* `SSH_PORT` is an app specific environment variable. Only strings, numbers and _ (underscore) are allowed. The author has to ensure that they don't clash with platform profided variable names. + +* `title` is a short one line information about this port/service. + +* `description` is a multi line description about this port/service. + +* `defaultValue` is the recommended port value to be shown in the app installation UI. + +* `containerPort` is the port that the app is listening on (recall that each app has it's own networking namespace). + +In more detail: + +* If the user decides to disable the SSH service, this environment variable `SSH_PORT` is absent. Applications _must_ detect this on + start up and disable these services. + +* `SSH_PORT` is set to the value of the exposed port. Should the user choose to expose the SSH server on port 6000, then the + value of SSH_PORT is 6000. + +* `defaultValue` is **only** used for display purposes in the app installation UI. This value is independent of the value + that the app is listening on. For example, the app can run an SSH server at port 22 but still recommend a value of 29418 to the user. + +* `containerPort` is the port that the app is listening on. The Cloudron runtime will _bridge_ the user chosen external port + with the app specific `containerPort`. Cloudron Apps are containerized and each app has it's own networking namespace. + As a result, different apps can have the same `containerPort` value because these values are namespaced. + +* The environment variable `SSH_PORT` may be used by the app to display external URLs. For example, the app might want to display + the SSH URL. In such a case, it would be incorrect to use the `containerPort` 22 or the `defaultValue` 29418 since this is not + the value chosen by the user. + +* `containerPort` is optional and can be omitted, in which case the bridged port numbers are the same internally and externally. + Some apps use the same variable (in their code) for listen port and user visible display strings. When packaging these apps, + it might be simpler to listen on `SSH_PORT` internally. In such cases, the app can omit the `containerPort` value and should + instead reconfigure itself to listen internally on `SSH_PORT` on each start up. + +### title + +Type: string + +Required: yes + +The `title` is the primary application title displayed on the Cloudron Store. + +Example: +``` + "title": "Gitlab" +``` + +### version + +Type: semver string + +Required: yes + +The `version` field specifies a [semver](http://semver.org/) string. The version is used by the Cloudron to compare versions and to +determine if an update is available. + +Example: +``` + "version": "1.1.0" +``` + +### website + +Type: url + +Required: yes + +The `website` field is a URL where the user can read more about the application. + +Example: +``` + "website": "https://example.com/myapp" +``` diff --git a/docs/references/recipes.md b/docs/references/recipes.md new file mode 100644 index 000000000..84d5962d1 --- /dev/null +++ b/docs/references/recipes.md @@ -0,0 +1,61 @@ +# Configuration Recipes + +## nginx + +`nginx` is often used as a reverse proxy in front of the application, to dispatch to different backend programs based on the request route or other characteristics. In such a case it is recommended to run nginx and the application through a process manager like `supervisor`. + +Example nginx supervisor configuration file: +``` +[program:nginx] +directory=/tmp +command=/usr/sbin/nginx -g "daemon off;" +user=root +autostart=true +autorestart=true +stdout_logfile=/var/log/supervisor/%(program_name)s.log +stderr_logfile=/var/log/supervisor/%(program_name)s.log +``` + +The nginx configuration, provided with the base image, can be used by adding an application specific config file under `/etc/nginx/sites-enabled/` when building the docker image. + +``` +ADD /etc/nginx/sites-enabled/ +``` + +Since the base image nginx configuration is unpatched from the ubuntu package, the application configuration has to ensure nginx is using `/run/` instead of `/var/lib/nginx/` to support the read-only filesystem nature of a Cloudron application. + +Example nginx app config file: +``` +client_body_temp_path /run/client_body; +proxy_temp_path /run/proxy_temp; +fastcgi_temp_path /run/fastcgi_temp; +scgi_temp_path /run/scgi_temp; +uwsgi_temp_path /run/uwsgi_temp; + +server { + listen 8000; + + root /app/code/dist; + + location /api/v1/ { + proxy_pass http://127.0.0.1:8001; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + proxy_read_timeout 86400; + } +} + +``` + +## supervisor + +Use this in the program's config: + +``` +[program:app] +stdout_logfile=/dev/stdout +stdout_logfile_maxbytes=0 +stderr_logfile=/dev/stderr +stderr_logfile_maxbytes=0 +``` diff --git a/docs/references/selfhosting.md b/docs/references/selfhosting.md new file mode 100644 index 000000000..cbcb9e78b --- /dev/null +++ b/docs/references/selfhosting.md @@ -0,0 +1,223 @@ +# Self host Cloudron + +The Cloudron platform can be installed on your own cloud server. The self hosted version comes with all the same features as the managed version. + +## The CLI tool + +The [cloudron tool](https://git.cloudron.io/cloudron/cloudron-cli) has a `machine` subcommand that can be used to create, update and maintain a self-hosted Cloudron. + +### Installation + +Installing the CLI tool requires node.js and npm. The CLI tool can be installed using the following command: + +``` +npm install -g cloudron +``` + +Depending on your setup, you may need to run this as root. + +You should now be able to run the `cloudron machine help` command in a shell. + + +### Machine subcommands + +``` +create Creates a new Cloudron +restore Restores a Cloudron +migrate Migrates a Cloudron +update Upgrade or updates a Cloudron +eventlog Get Cloudron eventlog +logs Get Cloudron logs +ssh Get remote SSH connection +backup Manage Cloudron backups +``` + +## AWS EC2 + +### Requirements + +To run the Cloudron on AWS, first sign up with [Amazon AWS](https://aws.amazon.com/). + +The Cloudron uses the following AWS services: + +* **EC2** for creating a virtual private server that runs the Cloudron code. +* **Route53** for DNS. The Cloudron will manage all app subdomains as well as the email related DNS records automatically. +* **S3** to store encrypted Cloudron backups. + +The minimum requirements for a Cloudron heavily depends on the apps installed. The absolute minimum required EC2 instance is `t2.small`. + +The Cloudron runs best on instances which do not have a burst mode VCPU. + +The system disk space usage of a Cloudron is around 15GB. This results in a minimum requirement of about 30GB to give some headroom for app +installations and user data. + +### Cost Estimation + +Taking the minimal requirements of hosting on EC2, with a backup retention of 2 days, the cost estimation per month is as follows: + +``` +Route53: 0.90 +EC2: 19.04 +EBS: 3.00 +S3: 1.81 +------------------------- +Total: $ 24.75/mth +``` + +For custom cost estimation, please use the [AWS Cost Calculator](http://calculator.s3.amazonaws.com/index.html) + +### Setup + +Open the AWS console and create the required resources: + +1. Create a Route53 zone for your domain. Be sure to set the Route53 nameservers for your domain in your name registrar. +2. Create a S3 bucket for backups. The bucket region *must* be the same region as where you intend to create your Cloudron (EC2). +3. Create an SSH key pair in the target region (`Key Pairs` in the left pane of the EC2 console). +4. Create AWS credentials. You can either use root **or** IAM credentials. + * For root credentials: + * In AWS Console, under your name in the menu bar, click `Security Credentials` + * Click on `Access Keys` and create a key pair. + * For IAM credentials: + * You can use the following policy to create IAM credentials: +``` +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": "route53:*", + "Resource": [ + "arn:aws:route53:::hostedzone/" + ] + }, + { + "Effect": "Allow", + "Action": "s3:*", + "Resource": [ + "arn:aws:s3:::", + "arn:aws:s3:::/*" + ] + }, + { + "Effect": "Allow", + "Action": "ec2:*", + "Resource": [ + "*" + ], + "Condition": { + "StringEquals": { + "ec2:Region": "" + } + } + } + ] +} +``` + +### Create the Cloudron + +Create the Cloudron using the `cloudron machine` command: + +``` +cloudron machine create ec2 \ + --region \ + --type t2.small \ + --disk-size 30 \ + --ssh-key \ + --access-key-id \ + --secret-access-key \ + --backup-bucket \ + --backup-key \ + --fqdn +``` + +The `--region` is the region where your Cloudron is to be created. For example, `us-west-1` for N. California and `eu-central-1` for Frankfurt. A complete list of available +regions is list here. + +The `--disk-size` parameter indicates the volume (hard disk) size to be allocated for the Cloudron. + +The `--backup-key ` will be used to encrypt all backups prior to uploading to S3. Keep that secret in a safe place, as you need it to restore your Cloudron from a backup! You can generate a random key using `pwgen -1y 64`. + +**NOTE**: The `cloudron machine create` subcommand will automatically create a corresponding VPC, subnet and security group for your Cloudron, unless `--subnet` and `--security-group` arguments are explicitly passed in. If you want to reuse existing resources, please ensure that the security group does not limit any traffic to the Cloudron since the Cloudron manages its own firewall and that the subnet has an internet gateway setup in the routing table. + +## First time setup + +Visit `https://my./setup.html` to do first time setup of your Cloudron. + +Once the setup is done, you can access the admin page in the future at `https://my.`. + +## Backups + +The Cloudron has a backup schedule of creating one once a day. In addition to regularly scheduled backups, a backup is also created if you update the Cloudron or any of the apps (in this case only the app in question will get backed up). + +Since this might result in a lot of backup data on your S3 backup bucket, we recommend adjusting the bucket properties. This can be done adding a lifecycle rule for that bucket, using the AWS console. S3 supports both permanent deletion or moving objects to the cheaper Glacier storage class based on an age attribute. With the current daily backup schedule a setting of two days should be already sufficient for most use-cases. + +You can list backups using the following command: +``` +cloudron machine backup list +``` + +## Restore + +The Cloudron can restore itself from a backup using the following command: +``` +cloudron machine create ec2 \ + --backup \ + --region \ + --type t2.small \ + --disk-size 30 \ + --ssh-key \ + --access-key-id \ + --secret-access-key \ + --backup-bucket \ + --backup-key \ + --fqdn +``` + +The backup id can be obtained by [listing the backup](/references/selfhosting.html#backups). Other arguments are similar to [Cloudron creation](/references/selfhosting.html#create-the-cloudron). Once the new instance has completely restored, you can safely terminate the old Cloudron from the AWS console. + +## Updates + +Apps installed from the Cloudron Store are updated automatically every night. + +The Cloudron platform itself updates in two ways: + +* An **update** is applied onto the running server instance. Such updates are performed every night. You can use the Cloudron UI to perform updates. + +* An **upgrade** requires a new OS image and thus has to be performed using the CLI tool. This process involves creating a new EC2 instance is created using the latest image and all the data and apps are restored. The `cloudron machine update` command can be used when an _upgrade_ is available (you will get a notification in the UI). +``` + cloudron machine update --ssh-key +``` +Once the upgrade is complete, you can safely terminate the old EC2 instance. + +The Cloudron will always make a complete backup before attempting an update or upgrade. In the unlikely case an update fails, it can be [restored](/references/selfhosting.html#restore). + +## SSH + +If you want to SSH into your Cloudron, you can +``` +ssh -p 202 -i ~/.ssh/ssh_key_name root@my. +``` + +If you are unable to connect, verify the following: +* Be sure to use the **my.** subdomain (eg. my.foobar.com). +* The SSH Key should be in PEM format. If you are using Putty PPK files, follow [this article](http://stackoverflow.com/questions/2224066/how-to-convert-ssh-keypairs-generated-using-puttygenwindows-into-key-pairs-use) to convert it to PEM format. +* The SSH Key must have correct permissions (400) set (this is a requirement of the ssh client). + +## Mail + +Spammers frequently abuse EC2 public IP addresses and as a result your Cloudron might possibly start out with a bad +reputation. The good news is that most IP based blacklisting services cool down over time. The Cloudron +sets up DNS entries for SPF, DKIM automatically and reputation should be easy to get back. + +* Once your Cloudron is ready, apply for a Reverse DNS record to be setup for your domain. You can find the AWS request +form [here](https://aws-portal.amazon.com/gp/aws/html-forms-controller/contactus/ec2-email-limit-rdns-request). + +* Check if your IP is listed in any DNSBL list [here](http://multirbl.valli.org/). In most cases, you can apply for removal +of your IP by filling out a form at the DNSBL manager site. + +* Finally, check your spam score at [mail-tester.com](https://www.mail-tester.com/). + +## Other Providers + +Currently, we do not support other cloud server provider. Please let us know at [support@cloudron.io](mailto:support@cloudron.io), if you want to see other providers supported. diff --git a/docs/references/usermanual.md b/docs/references/usermanual.md new file mode 100644 index 000000000..75379e2ec --- /dev/null +++ b/docs/references/usermanual.md @@ -0,0 +1,303 @@ +# User Manual + +## Introduction + +The Cloudron is the best way to run apps and manage users on your private server. +When we say `private`, we mean that we create a virtual server that is exclusive +to you. Each cloudron.io user gets their own server. + +You might wonder that there are many 1-click app solutions out there and what's so special +about Cloudron? Most 1-click solutions simply put code into a server and leave it at that. +There's so much more to do: +1. Configure a domain to point to your server +2. Setup SSL certificates and renew them periodically +3. Ensure app is backed up correctly +4. Ensure app is uptodate and secure +5. Have a mechanism to quickly restore the app from a backup +6. Manage users across all your apps +7. Notifications about the app status + +... and so on ... + +We made the Cloudron to dramatically lower the bar for people to run apps on servers. Just provide +a domain name, install apps and add users. All the server management listed above is completely automated. + +If you want to learn more about the secret sauce that makes the Cloudron, please read our [architecture overview](/references/architecture.html). + +## Use cases + +What can you run on a Cloudron? Here are some of the apps you can run on a Cloudron: +* RSS Reader +* Chat, IRC, Jabber servers +* Blog +* File syncing and sharing +* Code hosting +* Email + +Our list of apps is growing everyday, so be sure to [follow us on twitter](https://twitter.com/cloudron_io). + +## Activation + +When you first create the Cloudron, the setup wizard will ask you to setup an administrator +account. Don't worry, a Cloudron adminstrator doesn't need to know anything about maintaining +a server! It's the whole reason why we made the Cloudron. Being a Cloudron administrator is +more analagous to being the owner of a smartphone. You can always add more administrators to +the Cloudron from the `Users` menu item. + + + +Tip: The Cloudron administration panel is located at the `my` subdomain. You might want to bookmark +this link! + +## Apps + +### Installation + +You can install apps on the Cloudron by choosing the `App Store` menu item. Use the 'Search' bar +to search for apps. + +Clicking on app gives you information about the app. + + + +Clicking the `Install` button will show an install dialog like below: + + + +The `Location` field is the subdomain in which your app will be installed. For example, use the +`mail` location to access your web mail client or the `blog` location to access your Wordpress blog. + +Tip: You can access the apps directly on your browser using `blog.`. You don't have to +visit the Cloudron administration panel. + +`Access control` specifies who can access this app. + +* `Every Cloudron user` - Any user in your Cloudron can access the app. Initially, you are the only + user in your Cloudron. Unless you explicitly invite others, nobody else can access these apps. + Note that the term 'access' depends on the app. For a blog, this means that nobody can post new + blog posts (but anybody can view them). For a chat server, this means that nobody can access + your chat server. + +* `Restrict to groups` - Only users in the groups can access the app. + +### Updates + +All your apps automatically update as and when the application author releases an update. The Cloudron +will attempt to update around midnight of your timezone. + +Some app updates are not automatic. This can happen if a new version of the app has dropped some features +that you were relying on. In such a case, the update has to be manually approved. This is simply a matter +of clicking the `Update` button after you read about the changes. + +### Backups + +All your apps will automatically backup and those backups are stored encrypted in Amazon S3. You don't have +to do anything about it. + +### Configuration + +Apps can be reconfigured using the `Configure` dialog. Click on the wrench icon in the application grid +to bring up the following dialog: + + + +You can do the following: +* Change the location to move the app to another subdomain. Say, you want to move your blog from `blog` to `about`. +* Change who can access the app. + +Changing an app's configuration has a small downtime (usually around a minute). + +### Restore + +Apps can be restored to a previous backup by clicking on the `Restore` button. Note that restoring previous +data might also restore the previous version of the software. For example, you might be currently using +Version 5 of the app. If you restore to a backup that was made with Version 3 of the app, then the restore +operation will install Version 3 of the app. This is because the latest version may not be able to handle old data. + +### Uninstall + +You can uninstall an app by clicking the `Uninstall` button. Note that all data associated with the app will +be immediately removed from the Cloudron. App data might still persist in your old backups and the +[CLI tool](https://git.cloudron.io/cloudron/cloudron-cli) provides a way to restore from those old backups should +it be required. + +### Embedding Apps + +It is possible to embed Cloudron apps into other websites. By default, this is disabled to prevent +[Clickjacking](https://cloudron.io/blog/2016-07-15-site-embedding.html). + +You can set a website that is allowed to embed your Cloudron app using the Configure app dialog. + +## Custom domain + +When you create a Cloudron from cloudron.io, we provide a subdomain under `cloudron.me` like `girish.cloudron.me`. +Apps are available under that subdomain using a hyphenated name like `blog-girish.cloudron.me`. + +Domain names are a thing of pride and the Cloudron makes it easy to make your apps accessible from memorable locations like `blog.girish.in`. + +### Single app on a custom domain + +This approach is applicable if you desire that only a single app be accessing from a custom +domain. For this, open the app's configure dialog and choose `External Domain` in the location dropdown. + + + +This dialog will suggest you to add a `CNAME` record. Once you setup a CNAME record with your DNS provider, +the app will be accessible from that external domain. + +### Entire Cloudron on a custom domain + +This approach is applicable if you want all your apps to be accessible from subdomains of your custom domain. +For example, `blog.girish.in`, `notes.girish.in`, `owncloud.girish.in`, `mail.girish.in` and so on. This +approach is also the only way that the Cloudron supports for sending and receiving emails from your domain. + +For this, go to the 'Domains & Certs' menu item. + + + +Change the domain name to your custom domain. Currently, we require that your domain be hosted on AWS Route53. + + + +Moving to a custom domain will retain all your apps and data and will take around 15 minutes. If you require assistance with another provider, +just let us know. + +## User management + +### Users + +You can invite new users (friends, family, colleagues) with their email address from the `Users` menu. They will +receive an invite to sign up with your Cloudron. They can now access the apps that you have given them access +to. + + + +To remove a user, simply remove them from the list. Note that the removed user cannot access any app anymore. + +### Groups + +Groups provide a convenient way to restrict access to your apps. Simply add one or more users to a group +and restrict the access for an app to that group. You can create a group by using the `Groups` menu item. + + + +To set the access restriction use the app's configure dialog. + + + +## Login + +### Cloudron admin + +The Cloudron admin page is always located at the `my` subdomain of your Cloudron domain. For custom domains, +this will be like `my.girish.in`. For domains from cloudron.io, this will be like `my-girish.cloudron.me`. + +### Apps (single sign-on) + +An important feature of the Cloudron is Single Sign-On. You use the same username & password for logging in +to all your apps. No more having to manage separate set of credentials for each service! + +### Single user apps + +Some apps only work with a single user. For example, a notes app might allow only a single user to login and add +notes. For such apps, you will be prompted during installation to select the single user who can access the app. + + + +If you want multiple users to use the app independently, simply install the app multiple times to different locations. + +## Email + +The Cloudron has a built-in email server. The primary email address is the same as the username. Emails can be sent +and received from `@`. The Cloudron does not allow masquerading - one user cannot send email +pretending to be another user. + +### Receiving email (IMAP) + +Use the following settings to receive email. + + * Server Name - Use the `my` subdomain of your Cloudron + * Port - 993 + * Connection Security - TLS + * Username/password - Same as your Cloudron credentials + +### Sending email (SMTP) + +Use the following settings to send email. + + * Server Name - Use the `my` subdomain of your Cloudron + * Port - 587 + * Connection Security - STARTTLS + * Username/password - Same as your Cloudron credentials + +### Email filters (Sieve) + +Use the following settings to setup email filtering users via Manage Sieve. + + * Server Name - Use the `my` subdomain of your Cloudron + * Port - 4190 + * Connection Security - TLS + * Username/password - Same as your Cloudron credentials + +The [Rainloop](https://cloudron.io/appstore.html?app=net.rainloop.cloudronapp) and [Roundcube](https://cloudron.io/appstore.html?app=net.roundcube.cloudronapp) +apps are already pre-configured to use the above settings. + +### Aliases + +You can configure one or more aliases alongside the primary email address of each user. You can set aliases by editing the +user's settings, available behind the edit button in the user listing. Note that aliases cannot conflict with existing user names. + + + +Currently, it is not possible to login using the alias for SMTP/IMAP/Sieve services. Instead, add the alias as an identity in +your mail client but login using the Cloudron credentials. + +### Subaddresses + +Emails addressed to `+tag@` will be delivered to the `username` mailbox. You can use this feature to give out emails of the form +`username+kayak@`, `username+aws@` and so on and have them all delivered to your mailbox. + +## Graphs + +The Graphs view shows an overview of the disk and memory usage on your Cloudron. + + + +The `Disk Usage` graph shows you how much disk space you have left. Note that the Cloudron will +send the Cloudron admins an email notification when the disk is ~90% full. + +The `Apps` Memory graph shows the memory consumed by each installed app. You can click on each segment +on the graph to see the memory consumption over time in the chart below it. + +The `System` Memory graph shows the overall memory consumption on the entire Cloudron. If you see +the Free memory < 50MB frequently, you should consider upgrading to a Cloudron with more memory. + +## Activity log + +The `Activity` view shows the activity on your Cloudron. It includes information about who is using +the apps on your Cloudron and also tracks configuration changes. + + + +## Domains and SSL Certificates + +All apps on the Cloudron can only be reached by `https`. The Cloudron automatically installs and +renews certificates for your apps as needed. Should installation of certificate fail for reasons +beyond it's control, Cloudron admins will get a notification about it. + +## API Access + +All the operations listed in this manual like installing app, configuring users and groups, are +completely programmable with a [REST API](/references/api.html). + +## Moving to a larger Cloudron + +When using a Cloudron from cloudron.io, it is easy to migrate your apps and data to a bigger server. +In the `Settings` page, you can change the plan. + + + +## Command line tool + +If you are a software developer or a sysadmin, the Cloudron comes with a CLI tool that can be +used to develop custom apps for the Cloudron. Read more about it [here](https://git.cloudron.io/cloudron/cloudron-cli). diff --git a/docs/tutorials/node.md b/docs/tutorials/node.md new file mode 100644 index 000000000..73af603b6 --- /dev/null +++ b/docs/tutorials/node.md @@ -0,0 +1,621 @@ +# Overview + +This tutorial provides an introduction to developing applications +for the Cloudron using node.js. + +# Installation + +## Install CLI tool + +The Cloudron CLI tool allows you to install, configure and test apps on your Cloudron. + +Installing the CLI tool requires [node.js](https://nodejs.org/) and +[npm](https://www.npmjs.com/). You can then install the CLI tool using the following +command: + +``` + sudo npm install -g cloudron +``` + +Note: Depending on your setup, you can run the above command without `sudo`. + +## Testing your installation + +The `cloudron` command should now be available in your path. + +Let's login to the Cloudron as follows: + +``` +$ cloudron login +Cloudron Hostname: craft.selfhost.io + +Enter credentials for craft.selfhost.io: +Username: girish +Password: +Login successful. +``` + +## Your First Application + +Creating an application for Cloudron can be summarized as follows: + +1. Create a web application using any language/framework. This web application must run a HTTP server + and can optionally provide other services using custom protocols (like git, ssh, TCP etc). + +2. Create a [Dockerfile](http://docs.docker.com/engine/reference/builder/) that specifies how to create + an application ```image```. An ```image``` is essentially a bundle of the application source code + and it's dependencies. + +3. Create a [CloudronManifest.json](/references/manifest.html) file that provides essential information + about the app. This includes information required for the Cloudron Store like title, version, icon and + runtime requirements like `addons`. + +## Simple Web application + +To keep things simple, we will start by deploying a trivial node.js server running on port 8000. + +Create a new project folder `tutorial/` and add a file named `tutorial/server.js` with the following content: +```javascript +var http = require("http"); + +var server = http.createServer(function (request, response) { + response.writeHead(200, {"Content-Type": "text/plain"}); + response.end("Hello World\n"); +}); + +server.listen(8000); + +console.log("Server running at port 8000"); +``` + +## Dockerfile + +A Dockerfile contains commands to assemble an image. + +Create a file named `tutorial/Dockerfile` with the following content: + +```dockerfile +FROM cloudron/base:0.8.1 + +ADD server.js /app/code/server.js + +CMD [ "/usr/local/node-0.12.7/bin/node", "/app/code/server.js" ] +``` + +The `FROM` command specifies that we want to start off with Cloudron's [base image](/references/baseimage.html). +All Cloudron apps **must** start from this base image. + +The `ADD` command copies the source code of the app into the directory `/app/code`. +While this example only copies a single file, the ADD command can be used to copy directory trees as well. +See the [Dockerfile](https://docs.docker.com/reference/builder/#add) documentation for more details. + +The `CMD` command specifies how to run the server. There are multiple versions of node available under `/usr/local`. We +choose node v0.12.7 for our app. + +## CloudronManifest.json + +The `CloudronManifest.json` specifies + +* Information about displaying the app on the Cloudron Store. For example, + the title, author information, description etc + +* Information for installing the app on the Cloudron. This includes fields + like httpPort, tcpPorts. + +Create the CloudronManifest.json using the following command: + +``` +$ cloudron init +id: io.cloudron.tutorial # unique id for this app. use reverse domain name convention +author: John Doe # developer or company name of the for user +title: Tutorial App # Cloudron Store title of this app +description: App that uses node.js # A string or local file reference like file://DESCRIPTION.md +tagline: Changing the world one app at a time # A tag line for this app for the Cloudron Store +website: https://cloudron.io # A link to this app's website +contactEmail: support@cloudron.io # Contact email of developer or company +httPort: 8000 # The http port on which this application listens to +``` + +The above command creates a CloudronManifest.json: + +File ```tutorial/CloudronManifest.json``` + +```json +{ + "id": "io.cloudron.tutorial", + "author": "John Doe", + "title": "Tutorial App", + "description": "App that uses node.js", + "tagline": "Changing the world one app at a time", + "version": "0.0.1", + "healthCheckPath": "/", + "httpPort": 8000, + "addons": { + "localstorage": {} + }, + "minBoxVersion": "0.0.1", + "manifestVersion": 1, + "website": "https://cloudron.io", + "contactEmail": "support@cloudron.io", + "icon": "", + "mediaLinks": [] +} +``` + +You can read in more detail about each field in the [Manifest reference](/references/manifest.html). + +# Installing + +## Building + +We now have all the necessary files in place to build and deploy the app to the Cloudron. +Building creates an image of the app using the Dockerfile which can then be used to deploy +to the Cloudron. + +Building, pushing and pulling docker images is very bandwidth and CPU intensive. To alleviate this +problem, apps are built using the `build service` which uses `cloudron.io` account credentials. + +**Warning**: As of this writing, the build service uses the public Docker registry and the images that are built +can be downloaded by anyone. This means that your source code will be viewable by others. + +Initiate a build using ```cloudron build```: +``` +$ cloudron build +Building io.cloudron.tutorial@0.0.1 + +Appstore login: +Email: ramakrishnan.girish@gmail.com # cloudron.io account +Password: # Enter password +Login successful. + +Build scheduled with id 76cebfdd-7822-4f3d-af17-b3eb393ae604 +Downloading source +Building +Step 0 : FROM cloudron/base:0.8.1 + ---> 97583855cc0c +Step 1 : ADD server.js /app/code + ---> b09b97ecdfbc +Removing intermediate container 03c1e1f77acb +Step 2 : CMD /usr/local/node-0.12.7/bin/node /app/code/main.js + ---> Running in 370f59d87ab2 + ---> 53b51eabcb89 +Removing intermediate container 370f59d87ab2 +Successfully built 53b51eabcb89 +The push refers to a repository [cloudron/img-2074d69134a7e0da3d6cdf3c53e241c4] (len: 1) +Sending image list +Pushing repository cloudron/img-2074d69134a7e0da3d6cdf3c53e241c4 (1 tags) +Image already pushed, skipping 57f52d167bbb +Image successfully pushed b09b97ecdfbc +Image successfully pushed 53b51eabcb89 +Pushing tag for rev [53b51eabcb89] on {https://cdn-registry-1.docker.io/v1/repositories/cloudron/img-2074d69134a7e0da3d6cdf3c53e241c4/tags/76cebfdd-7822-4f3d-af17-b3eb393ae604} +Build succeeded +``` + +## Installing + +Now that we have built the image, we can install our latest build on the Cloudron +using the following command: + +``` +$ cloudron install +Using cloudron craft.selfhost.io +Using build 76cebfdd-7822-4f3d-af17-b3eb393ae604 from 1 hour ago +Location: tutorial # This is the location into which the application installs +App is being installed with id: 4dedd3bb-4bae-41ef-9f32-7f938995f85e + + => Waiting to start installation + => Registering subdomain . + => Verifying manifest . + => Downloading image .............. + => Creating volume . + => Creating container + => Setting up collectd profile ................ + => Waiting for DNS propagation ... + +App is installed. +``` + +This makes the app available at https://tutorial-craft.selfhost.io. + +Open the app in your default browser: +``` +cloudron open +``` + +You should see `Hello World`. + +# Testing + +The application testing cycle involves `cloudron build` and `cloudron install`. +Note that `cloudron install` updates an existing app in place. + +You can view the logs using `cloudron logs`. When the app is running you can follow the logs +using `cloudron logs -f`. + +For example, you can see the console.log output in our server.js with the command below: + +``` +$ cloudron logs +Using cloudron craft.selfhost.io +2015-05-08T03:28:40.233940616Z Server running at port 8000 +``` + +It is also possible to run a *shell* and *execute* arbitrary commands in the context of the application +process by using `cloudron exec`. By default, exec simply drops you into an interactive bash shell with +which you can inspect the file system and the environment. + +``` +$ cloudron exec +``` + +You can also execute arbitrary commands: +``` +$ cloudron exec env # display the env variables that your app is running with +``` + +# Storing data + +For file system storage, an app can use the `localstorage` addon to store data under `/app/data`. +When the `localstorage` addon is active, any data under /app/data is automatically backed up. When an +app is updated, /app/data already contains the data generated by the previous version. + +*Note*: For convenience, the initial CloudronManifest.json generated by `cloudron init` already contains this +addon. + +Let us put this theory into action by saving a *visit counter* as a file. +*server.js* has been modified to count the number of visitors on the site by storing a counter +in a file named ```counter.dat```. + +File ```tutorial/server.js``` + +```javascript +var http = require('http'), + fs = require('fs'), + util = require('util'); + +var COUNTER_FILE = '/app/data/counter.dat'; + +var server = http.createServer(function (request, response) { + var counter = 0; + if (fs.existsSync(COUNTER_FILE)) { + // read existing counter if it exists + counter = parseInt(fs.readFileSync(COUNTER_FILE, 'utf8'), 10); + } + + response.writeHead(200, {"Content-Type": "text/plain"}); + response.end(util.format("Hello World. %s visitors have visited this page\n", counter)); + ++counter; // bump the counter + fs.writeFileSync(COUNTER_FILE, counter + '', 'utf8'); // save back counter +}); + +server.listen(8000); + +console.log("Server running at port 8000"); +``` + +Now every time you refresh the page you will notice that the counter bumps up. You will +also notice that if you make changes to the app and do a `cloudron install`, the `counter.dat` +is *retained* across updates. + +# Database + +Most web applications require a database of some form. In theory, it is possible to run any +database you want as part of the application image. This is, however, a waste of server resources +should every app runs it's own database server. + +To solve this, the Cloudron provides shareable resources like databases in form of ```addons```. +The database server is managed by the Cloudron and the application simply needs to request access to +the database in the CloudronManifest.json. While the database server itself is a shared resource, the +databases are exclusive to the application. Each database is password protected and accessible only +to the application. Databases and tables can be configured without restriction as the application +requires. + +Cloudron currently provides `mysql`, `postgresql`, `mongodb`, `redis` database addons. + +For this tutorial, let us try to save the counter in `redis` addon. For this, we make use of the +[redis](https://www.npmjs.com/package/redis) module. + +Since this is a node.js app, let's add a very basic `package.json` containing the `redis` module dependency. + +File `tutorial/package.json` +```json +{ + "name": "tutorial", + "version": "1.0.0", + "dependencies": { + "redis": "^0.12.1" + } +} +``` + +and modify our Dockerfile to look like this: + +File `tutorial/Dockerfile` + +```dockerfile +FROM cloudron/base:0.8.1 + +ADD server.js /app/code/server.js +ADD package.json /app/code/package.json + +WORKDIR /app/code +RUN npm install --production + +CMD [ "/usr/local/node-0.12.7/bin/node", "/app/code/server.js" ] +``` + +Notice the new `RUN` command which installs the node module dependencies in package.json using `npm install`. + +Since we want to use redis, we have to modify the CloudronManifest.json to make redis available for this app. + +File `tutorial/CloudronManifest.json` + +```json +{ + "id": "io.cloudron.tutorial", + "author": "John Doe", + "title": "Tutorial App", + "description": "App that uses node.js", + "tagline": "Changing the world one app at a time", + "version": "0.0.1", + "healthCheckPath": "/", + "httpPort": 8000, + "addons": { + "localstorage": {}, + "redis": {} + }, + "minBoxVersion": "0.0.1", + "manifestVersion": 1, + "website": "https://cloudron.io", + "contactEmail": "support@cloudron.io", + "icon": "", + "mediaLinks": [] +} +``` + +When the application runs, environment variables `REDIS_HOST`, `REDIS_PORT` and +`REDIS_PASSWORD` are injected. You can read about the environment variables in the +[Redis reference](/references/addons.html#redis). + +Let's change `server.js` to use redis instead of file backed counting: + +File ```tutorial/server.js``` + +```javascript +var http = require('http'), + fs = require('fs'), + util = require('util'), + redis = require('redis'); + +var redisClient = redis.createClient(process.env.REDIS_PORT, process.env.REDIS_HOST); +redisClient.auth(process.env.REDIS_PASSWORD); +redisClient.on("error", function (err) { + console.log("Redis Client Error " + err); +}); + +var COUNTER_KEY = 'counter'; + +var server = http.createServer(function (request, response) { + redisClient.get(COUNTER_KEY, function (err, reply) { + var counter = (!err && reply) ? parseInt(reply, 10) : 0; + response.writeHead(200, {"Content-Type": "text/plain"}); + response.end(util.format("Hello World. %s visitors have visited this page\n", counter)); + redisClient.incr(COUNTER_KEY); + }); +}); + +server.listen(8000); + +console.log("Server running at port 8000"); +``` + +Simply `cloudron build` and `cloudron install` to test your app! + +# Authentication + +The Cloudron has a centralized panel for managing users and groups. Apps can integrate Single Sign-On +authentication using LDAP or OAuth. + +Note that apps that are single user can skip Single Sign-On support. The Cloudron implements an `OAuth +proxy` (accessed through the app configuration dialog) that optionally lets the Cloudron admin make the +app visible only for logged in users. + +## LDAP + +Let's start out by adding the [ldap](/references/addons.html#ldap) addon to the manifest. + +File `tutorial/CloudronManifest.json` +```json +{ + "id": "io.cloudron.tutorial", + "author": "John Doe", + "title": "Tutorial App", + "description": "App that uses node.js", + "tagline": "Changing the world one app at a time", + "version": "0.0.1", + "healthCheckPath": "/", + "httpPort": 8000, + "addons": { + "localstorage": {}, + "ldap": {} + }, + "minBoxVersion": "0.0.1", + "manifestVersion": 1, + "website": "https://cloudron.io", + "contactEmail": "support@cloudron.io", + "icon": "", + "mediaLinks": [] +} +``` + +Building and installing the app shows that the app gets new LDAP specific environment variables. + +``` +$ cloudron build +$ cloudron install +$ cloudron exec env | grep LDAP +LDAP_SERVER=172.17.42.1 +LDAP_PORT=3002 +LDAP_URL=ldap://172.17.42.1:3002 +LDAP_USERS_BASE_DN=ou=users,dc=cloudron +LDAP_GROUPS_BASE_DN=ou=groups,dc=cloudron +``` + +Let's test the environment variables to use by using the [ldapjs](http://www.ldapjs.org) npm module. +We start by adding ldapjs to package.json. + +File `tutorial/package.json` +```json +{ + "name": "tutorial", + "version": "1.0.0", + "dependencies": { + "ldapjs": "^0.7.1" + } +} +``` + +The server code has been modified to authenticate using the `X-Username` and `X-Password` headers for +any path other than '/'. + +File `tutorial/server.js` +```javascript +var http = require("http"), + ldap = require('ldapjs'); + +var ldapClient = ldap.createClient({ url: process.env.LDAP_URL }); + +var server = http.createServer(function (request, response) { + if (request.url === '/') { + response.writeHead(200, {"Content-Type": "text/plain"}); + return response.end(); + } + + var username = request.headers['x-username'] || ''; + var password = request.headers['x-password'] || ''; + var ldapDn = 'cn=' + username + ',' + process.env.LDAP_USERS_BASE_DN; + + ldapClient.bind(ldapDn, password, function (error) { + if (error) { + response.writeHead(401, {"Content-Type": "text/plain"}); + response.end('Failed to authenticate: ' + error); + } else { + response.writeHead(200, {"Content-Type": "text/plain"}); + response.end('Successfully authenticated'); + } + }); +}); + +server.listen(8000); + +console.log("Server running at port 8000"); +``` + +Once we have used `cloudron build` and `cloudron install`, you can use `curl` to test +credentials as follows: + +```bash + # Test with various credentials here. Your cloudon admin username and password should succeed. + curl -X 'X-Username: admin' -X 'X-Password: pass' https://tutorial-craft.selfhost.io/login +``` + +## OAuth + +An app can integrate with OAuth 2.0 Authorization code grant flow by adding +[oauth](/references/addons.html#oauth) to CloudronManifest.json `addons` section. + +Doing so will get the following environment variables: +``` +$ cloudron exec env +OAUTH_CLIENT_ID=cid-addon-4089f65a-2adb-49d2-a6d1-e519b7d85e8d +OAUTH_CLIENT_SECRET=5af99a9633283aa15f5e6df4a108ff57f82064e4845de8bce8ad3af54dfa9dda +OAUTH_ORIGIN=https://my-craft.selfhost.io +API_ORIGIN=https://my-craft.selfhost.io +HOSTNAME=tutorial-craft.selfhost.io +``` + +OAuth Authorization code grant flow works as follows: +* App starts the flow by redirecting the user to Cloudron authorization endpoint of the following format: +``` +https://API_ORIGIN/api/v1/oauth/dialog/authorize?response_type=code&client_id=OAUTH_CLIENT_ID&redirect_uri=CALLBACK_URL&scope=profile +``` + + In the above URL, API_ORIGIN and OAUTH_CLIENT_ID are environment variables. CALLBACK_URL is a url of the app +to which the user will be redirected back to after successful authentication. CALLBACK_URL has to have the +same origin as the app. + +* The Cloudron OAuth server authenticates the user (using a password form) at the above URL. It also establishes +that the user grants the client's access request. + +* If the user authenticated successfully, it will redirect the browser to CALLBACK_URL with a `code` query parameter. + +* The app can exchange the `code` above for a `access token` by using the `OAUTH_CLIENT_SECRET`. It does so by making + a _POST_ request to the following url: +``` +https://API_ORIGIN/api/v1/oauth/token?response_type=token&client_id=OAUTH_CLIENT_ID +``` +with the following request body (json): +```json +{ + "grant_type": "authorization_code", + "code": "", + "redirect_uri": "https://", + "client_id": "", + "client_secret": "" +} +``` + + In the above URL, API_ORIGIN, OAUTH_CLIENT_ID and HOSTNAME are environment variables. The response contains +the `access_token` in the body. + +* The `access_token` can be used to get the [user's profile](/references/api.html#profile) using the following url: +``` +https://API_ORIGIN/api/v1/profile?access_token=ACCESS_TOKEN +``` + + The `access_token` may also be provided in the `Authorization` header as `Bearer: `. + +An implementation of the above OAuth logic is at [ircd-app](https://github.com/cloudron-io/ircd-app/blob/master/settings/app.js). + +The following libraries implement Cloudron OAuth for Ruby and Javascript. + + * [omniauth-cloudron](https://github.com/cloudron-io/omniauth-cloudron) + * [passport-cloudron](https://github.com/cloudron-io/passport-cloudron) + +# Beta Testing + +Once your app is ready, you can upload it to the store for `beta testing` by +other Cloudron users. This can be done using: + +``` + cloudron upload +``` + +The app should now be visible in the Store view of your cloudron under +the 'Testing' section. You can check if the icon, description and other details +appear correctly. + +Other Cloudron users can install your app on their Cloudron's using +`cloudron install --appstore-id `. Note that this currently +requires your beta testers to install the CLI tool and put their Cloudron in +developer mode. + +# Publishing + +Once you are satisfied with the beta testing, you can submit it for review. + +``` + cloudron submit +``` + +The cloudron.io team will review the app and publish the app to the store. + +# Next steps + +Congratulations! You are now well equipped to build web applications for the Cloudron. + +# Samples + + * [Lets Chat](https://github.com/cloudron-io/letschat-app) + * [Haste bin](https://github.com/cloudron-io/haste-app) + * [Pasteboard](https://github.com/cloudron-io/pasteboard-app) diff --git a/docs/tutorials/packaging.md b/docs/tutorials/packaging.md new file mode 100644 index 000000000..b75f2f118 --- /dev/null +++ b/docs/tutorials/packaging.md @@ -0,0 +1,483 @@ +# Overview + +This tutorial outlines how to package an existing web application for the Cloudron. + +If you are aware of Docker and Heroku, you should feel at home packaging for the +Cloudron. Roughly, the steps involved are: + +* Create a Dockerfile for your application. If your application already has + a Dockerfile, you should able to reuse most of it. By virtue of Docker, the Cloudron + is able to run apps written in any language/framework. + +* Create a CloudronManifest.json that provides information like title, author, description + etc. You can also specify the addons (like database) required + to run your app. When the app runs on the Cloudron, it will have environment + variables set for connecting to the addon. + +* Test the app on your Cloudron with the CLI tool. + +* Optionally, submit the app to [Cloudron Store](/appstore.html). + +# Prerequisites + +## Install CLI tool + +The Cloudron CLI tool allows you to install, configure and test apps on your Cloudron. + +Installing the CLI tool requires [node.js](https://nodejs.org/) and +[npm](https://www.npmjs.com/). You can then install the CLI tool using the following +command: + +``` + sudo npm install -g cloudron +``` + +Note: Depending on your setup, you can run the above command without `sudo`. + +## Login to Cloudron + +The `cloudron` command should now be available in your path. + +You can login to your Cloudron now: + +``` +$ cloudron login +Cloudron Hostname: craft.selfhost.io + +Enter credentials for craft.selfhost.io: +Username: girish +Password: +Login successful. +``` + +# Basic app + +We will first package a very simple app to understand how the packaging works. +You can clone this app from https://git.cloudron.io/cloudron/tutorial-basic. + +## The server + +The basic app server is a very simple HTTP server that runs on port 8000. +While the server in this tutorial uses node.js, you can write your server +in any language you want. + +```server.js +var http = require("http"); + +var server = http.createServer(function (request, response) { + response.writeHead(200, {"Content-Type": "text/plain"}); + response.end("Hello World\n"); +}); + +server.listen(8000); + +console.log("Server running at port 8000"); +``` + +## Dockerfile + +The Dockerfile contains instructions on how to create an image for your application. + +```Dockerfile +FROM cloudron/base:0.8.1 + +ADD server.js /app/code/server.js + +CMD [ "/usr/local/node-4.2.1/bin/node", "/app/code/server.js" ] +``` + +The `FROM` command specifies that we want to start off with Cloudron's [base image](/references/baseimage.html). +All Cloudron apps **must** start from this base image. This approach conserves space on the Cloudron since +Docker images tend to be quiet large. + +The `ADD` command copies the source code of the app into the directory `/app/code`. There is nothing special +about the `/app/code` directory and it is merely a convention we use to store the application code. + +The `CMD` command specifies how to run the server. The base image already contains many different versions of +node.js. We use Node 4.2.1 here. + +This Dockerfile can be built and run locally as: +``` +docker build -t tutorial . +docker run -p 8000:8000 -ti tutorial +``` + +## Manifest + +The `CloudronManifest.json` specifies + +* Information for installing and running the app on the Cloudron. This includes fields like addons, httpPort, tcpPorts. + +* Information about displaying the app on the Cloudron Store. For example, fields like title, author, description. + +Create the CloudronManifest.json using `cloudron init` as follows: + +``` +$ cloudron init +id: io.cloudron.tutorial # unique id for this app. use reverse domain name convention +author: John Doe # developer or company name of the for user +title: Tutorial App # Cloudron Store title of this app +description: App that uses node.js # A string or local file reference like file://DESCRIPTION.md +tagline: Changing the world one app at a time # A tag line for this app for the Cloudron Store +website: https://cloudron.io # A link to this app's website +contactEmail: support@cloudron.io # Contact email of developer or company +httPort: 8000 # The http port on which this application listens to +``` + +The above command creates a CloudronManifest.json: + +File ```tutorial/CloudronManifest.json``` + +```json +{ + "id": "io.cloudron.tutorial", + "title": "Tutorial App", + "author": "John Doe", + "description": "file://DESCRIPTION.md", + "changelog": "file://CHANGELOG", + "tagline": "Changing the world one app at a time", + "version": "0.0.1", + "healthCheckPath": "/", + "httpPort": 8000, + "addons": { + "localstorage": {} + }, + "manifestVersion": 1, + "website": "https://cloudron.io", + "contactEmail": "support@cloudron.io", + "icon": "", + "tags": [ + "changme" + ], + "mediaLinks": [ ] +} +``` + +You can read in more detail about each field in the [Manifest reference](/references/manifest.html). The +`localstorage` addon allows the app to store files in `/app/data`. We will explore addons further further +down in this tutorial. + +Additional files created by `init` are: +* `DESCRIPTION.md` - A markdown file providing description of the app for the Cloudron Store. +* `CHANGELOG` - A file containing change information for each version released to the Cloudron Store. This + information is shown when the user updates the app. + +# Installing + +We now have all the necessary files in place to build and deploy the app to the Cloudron. + +## Building + +Building, pushing and pulling docker images can be very bandwidth and CPU intensive. To alleviate this +problem, apps are built using the `build service` which uses `cloudron.io` account credentials. + +**Warning**: As of this writing, the build service uses the public Docker registry and the images that are built +can be downloaded by anyone. This means that your source code will be viewable by others. + +Initiate a build using ```cloudron build```: +``` +$ cloudron build +Building io.cloudron.tutorial@0.0.1 + +Appstore login: +Email: ramakrishnan.girish@gmail.com # cloudron.io account +Password: # Enter password +Login successful. + +Build scheduled with id e7706847-f2e3-4ba2-9638-3f334a9453a5 +Waiting for build to begin, this may take a bit... +Downloading source +Building +Step 1 : FROM cloudron/base:0.8.1 + ---> be9fc6312b2d +Step 2 : ADD server.js /app/code/server.js + ---> 10513e428d7a +Removing intermediate container 574573f6ed1c +Step 3 : CMD /usr/local/node-4.2.1/bin/node /app/code/server.js + ---> Running in b541d149b6b9 + ---> 51aa796ea6e5 +Removing intermediate container b541d149b6b9 +Successfully built 51aa796ea6e5 +Pushing +The push refers to a repository [docker.io/cloudron/img-062037096d69bbf3ffb5b9316ad89cb9] (len: 1) +Pushed 51aa796ea6e5 +Pushed 10513e428d7a +Image already exists be9fc6312b2d +Image already exists a0261a2a7c75 +Image already exists f9d4f0f1eeed +Image already exists 2b650158d5d8 +e7706847-f2e3-4ba2-9638-3f334a9453a5: digest: sha256:8241d68b65874496191106ecf2ee8f3df2e05a953cd90ff074a6f8815a49389c size: 26098 +Build succeeded +Success +``` + +## Installing + +Now that we have built the image, we can install our latest build on the Cloudron +using the following command: + +``` +$ cloudron install +Using cloudron craft.selfhost.io +Using build 76cebfdd-7822-4f3d-af17-b3eb393ae604 from 1 hour ago +Location: tutorial # This is the location into which the application installs +App is being installed with id: 4dedd3bb-4bae-41ef-9f32-7f938995f85e + + => Waiting to start installation + => Registering subdomain . + => Verifying manifest . + => Downloading image .............. + => Creating volume . + => Creating container + => Setting up collectd profile ................ + => Waiting for DNS propagation ... + +App is installed. +``` + +Open the app in your default browser: +``` +cloudron open +``` + +You should see `Hello World`. + +# Testing + +The application testing cycle involves `cloudron build` and `cloudron install`. +Note that `cloudron install` updates an existing app in place. + +You can view the logs using `cloudron logs`. When the app is running you can follow the logs +using `cloudron logs -f`. + +For example, you can see the console.log output in our server.js with the command below: + +``` +$ cloudron logs +Using cloudron craft.selfhost.io +16:44:11 [main] Server running at port 8000 +``` + +It is also possible to run a *shell* and *execute* arbitrary commands in the context of the application +process by using `cloudron exec`. By default, exec simply drops you into an interactive bash shell with +which you can inspect the file system and the environment. + +``` +$ cloudron exec +``` + +You can also execute arbitrary commands: +``` +$ cloudron exec env # display the env variables that your app is running with +``` + +### DevelopmentMode + +When debugging complex startup scripts, one can specify `"developmentMode": true,` in the CloudronManifest.json. +This will ignore the `RUN` command, specified in the Dockerfile and allows the developer to interactively test +the startup scripts using `cloudron exec`. + +**Note:** that an app running in this mode has full read/write access to the filesystem and all memory limits are lifted. + + +# Addons + +## Filesystem + +The application container created on the Cloudron has a `readonly` file system. Writing to any location +other than the below will result in an error: + +* `/tmp` - Use this location for temporary files. The Cloudron will cleanup any files in this directory + periodically. + +* `/run` - Use this location for runtime configuration and dynamic data. These files should not be expected + to persist across application restarts (for example, after an update or a crash). + +* `/app/data` - Use this location to store application data that is to be backed up. To use this location, + you must use the [localstorage](/references/addons.html#localstorage) addon. For convenience, the initial CloudronManifest.json generated by + `cloudron init` already contains this addon. + +## Database + +Most web applications require a database of some form. In theory, it is possible to run any +database you want as part of the application image. This is, however, a waste of server resources +should every app runs it's own database server. + +Cloudron currently provides [mysql](/references/addons.html#mysql), [postgresql](/references/addons.html#postgresql), +[mongodb](/references/addons.html#mongodb), [redis](/references/addons.html#redis) database addons. When choosing +these addons, the Cloudron will inject environment variables that contain information on how to connect +to the addon. + +See https://git.cloudron.io/cloudron/tutorial-redis for a simple example of how redis can be used by +an application. The server simply uses the environment variables to connect to redis. + +## Email + +Cloudron applications can send email using the `sendmail` addon. Using the `sendmail` addon provides +the SMTP server and authentication credentials in environment variables. + +Cloudron applications can also receive mail via IMAP using the `recvmail` addon. + +## Authentication + +The Cloudron has a centralized panel for managing users and groups. Apps can integrate Single Sign-On +authentication using LDAP or OAuth. + +Apps can integrate with the Cloudron authentication system using LDAP, OAuth or Simple Auth. See the +[authentication](/references/authentication.html) reference page for more details. + +See https://git.cloudron.io/cloudron/tutorial-ldap for a simple example of how to authenticate via LDAP. + +For apps that are single user can skip Single Sign-On support by setting the `"singleUser": true` +in the manifest. By doing so, the Cloudron will installer will show a dialog to choose a user. + +For app that have no user management at all, the Cloudron implements an `OAuth proxy` that +optionally lets the Cloudron admin make the app visible only for logged in users. + +# Best practices + +## No Setup + +A Cloudron app is meant to instantly usable after installation. For this reason, Cloudron apps must not +show any setup screen after installation and should simply choose reasonable defaults. + +Databases, email configuration should be automatically picked up from the environment variables using +addons. + +## Dockerfile + +The app is run as a read-only docker container. Because of this: +* Install any required packages in the Dockerfile. +* Create static configuration files in the Dockerfile. +* Create symlinks to dynamic configuration files under /run in the Dockerfile. + +## Process manager + +Docker supports restarting processes natively. Should your application crash, it will be restarted +automatically. If your application is a single process, you do not require any process manager. + +Use supervisor, pm2 or any of the other process managers if you application has more then one component. +This **excludes** web servers like apache, nginx which can already manage their children by themselves. +Be sure to pick a process manager that forwards signals to child processes. + +## Automatic updates + +Some apps support automatic updates by overwriting themselves. A Cloudron app cannot overwrite itself +because of the read-only file system. For this reason, disable auto updates for app and let updates be +triggered through the Cloudron Store. This ties in better to the Cloudron's update and restore approach +should something go wrong with the update. + +## Logging + +Cloudron applications stream their logs to stdout and stderr. In practice, this ideal is hard to achieve. +Some programs like apache simply don't log to stdout. In those cases, simply log to `/tmp` or `/run`. + +Logging to stdout has many advantages: +* App does not need to rotate logs and the Cloudron takes care of managing logs. +* App does not need special mechanism to release log file handles (on a log rotate). +* Integrates better with tooling like cloudron cli. + +## Memory + +By default, applications get 200MB RAM (including swap). This can be changed using the `memoryLimit` +field in the manifest. + +Design your application runtime for concurrent use by 50 users. The Cloudron is not designed for +concurrent access by 100s or 1000s of users. + +## Authentication + +Apps should integrate with one of the [authentication strategies](/references/authentication.html). +This saves the user from having to manage separate set of credentials for each app. + +## Startup Script + +Many apps do not launch the server directly, as we did in our basic example. Instead, they execute +a `start.sh` script (named so by convention) which launches the server. Before starting the server, +the `start.sh` script does the following: + + * When using the `localstorage` addon, it changes the ownership of files in `/app/data` as desired using `chown`. This + is necessary because file permissions may not be correctly preserved across backup, restore, application and base image + updates. + + * Addon information (mail, database) exposed as environment are subject to change across restarts and an application + must use these values directly (i.e not cache them across restarts). For this reason, it usually regenerates + any config files with the current database settings on each invocation. + + * Finally, it starts the server as a non-root user. + +The app's main process must handle SIGTERM and forward it as required to child processes. bash does not +automatically forward signals to child processes. For this reason, when using a startup shell script, +remember to use exec as the last line. Doing so will replace bash with your program and allows +your program to handle signals as required. + +# Beta Testing + +Once your app is ready, you can upload it to the store for `beta testing` by +other Cloudron users. This can be done using: + +``` + cloudron upload +``` + +The app should now be visible in the Store view of your cloudron under +the 'Testing' section. You can check if the icon, description and other details +appear correctly. + +Other Cloudron users can install your app on their Cloudron's using +`cloudron install --appstore-id `. + +# Publishing + +Once you are satisfied with the beta testing, you can submit it for review. + +``` + cloudron submit +``` + +The cloudron.io team will review the app and publish the app to the store. + +# Updating the app + +## Versioning + +To create an update for an app, simply bump up the [semver version](/references/manifest.html#version) field in +the manifest and publish a new version to the store. + +The Cloudron chooses the next app version to update to based on the following algorithm: +* Choose the maximum `patch` version matching the app's current `major` and `minor` version. +* Failing the above, choose the maximum patch version of the next minor version matching the app's current `major` version. +* Failing the above, choose the maximum patch and minor version of the next major version + +For example, let's assume the versions 1.1.3, 1.1.4, 1.1.5, 1.2.4, 1.2.6, 1.3.0, 2.0.0 are published. + +* If the app is running 1.1.3, then app will directly update to 1.1.5 (skipping 1.1.4) +* Once in 1.1.5, the app will update to 1.2.6 (skipping 1.2.4) +* Once in 1.2.6, the app will update to 1.3.0 +* Once in 1.3.0, the app will update to 2.0.0 + +The Cloudron admins get notified by email for any major or minor app releases. + +## Failed updates + +The Cloudron always makes a backup of the app before making an update. Should the +update fail, the user can restore to the backup (which will also restore the app's +code to the previous version). + +# Cloudron Button + +The [Cloudron Button](/references/button.html) allows anyone to install your application with the click of a button +on their Cloudron. + +The button can be added to just about any website including the application's website +and README.md files in GitHub repositories. + +# Next steps + +Congratulations! You are now well equipped to build web applications for the Cloudron. + +You can see some examples of how real apps are packaged here: + + * [Lets Chat](https://git.cloudron.io/cloudron/letschat-app) + * [Haste bin](https://git.cloudron.io/cloudron/haste-app) + * [Pasteboard](https://git.cloudron.io/cloudron/pasteboard-app)