wangchunlin
/
cvat
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Feature/update https setup readme (#2656)

Browse Source 
* updated README section HTTPS deploy

* updated CHANGELOG.md

* removed security headers that brokes kibana

* fixed codacy warnings in installation.md

* updated README section HTTPS deploy
updated CHANGELOG.md

removed security headers that brokes kibana


fixed codacy warnings in installation.md

* updated TOC
main
Vladimir Skubriev 5 years ago committed by GitHub
parent 30678f9726
commit 2e9b17afad
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 92 additions and 81 deletions
Show Stats Download Patch File Download Diff File

3
CHANGELOG.md
Unescape Escape View File

@ -12,8 +12,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- CVAT-3D: support lidar data on the server side (<https://github.com/openvinotoolkit/cvat/pull/2534>) - CVAT-3D: support lidar data on the server side (<https://github.com/openvinotoolkit/cvat/pull/2534>)
### Changed ### Changed
- Updated HTTPS install README section (cleanup and described more robust deploy)
-
### Deprecated ### Deprecated

170
cvat/apps/documentation/installation.md
Unescape Escape View File

@ -10,14 +10,13 @@
- [Advanced settings](#advanced-settings) - [Advanced settings](#advanced-settings)
- [Share path](#share-path) - [Share path](#share-path)
- [Email verification](#email-verification) - [Email verification](#email-verification)
- [Serving over HTTPS](#serving-over-https) - [Deploy secure CVAT instance with HTTPS](#deploy-secure-cvat-instance-with-https)
- [Prerequisites](#prerequisites) - [Prerequisites](#prerequisites)
- [Roadmap](#roadmap) - [Roadmap](#roadmap)
- [Step-by-step instructions](#step-by-step-instructions) - [Step-by-step instructions](#step-by-step-instructions)
- [1. Make the proxy listen on standard port 80 and prepare nginx for the ACME challenge via webroot method](#1-make-the-proxy-listen-on-standard-port-80-and-prepare-nginx-for-the-acme-challenge-via-webroot-method) - [1. Make the proxy listen on 80 and 443 ports](#1-make-the-proxy-listen-on-80-and-443-ports)
- [2. Setting up HTTPS with `acme.sh` helper](#2-setting-up-https-with-acmesh-helper) - [2. Issue a certificate and run HTTPS versions with `acme.sh` helper](#2-issue-a-certificate-and-run-https-versions-with-acmesh-helper)
- [Create certificate files using an ACME challenge on docker host](#create-certificate-files-using-an-acme-challenge-on-docker-host) - [Create certificate files using an ACME challenge on docker host](#create-certificate-files-using-an-acme-challenge-on-docker-host)
# Quick installation guide # Quick installation guide
Before you can use CVAT, you’ll need to get it installed. The document below Before you can use CVAT, you’ll need to get it installed. The document below
@ -373,45 +372,46 @@ This depends on the email server you are using and is not covered in this tutori
[Django SMTP backend configuration](https://docs.djangoproject.com/en/3.1/topics/email/#django.core.mail.backends.smtp.EmailBackend) [Django SMTP backend configuration](https://docs.djangoproject.com/en/3.1/topics/email/#django.core.mail.backends.smtp.EmailBackend)
for details. for details.
### Serving over HTTPS ### Deploy secure CVAT instance with HTTPS
We will add [letsencrypt.org](https://letsencrypt.org/) issued certificate to secure Certificates (issued by let's encrypt) to cloud instance.
our server connection.
#### Prerequisites #### Prerequisites
We assume that We assume that:
- you have a virtual instance (machine) in the cloud provider with docker installed;
- there is no root permissions required if user is in docker group;
- there is no services listen 80 and 443 tcp ports on virtual instance.
- you have sudo access on your server machine, There are multiple approaches. Our approach suggests:
- you have an IP address to use for remote access, and
- that the local CVAT installation works on your server.
If this is not the case, please complete the steps in the installation manual first. - easy setup automatic certificate updates;
- leave certificates in safe place on docker host (protect from `docker-compose down` cleanup);
- no unnecessary certificate files copying between container and host.
#### Roadmap #### Roadmap
We will go through the following sequence of steps to get CVAT over HTTPS: We will go through the following sequence of steps to get CVAT over HTTPS:
- Setup containers on default 80/tcp port. Checkin and then down the containers. - Install [acme.sh](https://github.com/acmesh-official/acme.sh) on the virtual instance (docker host).
- Configure Nginx to pass one of the [ACME challenges](https://letsencrypt.org/docs/challenge-types/) - webroot. - Configure Nginx site template `HOME/cvat/cvat_proxy/conf.d/cvat.conf.template` used in `cvat_proxy` container.
- Create the certificate files using [acme.sh](https://github.com/acmesh-official/acme.sh). - Deploy CVAT services in the most common way with docker-compose utilizes default HTTP scheme.
- Reconfigure Nginx to serve over HTTPS and map CVAT to Docker Compose port 443. - Create the https certificates with `acme.sh` client.
- Reconfigure Nginx to serve over HTTPS.
- Make sure that certificates will be able to automatically update via cron job.
#### Step-by-step instructions #### Step-by-step instructions
##### 1. Make the proxy listen on standard port 80 and prepare nginx for the ACME challenge via webroot method ##### 1. Make the proxy listen on 80 and 443 ports
> The configuration assumes that on the docker host there will be only one instance of the CVAT site listens for incoming connections on 80 and 443 port. Also redirecting everything that does not concern renewal of certificates to the site via secure HTTPS protocol.
Let's assume the server will be at `my-cvat-server.org`. Prepare nginx for the ACME challenge via webroot method
Point you shell in cvat repository directory, usually `cd $HOME/cvat`: Let's assume the server domain name is `CVAT.example.com`.
Add the following into your `docker-compose.override.yml`, replacing `my-cvat-server.org` with your own IP address. This file lives in the same directory as `docker-compose.yml`. Clone repo and point you shell in cvat repository directory, usually `cd $HOME/cvat`:
Create the required directories for letsencrypt webroot operation and acme folder passthrough. Install and create the required directories for letsencrypt webroot operation and acme folder passthrough.
Now restart the containers with new configurations updated in `docker-compose.override.yml`
```bash ```bash
# on the docker host # on the docker host
@ -423,29 +423,30 @@ curl https://get.acme.sh | sh
mkdir -p $HOME/cvat/letsencrypt-webroot/.well-known/acme-challenge mkdir -p $HOME/cvat/letsencrypt-webroot/.well-known/acme-challenge
``` ```
Create `docker-compose.override.yml` in repo root like follows:
> modify CVAT_HOST with your own domain name
> (nginx tests the request’s header field “Host” to determine which server the request should be routed to)
```yaml ```yaml
# docker-compose.override.yml
version: '3.3' version: '3.3'
services: services:
cvat_proxy: cvat_proxy:
environment: environment:
CVAT_HOST: my-cvat-server.org CVAT_HOST: CVAT.example.com
ports: ports:
- '80:80' - '80:80'
- '443:443' - '443:443'
volumes: volumes:
- ./letsencrypt-webroot:/var/tmp/letsencrypt-webroot - ./letsencrypt-webroot:/var/tmp/letsencrypt-webroot
- /root/.acme.sh:/root/.acme.sh - /etc/ssl/private:/etc/ssl/private
cvat: cvat:
environment: environment:
ALLOWED_HOSTS: '*' ALLOWED_HOSTS: '*'
``` ```
This will enable serving `http://my-cvat-server.org/.well-known/acme-challenge/`
route from `/var/tmp/letsencrypt-webroot` directory on the container's filesystem which is bind mounted from docker host `$HOME/cvat/letsencrypt-webroot`. That volume needed for issue and renewing certificates only.
Update a CVAT site proxy template `$HOME/cvat/cvat_proxy/conf.d/cvat.conf.template` on docker(system) host. Site config updates from this template each time `cvat_proxy` container start. Update a CVAT site proxy template `$HOME/cvat/cvat_proxy/conf.d/cvat.conf.template` on docker(system) host. Site config updates from this template each time `cvat_proxy` container start.
Add a location to server with `server_name ${CVAT_HOST};` ahead others: Add a location to server with `server_name ${CVAT_HOST};` ahead others:
@ -457,81 +458,99 @@ Add a location to server with `server_name ${CVAT_HOST};` ahead others:
} }
``` ```
You can use the [Nginx quickstart guide](http://nginx.org/en/docs/beginners_guide.html) for reference. Make the changes where necessary, e.g. base.py or somewhere else.
Build the containers with new configurations updated in `docker-compose.override.yml`
E.g. including `analytics` module:
```bash ```
# on the docker host docker-compose -f docker-compose.yml -f components/analytics/docker-compose.analytics.yml -f docker-compose.override.yml up -d --build
docker-compose down
docker-compose up -d
``` ```
Your server should still be visible (and unsecured) at `http://my-cvat-server.org` Your server should be available (and unsecured) at `http://CVAT.example.com`
but you won't see any behavior changes.
At this point your deployment is up and running, ready for run acme-challenge. Something went wrong ? The most common cause is a containers and images cache which were builded earlier.
##### 2. Setting up HTTPS with `acme.sh` helper This will enable serving `http://CVAT.example.com/.well-known/acme-challenge/`
route from `/var/tmp/letsencrypt-webroot` directory on the container's filesystem which is bind mounted from docker host `$HOME/cvat/letsencrypt-webroot`. That volume needed for issue and renewing certificates only.
There are multiple approaches. First one is to use helper on docker host. Another volume `/etc/ssl/private` should be used within web server according to [acme.sh](https://github.com/acmesh-official/acme.sh#3-install-the-cert-to-apachenginx-etc) documentation
In a our approach At this point your deployment is up and running, ready for run acme-challenge for issue a new certificate
- it is easier to setup automatic certificate updates and (than it can be done in the container). ##### 2. Issue a certificate and run HTTPS versions with `acme.sh` helper
- leave certificates in safe place on docker host (protect from `docker-compose down` cleanup)
- no unnecessary certificate files copying between container and host.
###### Create certificate files using an ACME challenge on docker host ###### Create certificate files using an ACME challenge on docker host
**Prepare certificates.** ####### Prepare certificates
Point you shell in cvat repository directory, usually `cd $HOME/cvat` on docker host. Point you shell in cvat repository directory, usually `cd $HOME/cvat` on docker host.
> Certificate issue and updates should be on docker host in this approach.
Let’s Encrypt provides rate limits to ensure fair usage by as many people as possible. They recommend utilize their staging environment instead of the production API during testing. So first try to get a test certificate. Let’s Encrypt provides rate limits to ensure fair usage by as many people as possible. They recommend utilize their staging environment instead of the production API during testing. So first try to get a test certificate.
``` ```
~/.acme.sh/acme.sh --issue --staging -d my-cvat-server.org -w $HOME/cvat/letsencrypt-webroot --debug ~/.acme.sh/acme.sh --issue --staging -d CVAT.example.com -w $HOME/cvat/letsencrypt-webroot --debug
``` ```
> Debug note: nginx server logs for cvat_proxy are not saved in container. You shall see it at docker host by with: `docker logs cvat_proxy`. > Debug note: nginx server logs for cvat_proxy are not saved in container. You shall see it at docker host by with: `docker logs cvat_proxy`.
If certificates is issued a successful we should test a renew: If certificates is issued a successful we can test a renew:
``` ```
~/.acme.sh/acme.sh --renew --force --staging -d my-cvat-server.org -w $HOME/cvat/letsencrypt-webroot --debug ~/.acme.sh/acme.sh --renew --force --staging -d CVAT.example.com -w $HOME/cvat/letsencrypt-webroot --debug
``` ```
If success: ####### Remove test certificate, if success
- remove test certificate ```
- issue a production certificate ~/.acme.sh/acme.sh --remove -d CVAT.example.com --debug
- create a cron job for user (`crontab -e`). rm -r /root/.acme.sh/CVAT.example.com
```
####### Issue a production certificate
``` ```
~/.acme.sh/acme.sh --remove -d my-cvat-server.org --debug ~/.acme.sh/acme.sh --issue -d CVAT.example.com -w $HOME/cvat/letsencrypt-webroot --debug
rm -r /root/.acme.sh/my-cvat-server.org ```
####### Install production certificate and a user cron job (`crontab -e`) for update it
This will copy necessary certificate files to a permanent directory for serve.
According to acme.sh [documentation](https://github.com/acmesh-official/acme.sh#3-install-the-cert-to-apachenginx-etc)
Additionally, we must create a directory for our domain.
Acme supports a valid install configuration options in domain config file
E.g. `~/.acme.sh/CVAT.example.com/lsoft-cvat.cvisionlab.com.conf`.
~/.acme.sh/acme.sh --issue -d my-cvat-server.org -w $HOME/cvat/letsencrypt-webroot --debug ```
mkdir /etc/ssl/private/CVAT.example.com
~/.acme.sh/acme.sh --install-cronjob acme.sh --install-cert -d CVAT.example.com \
--cert-file /etc/ssl/private/CVAT.example.com/site.cer \
--key-file /etc/ssl/private/CVAT.example.com/site.key \
--fullchain-file /etc/ssl/private/CVAT.example.com/fullchain.cer \
--reloadcmd "/usr/bin/docker restart cvat_proxy"
``` ```
Down the cvat_proxy container for setup https with issued certificates. Down the cvat_proxy container for setup https with issued certificate.
```bash ```bash
docker stop cvat_proxy docker stop cvat_proxy
``` ```
**Reconfigure nginx for use certificates.** ####### Reconfigure nginx for use certificates
Bring the configuration file `$HOME/cvat/cvat_proxy/conf.d/cvat.conf.template` to the following form: Bring the configuration file `$HOME/cvat/cvat_proxy/conf.d/cvat.conf.template` to the following form:
- add location with redirect `return 301` to 80/tcp server. - add location with redirect `return 301` from http to https port;
- change listen to `listen 443 ssl;` in main configururation server - change main cvat server to listen on 443 port;
- add ssl certificates options and secure them. - add ssl certificates options.
Final configuration file should look like:
Example of configuration file: > for a more accurate proxy configuration according to upstream,
> do not neglect the verification with
> this configuration [file](https://github.com/openvinotoolkit/cvat/blob/v1.2.0/cvat_proxy/conf.d/cvat.conf.template).
``` ```
server { server {
@ -546,7 +565,7 @@ server {
location ^~ /.well-known/acme-challenge/ { location ^~ /.well-known/acme-challenge/ {
default_type "text/plain"; default_type "text/plain";
root /var/tmp/letsencrypt; root /var/tmp/letsencrypt-webroot;
} }
location / { location / {
@ -556,9 +575,11 @@ server {
server { server {
listen 443 ssl; listen 443 ssl;
ssl_certificate /root/.acme.sh/my-cvat-server.org/my-cvat-server.org.cer; server_name ${CVAT_HOST};
ssl_certificate_key /root/.acme.sh/my-cvat-server.org/my-cvat-server.org.key;
ssl_trusted_certificate /root/.acme.sh/my-cvat-server.org/fullchain.cer; ssl_certificate /etc/ssl/private/${CVAT_HOST}/site.cer;
ssl_certificate_key /etc/ssl/private/${CVAT_HOST}/site.key;
ssl_trusted_certificate /etc/ssl/private/${CVAT_HOST}/fullchain.cer;
# security options # security options
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
@ -568,16 +589,6 @@ server {
ssl_session_cache shared:SSL:2m; ssl_session_cache shared:SSL:2m;
ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:!3DES'; ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:!3DES';
# security headers
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "no-referrer-when-downgrade" always;
add_header Content-Security-Policy "default-src 'self' http: https: data: blob: 'unsafe-inline'" always;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
server_name ${CVAT_HOST};
proxy_pass_header X-CSRFToken; proxy_pass_header X-CSRFToken;
proxy_set_header Host $http_host; proxy_set_header Host $http_host;
proxy_pass_header Set-Cookie; proxy_pass_header Set-Cookie;
@ -590,6 +601,7 @@ server {
proxy_pass http://cvat_ui; proxy_pass http://cvat_ui;
} }
} }
``` ```
Start cvat_proxy container with https enabled. Start cvat_proxy container with https enabled.

Write Preview
Loading…
Cancel
Save