diff --git a/site/content/en/docs/administration/advanced/backup_guide.md b/site/content/en/docs/administration/advanced/backup_guide.md index 7fa7deb3..af5230ed 100644 --- a/site/content/en/docs/administration/advanced/backup_guide.md +++ b/site/content/en/docs/administration/advanced/backup_guide.md @@ -31,7 +31,7 @@ Docker volumes are used to store all CVAT data: All CVAT containers should be stopped before backup: -```bash +```shell docker-compose stop ``` @@ -40,19 +40,19 @@ using the `-f` parameter. Backup data: -```bash +```shell mkdir backup -docker run --rm --name temp_backup --volumes-from cvat_db -v $(pwd)/backup:/backup ubuntu tar -cjvf /backup/cvat_db.tar.bz2 /var/lib/postgresql/data -docker run --rm --name temp_backup --volumes-from cvat_server -v $(pwd)/backup:/backup ubuntu tar -cjvf /backup/cvat_data.tar.bz2 /home/django/data +docker run --rm --name temp_backup --volumes-from cvat_db -v $(pwd)/backup:/backup ubuntu tar -czvf /backup/cvat_db.tar.gz /var/lib/postgresql/data +docker run --rm --name temp_backup --volumes-from cvat_server -v $(pwd)/backup:/backup ubuntu tar -czvf /backup/cvat_data.tar.gz /home/django/data # [optional] -docker run --rm --name temp_backup --volumes-from cvat_elasticsearch -v $(pwd)/backup:/backup ubuntu tar -cjvf /backup/cvat_events.tar.bz2 /usr/share/elasticsearch/data +docker run --rm --name temp_backup --volumes-from cvat_elasticsearch -v $(pwd)/backup:/backup ubuntu tar -czvf /backup/cvat_events.tar.gz /usr/share/elasticsearch/data ``` Make sure the backup archives have been created, the output of `ls backup` command should look like this: -```bash +```shell ls backup -cvat_data.tar.bz2 cvat_db.tar.bz2 cvat_events.tar.bz2 +cvat_data.tar.gz cvat_db.tar.gz cvat_events.tar.gz ``` ## How to restore CVAT from backup @@ -65,23 +65,23 @@ your data properly internally.** Note: CVAT containers must exist (if no, please follow the [installation guide](/docs/administration/basics/installation/#quick-installation-guide)). Stop all CVAT containers: -```bash +```shell docker-compose stop ``` Restore data: -```bash +```shell cd -docker run --rm --name temp_backup --volumes-from cvat_db -v $(pwd):/backup ubuntu bash -c "cd /var/lib/postgresql/data && tar -xvf /backup/cvat_db.tar.bz2 --strip 4" -docker run --rm --name temp_backup --volumes-from cvat_server -v $(pwd):/backup ubuntu bash -c "cd /home/django/data && tar -xvf /backup/cvat_data.tar.bz2 --strip 3" +docker run --rm --name temp_backup --volumes-from cvat_db -v $(pwd):/backup ubuntu bash -c "cd /var/lib/postgresql/data && tar -xvf /backup/cvat_db.tar.gz --strip 4" +docker run --rm --name temp_backup --volumes-from cvat_server -v $(pwd):/backup ubuntu bash -c "cd /home/django/data && tar -xvf /backup/cvat_data.tar.gz --strip 3" # [optional] -docker run --rm --name temp_backup --volumes-from cvat_elasticsearch -v $(pwd):/backup ubuntu bash -c "cd /usr/share/elasticsearch/data && tar -xvf /backup/cvat_events.tar.bz2 --strip 4" +docker run --rm --name temp_backup --volumes-from cvat_elasticsearch -v $(pwd):/backup ubuntu bash -c "cd /usr/share/elasticsearch/data && tar -xvf /backup/cvat_events.tar.gz --strip 4" ``` After that run CVAT as usual: -```bash +```shell docker-compose up -d ``` diff --git a/site/content/en/docs/administration/advanced/upgrade_guide.md b/site/content/en/docs/administration/advanced/upgrade_guide.md new file mode 100644 index 00000000..ee75c754 --- /dev/null +++ b/site/content/en/docs/administration/advanced/upgrade_guide.md @@ -0,0 +1,71 @@ +--- +title: 'Upgrade guide' +linkTitle: 'Upgrade guide' +weight: 11 +description: 'Instructions for upgrading CVAT deployed with docker compose' +--- + + + +## Upgrade guide + +To upgrade CVAT, follow these steps: + +- It is highly recommended backup all CVAT data before updating, follow the + [backup guide](/docs/administration/advanced/backup_guide/) and backup all CVAT volumes. + +- Go to the previously cloned CVAT directory and stop all CVAT containers with: + ```shell + docker-compose down + ``` + If you have included [additional components](/docs/administration/basics/installation/#additional-components), + include all compose configuration files that are used, e.g.: + ```shell + docker-compose -f docker-compose.yml -f components/analytics/docker-compose.analytics.yml down + ``` + +- Update CVAT source code by any preferable way: clone with git or download zip file from GitHub. + Note that you need to download the entire source code, not just the `docker-compose` configuration file. + Check the + [installation guide](/docs/administration/basics/installation/#how-to-get-cvat-source-code) for details. + +- Verify settings: + The installation process is changed/modified from version to version and + you may need to export some environment variables, for example + [CVAT_HOST](/docs/administration/basics/installation/#use-your-own-domain). + +- Update local CVAT images. + Pull or build new CVAT images, see + [How to pull/build/update CVAT images section](/docs/administration/basics/installation/#how-to-pullbuildupdate-cvat-images) + for details. + +- Start CVAT with: + ```shell + docker-compose up -d + ``` + When CVAT starts, it will upgrade its DB in accordance with the latest schema. + It can take time especially if you have a lot of data. + Please do not terminate the migration and wait till the process is complete. + You can monitor the startup process with the following command: + ```shell + docker logs cvat_server -f + ``` + +## How to udgrade CVAT from v1.7.0 to v2.1.0. + +Step by step commands how to udgrade CVAT from v1.7.0 to v2.1.0. +Let's assume that you have CVAT v1.7.0 working. +```shell +cd cvat +docker-compose down +cd .. +mv cvat cvat_old +wget https://github.com/opencv/cvat/archive/refs/tags/v2.1.0.zip +unzip v2.1.0.zip && mv cvat-2.1.0 cvat +cd cvat +docker pull cvat/server:v2.1.0 +docker tag cvat/server:v2.1.0 openvino/cvat_server:latest +docker pull cvat/ui:v2.1.0 +docker tag cvat/ui:v2.1.0 openvino/cvat_ui:latest +docker-compose up -d +``` diff --git a/site/content/en/docs/administration/basics/installation.md b/site/content/en/docs/administration/basics/installation.md index 773c2469..536a662e 100644 --- a/site/content/en/docs/administration/basics/installation.md +++ b/site/content/en/docs/administration/basics/installation.md @@ -31,7 +31,7 @@ For access from China, read [sources for users from China](#sources-for-users-fr - Type commands below into the terminal window to install `docker`. More instructions can be found [here](https://docs.docker.com/install/linux/docker-ce/ubuntu/). - ```bash + ```shell sudo apt-get update sudo apt-get --no-install-recommends install -y \ apt-transport-https \ @@ -51,7 +51,7 @@ For access from China, read [sources for users from China](#sources-for-users-fr - Perform [post-installation steps](https://docs.docker.com/install/linux/linux-postinstall/) to run docker without root permissions. - ```bash + ```shell sudo groupadd docker sudo usermod -aG docker $USER ``` @@ -63,51 +63,48 @@ For access from China, read [sources for users from China](#sources-for-users-fr - Install docker-compose (1.19.0 or newer). Compose is a tool for defining and running multi-container docker applications. - ```bash + ```shell sudo apt-get --no-install-recommends install -y python3-pip python3-setuptools sudo python3 -m pip install setuptools docker-compose ``` - Clone _CVAT_ source code from the - [GitHub repository](https://github.com/opencv/cvat). + [GitHub repository](https://github.com/opencv/cvat) with Git. - ```bash + ```shell sudo apt-get --no-install-recommends install -y git git clone https://github.com/opencv/cvat cd cvat ``` + See [alternatives](#how-to-get-cvat-source-code) with `wget` or `curl`. + - To access CVAT over a network or through a different system, export `CVAT_HOST` environment variable - ```bash + ```shell export CVAT_HOST=your-ip-address ``` - Run docker containers. It will take some time to download the latest CVAT release and other required images like postgres, redis, etc. from DockerHub and create containers. - ```bash + ```shell docker-compose up -d ``` - Use `CVAT_VERSION` environment variable to specify the version of CVAT you want to install (e.g `v2.1.0`, `dev`): - ```bash + ```shell CVAT_VERSION=dev docker-compose up -d ``` - Alternative: if you want to build the images locally with unreleased changes - run the following command. It will take some time to build CVAT images. - - ```bash - docker-compose -f docker-compose.yml -f docker-compose.dev.yml build - docker-compose up -d - ``` + see [How to pull/build/update CVAT images section](#how-to-pullbuildupdate-cvat-images) - You can register a user but by default it will not have rights even to view list of tasks. Thus you should create a superuser. A superuser can use an admin panel to assign correct groups to the user. Please use the command below: - ```bash + ```shell docker exec -it cvat_server bash -ic 'python3 ~/manage.py createsuperuser' ``` @@ -117,7 +114,7 @@ For access from China, read [sources for users from China](#sources-for-users-fr - Google Chrome is the only browser which is supported by CVAT. You need to install it as well. Type commands below in a terminal window: - ```bash + ```shell curl https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - sudo sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list' sudo apt-get update @@ -155,7 +152,7 @@ For access from China, read [sources for users from China](#sources-for-users-fr - Clone _CVAT_ source code from the [GitHub repository](https://github.com/opencv/cvat). - ```bash + ```shell git clone https://github.com/opencv/cvat cd cvat ``` @@ -163,30 +160,25 @@ For access from China, read [sources for users from China](#sources-for-users-fr - Run docker containers. It will take some time to download the latest CVAT release and other required images like postgres, redis, etc. from DockerHub and create containers. - ```bash + ```shell docker-compose up -d ``` - Alternative: if you want to build the images locally with unreleased changes - run the following command. It will take some time to build CVAT images. - - ```bash - docker-compose -f docker-compose.yml -f docker-compose.dev.yml build - docker-compose up -d - ``` + see [How to pull/build/update CVAT images section](#how-to-pullbuildupdate-cvat-images) - You can register a user but by default it will not have rights even to view list of tasks. Thus you should create a superuser. A superuser can use an admin panel to assign correct groups to other users. Please use the command below: - ```bash + ```shell winpty docker exec -it cvat_server bash -ic 'python3 ~/manage.py createsuperuser' ``` If you don't have winpty installed or the above command does not work, you may also try the following: - ```bash + ```shell # enter docker image first docker exec -it cvat_server /bin/bash # then run @@ -213,7 +205,7 @@ For access from China, read [sources for users from China](#sources-for-users-fr install the Xcode Command Line Tools. On Mavericks (10.9) or above you can do this simply by trying to run git from the Terminal the very first time. - ```bash + ```shell git --version ``` @@ -229,34 +221,30 @@ For access from China, read [sources for users from China](#sources-for-users-fr launch Spotlight and type "Terminal," then double-click the search result. - Clone _CVAT_ source code from the - [GitHub repository](https://github.com/opencv/cvat). + [GitHub repository](https://github.com/opencv/cvat) with Git. - ```bash + ```shell git clone https://github.com/opencv/cvat cd cvat ``` + See [alternatives](#how-to-get-cvat-source-code) with `wget` or `curl`. - Run docker containers. It will take some time to download the latest CVAT release and other required images like postgres, redis, etc. from DockerHub and create containers. - ```bash + ```shell docker-compose up -d ``` - Alternative: if you want to build the images locally with unreleased changes - run the following command. It will take some time to build CVAT images. - - ```bash - docker-compose -f docker-compose.yml -f docker-compose.dev.yml build - docker-compose up -d - ``` + see [How to pull/build/update CVAT images section](#how-to-pullbuildupdate-cvat-images) - You can register a user but by default it will not have rights even to view list of tasks. Thus you should create a superuser. A superuser can use an admin panel to assign correct groups to other users. Please use the command below: - ```bash + ```shell docker exec -it cvat_server bash -ic 'python3 ~/manage.py createsuperuser' ``` @@ -270,6 +258,66 @@ For access from China, read [sources for users from China](#sources-for-users-fr ## Advanced Topics +### How to get CVAT source code + +#### Git (Linux, Mac, Windows) + +1. Install Git on your system if it's not already installed + - Ubuntu: + ```shell + sudo apt-get --no-install-recommends install -y git + ``` + + - Windows: + Follow instructions from [https://git-scm.com/download/win](https://git-scm.com/download/win) + +2. Clone _CVAT_ source code from the + [GitHub repository](https://github.com/opencv/cvat). + + The command below will clone the default branch (develop): + ```shell + git clone https://github.com/opencv/cvat + cd cvat + ``` + + To clone specific tag, e.g. v2.1.0: + ```shell + git clone -b v2.1.0 https://github.com/opencv/cvat + cd cvat + ``` + +#### Wget (Linux, Mac) + +To download latest develop branch: +```shell +wget https://github.com/opencv/cvat/archive/refs/heads/develop.zip +unzip develop.zip && mv cvat-develop cvat +cd cvat +``` + +To download specific tag: +```shell +wget https://github.com/opencv/cvat/archive/refs/tags/v1.7.0.zip +unzip v1.7.0.zip && mv cvat-1.7.0 cvat +cd cvat +``` + +#### Curl (Linux, Mac) + +To download latest develop branch: +```shell +curl -LO https://github.com/opencv/cvat/archive/refs/heads/develop.zip +unzip develop.zip && mv cvat-develop cvat +cd cvat +``` + +To download specific tag: +```shell +curl -LO https://github.com/opencv/cvat/archive/refs/tags/v1.7.0.zip +unzip v1.7.0.zip && mv cvat-1.7.0 cvat +cd cvat +``` + ### Deploying CVAT behind a proxy If you deploy CVAT behind a proxy and do not plan to use any of [serverless functions](#semi-automatic-and-automatic-annotation) @@ -331,7 +379,7 @@ if you want to keep the dashboard in production you should read Traefik's - [Analytics: management and monitoring of data annotation team](/docs/administration/advanced/analytics/) -```bash +```shell # Build and run containers with Analytics component support: docker-compose -f docker-compose.yml \ -f components/analytics/docker-compose.analytics.yml up -d --build @@ -343,10 +391,9 @@ Please follow this [guide](/docs/administration/advanced/installation_automatic_ ### Stop all containers -The command below stops and removes containers, networks, volumes, and images -created by `up`. +The command below stops and removes containers and networks created by `up`. -```bash +```shell docker-compose down ``` @@ -355,7 +402,7 @@ docker-compose down If you want to access your instance of CVAT outside of your localhost (on another domain), you should specify the `CVAT_HOST` environment variable, like this: -```bash +```shell export CVAT_HOST= ``` @@ -426,14 +473,14 @@ enabling you to use HTTPS protocol to access your website. To enable this, first set the the `CVAT_HOST` (the domain of your website) and `ACME_EMAIL` (contact email for Let's Encrypt) environment variables: -```bash +```shell export CVAT_HOST= export ACME_EMAIL= ``` Then, use the `docker-compose.https.yml` file to override the base `docker-compose.yml` file: -```bash +```shell docker-compose -f docker-compose.yml -f docker-compose.https.yml up -d ``` @@ -441,6 +488,38 @@ docker-compose -f docker-compose.yml -f docker-compose.https.yml up -d Then, the CVAT instance will be available at your domain on ports 443 (HTTPS) and 80 (HTTP, redirects to 443). +### How to pull/build/update CVAT images + +- **For a CVAT version lower or equal to 2.1.0**, you need to pull images using docker because + the compose configuration always points to the latest image tag, e.g. + ```shell + docker pull cvat/server:v1.7.0 + docker tag cvat/server:v1.7.0 openvino/cvat_server:latest + + docker pull cvat/ui:v1.7.0 + docker tag cvat/ui:v1.7.0 openvino/cvat_ui:latest + ``` + **For CVAT version more than v2.1.0** it's possible to pull specific version of + prebuilt images from DockerHub using `CVAT_VERSION` environment variable to specify + the version (e.g. `dev`): + ```shell + CVAT_VERSION=dev docker-compose pull + ``` + +- To build images yourself include `docker-compose.dev.yml` compose config file to `docker-compose` command. + This can be useful if you want to build a CVAT with some source code changes. + ```shell + docker-compose -f docker-compose.yml -f docker-compose.dev.yml build + ``` +- To update local images to `latest` or `dev` tags run: + ```shell + CVAT_VERSION=dev docker-compose pull + ``` + or + ```shell + CVAT_VERSION=latest docker-compose pull + ``` + ## Troubleshooting ### Sources for users from China @@ -452,14 +531,14 @@ If you stay in China, for installation you need to override the following source [Ubuntu mirroring help](https://mirrors.tuna.tsinghua.edu.cn/help/ubuntu/) Pre-compiled packages: - ```bash + ```shell deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal main restricted universe multiverse deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-updates main restricted universe multiverse deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-backports main restricted universe multiverse deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-security main restricted universe multiverse ``` Or source packages: - ```bash + ```shell deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal main restricted universe multiverse deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-updates main restricted universe multiverse deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-backports main restricted universe multiverse @@ -484,14 +563,14 @@ If you stay in China, for installation you need to override the following source - For using `pip`: [PyPI mirroring help](https://mirrors.tuna.tsinghua.edu.cn/help/pypi/) - ```bash + ```shell pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple ``` - For using `npm`: [npm mirroring help](https://npmmirror.com/) - ```bash + ```shell npm config set registry https://registry.npm.taobao.org/ ``` @@ -500,7 +579,7 @@ If you stay in China, for installation you need to override the following source [CVAT repository on gitee.com](https://gitee.com/monkeycc/cvat) - For replace acceleration source `docker.com` run: - ```bash + ```shell curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - sudo add-apt-repository \ "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ @@ -508,7 +587,7 @@ If you stay in China, for installation you need to override the following source ``` - For replace acceleration source `google.com` run: - ```bash + ```shell curl https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - ``` @@ -517,7 +596,7 @@ If you stay in China, for installation you need to override the following source If you're having trouble with SSL connection, to find the cause, you'll need to get the logs from traefik by running: -```bash +```shell docker logs traefik ``` @@ -530,19 +609,19 @@ If the error is related to a firewall, then: After `acme.json` is removed, stop all cvat docker containers: -```bash +```shell docker-compose -f docker-compose.yml -f docker-compose.https.yml down ``` Make sure variables set (with your values): -```bash +```shell export CVAT_HOST= export ACME_EMAIL= ``` and restart docker: -```bash +```shell docker-compose -f docker-compose.yml -f docker-compose.https.yml up -d ``` diff --git a/site/content/en/docs/faq.md b/site/content/en/docs/faq.md index 7a2d766f..36a67a49 100644 --- a/site/content/en/docs/faq.md +++ b/site/content/en/docs/faq.md @@ -24,26 +24,12 @@ Please be patient. https://user-images.githubusercontent.com/40690625/180879954-44afcd95-1e94-451a-9a60-2f3bd6482cbf.gif) -## How to update CVAT +## How to upgrade CVAT -Before updating, please follow the [backup guide](/docs/administration/advanced/backup_guide/) +Before upgrading, please follow the [backup guide](/docs/administration/advanced/backup_guide/) and backup all CVAT volumes. -To update CVAT, you should clone or download the new version of CVAT and rebuild the CVAT docker images as usual. - -```bash -docker-compose build -``` - -and run containers: - -```bash -docker-compose up -d -``` - -Sometimes the update process takes a lot of time due to changes in the database schema and data. -You can check the current status with `docker logs cvat`. -Please do not terminate the migration and wait till the process is complete. +Follow the [upgrade guide](/docs/administration/advanced/upgrade_guide/). ## Kibana app works, but no logs are displayed