From d651cd6997286f2f903c07d74186df526cc11e9f Mon Sep 17 00:00:00 2001 From: Timur Osmanov <54434686+TOsmanov@users.noreply.github.com> Date: Thu, 30 Dec 2021 21:25:33 +0300 Subject: [PATCH] Improved documentation site. Adding a copy button for code blocks (#4024) --- README.md | 4 + site/assets/scss/_custom.scss | 39 ++++- site/config.toml | 2 +- .../docs/administration/advanced/analytics.md | 2 + .../administration/advanced/backup_guide.md | 12 +- .../installation_automatic_annotation.md | 4 +- .../basics/AWS-Deployment-Guide.md | 2 +- .../administration/basics/installation.md | 56 +++---- .../contributing/development-environment.md | 18 +- .../setup-additional-components.md | 10 +- site/content/en/docs/faq.md | 14 +- .../en/docs/manual/advanced/formats/_index.md | 4 + .../advanced/formats/format-cityscapes.md | 2 +- .../manual/advanced/serverless-tutorial.md | 158 +++++++++++------- .../manual/basics/attach-cloud-storage.md | 10 +- .../en/docs/manual/basics/authorization.md | 4 +- 16 files changed, 211 insertions(+), 130 deletions(-) diff --git a/README.md b/README.md index 29d59df4..3a9e092e 100644 --- a/README.md +++ b/README.md @@ -70,6 +70,10 @@ For more information about supported formats look at the | [VGGFace2](https://github.com/ox-vgg/vgg_face2) | X | X | | [Market-1501](https://www.aitribune.com/dataset/2018051063) | X | X | | [ICDAR13/15](https://rrc.cvc.uab.es/?ch=2) | X | X | +| [Open Images V6](https://storage.googleapis.com/openimages/web/index.html) | X | X | +| [Cityscapes](https://www.cityscapes-dataset.com/login/) | X | X | +| [KITTI](http://www.cvlibs.net/datasets/kitti/) | X | X | +| [LFW](http://vis-www.cs.umass.edu/lfw/) | X | X | diff --git a/site/assets/scss/_custom.scss b/site/assets/scss/_custom.scss index 4eb16946..4e7f9fa5 100644 --- a/site/assets/scss/_custom.scss +++ b/site/assets/scss/_custom.scss @@ -2,6 +2,41 @@ // // SPDX-License-Identifier: MIT +/* Code blocks */ + +div.code-toolbar .toolbar { + padding-right: 0.6em; + opacity: 1 !important; +} + +pre[class*="language-"] { + background: #f8f9fa !important; +} + +details { + max-width: 80%; +} + +@media (max-width: 991px) { + details { + max-width: 100% !important; + } +} + +li > details { + max-width: 100%; +} + +.code-toolbar { + max-width: 80%; +} + +.highlight > .code-toolbar, +details > summary > .code-toolbar, +details > .code-toolbar { + max-width: 100%; +} + /* Increased left padding on the sidebar of documentation */ .td-sidebar-nav__section .ul-1 ul { @@ -45,7 +80,7 @@ text-shadow: 0 0 3px rgb(27, 27, 27); } -/* block location */ +/* Location block */ .location { width: 70%; @@ -122,7 +157,7 @@ border: 4px white solid; } -/* cover block on about page */ +/* Cover block on about page */ #td-cover-block-0 { background-image: url("../images/background-about-page.jpg"); diff --git a/site/config.toml b/site/config.toml index fbd5ccfe..1388566d 100644 --- a/site/config.toml +++ b/site/config.toml @@ -134,7 +134,7 @@ algolia_docsearch = false offlineSearch = true # Enable syntax highlighting and copy buttons on code blocks with Prism -prism_syntax_highlighting = false +prism_syntax_highlighting = true # User interface configuration [params.ui] diff --git a/site/content/en/docs/administration/advanced/analytics.md b/site/content/en/docs/administration/advanced/analytics.md index c52b0365..d2b3776e 100644 --- a/site/content/en/docs/administration/advanced/analytics.md +++ b/site/content/en/docs/administration/advanced/analytics.md @@ -33,12 +33,14 @@ At the moment it is not possible to save advanced settings. Below values should ## Time picker default +```json { "from": "now/d", "to": "now/d", "display": "Today", "section": 0 } +``` ## Time picker quick ranges diff --git a/site/content/en/docs/administration/advanced/backup_guide.md b/site/content/en/docs/administration/advanced/backup_guide.md index 6fe1e15c..09091df3 100644 --- a/site/content/en/docs/administration/advanced/backup_guide.md +++ b/site/content/en/docs/administration/advanced/backup_guide.md @@ -31,7 +31,7 @@ Docker volumes are used to store all CVAT data: All CVAT containers should be stopped before backup: -```console +```bash docker-compose stop ``` @@ -40,7 +40,7 @@ using the `-f` parameter. Backup data: -```console +```bash mkdir backup docker run --rm --name temp_backup --volumes-from cvat_db -v $(pwd)/backup:/backup ubuntu tar -cjvf /backup/cvat_db.tar.bz2 /var/lib/postgresql/data docker run --rm --name temp_backup --volumes-from cvat -v $(pwd)/backup:/backup ubuntu tar -cjvf /backup/cvat_data.tar.bz2 /home/django/data @@ -50,7 +50,7 @@ docker run --rm --name temp_backup --volumes-from cvat_elasticsearch -v $(pwd)/b Make sure the backup archives have been created, the output of `ls backup` command should look like this: -```console +```bash ls backup cvat_data.tar.bz2 cvat_db.tar.bz2 cvat_events.tar.bz2 ``` @@ -60,13 +60,13 @@ cvat_data.tar.bz2 cvat_db.tar.bz2 cvat_events.tar.bz2 Note: CVAT containers must exist (if no, please follow the [installation guide](/docs/administration/basics/installation/#quick-installation-guide)). Stop all CVAT containers: -```console +```bash docker-compose stop ``` Restore data: -```console +```bash cd docker run --rm --name temp_backup --volumes-from cvat_db -v $(pwd):/backup ubuntu bash -c "cd /var/lib/postgresql/data && tar -xvf /backup/cvat_db.tar.bz2 --strip 4" docker run --rm --name temp_backup --volumes-from cvat -v $(pwd):/backup ubuntu bash -c "cd /home/django/data && tar -xvf /backup/cvat_data.tar.bz2 --strip 3" @@ -76,7 +76,7 @@ docker run --rm --name temp_backup --volumes-from cvat_elasticsearch -v $(pwd):/ After that run CVAT as usual: -```console +```bash docker-compose up -d ``` diff --git a/site/content/en/docs/administration/advanced/installation_automatic_annotation.md b/site/content/en/docs/administration/advanced/installation_automatic_annotation.md index 8e1465ad..0457631a 100644 --- a/site/content/en/docs/administration/advanced/installation_automatic_annotation.md +++ b/site/content/en/docs/administration/advanced/installation_automatic_annotation.md @@ -34,13 +34,13 @@ description: 'Information about the installation of components needed for semi-a [docker-compose.serverless.yml](https://github.com/openvinotoolkit/cvat/blob/develop/components/serverless/docker-compose.serverless.yml). For example, using wget. - ``` + ```bash wget https://github.com/nuclio/nuclio/releases/download//nuctl--linux-amd64 ``` After downloading the nuclio, give it a proper permission and do a softlink. - ``` + ```bash sudo chmod +x nuctl--linux-amd64 sudo ln -sf $(pwd)/nuctl--linux-amd64 /usr/local/bin/nuctl ``` diff --git a/site/content/en/docs/administration/basics/AWS-Deployment-Guide.md b/site/content/en/docs/administration/basics/AWS-Deployment-Guide.md index 786a070e..1a8afa49 100644 --- a/site/content/en/docs/administration/basics/AWS-Deployment-Guide.md +++ b/site/content/en/docs/administration/basics/AWS-Deployment-Guide.md @@ -25,7 +25,7 @@ There are two ways of deploying the CVAT. For any of above, don't forget to set the `CVAT_HOST` environemnt variable to the exposed AWS public IP address or hostname: -``` +```bash export CVAT_HOST=your-instance.amazonaws.com ``` diff --git a/site/content/en/docs/administration/basics/installation.md b/site/content/en/docs/administration/basics/installation.md index fcbc6b8e..c7cd68f9 100644 --- a/site/content/en/docs/administration/basics/installation.md +++ b/site/content/en/docs/administration/basics/installation.md @@ -31,7 +31,7 @@ For access from China, read [sources for users from China](#sources-for-users-fr - Type commands below into the terminal window to install `docker`. More instructions can be found [here](https://docs.docker.com/install/linux/docker-ce/ubuntu/). - ```sh + ```bash sudo apt-get update sudo apt-get --no-install-recommends install -y \ apt-transport-https \ @@ -51,7 +51,7 @@ For access from China, read [sources for users from China](#sources-for-users-fr - Perform [post-installation steps](https://docs.docker.com/install/linux/linux-postinstall/) to run docker without root permissions. - ```sh + ```bash sudo groupadd docker sudo usermod -aG docker $USER ``` @@ -77,15 +77,15 @@ For access from China, read [sources for users from China](#sources-for-users-fr cd cvat ``` - To access CVAT over a network or through a different system, export `CVAT_HOST` environment variable - + ```bash export CVAT_HOST=your-ip-address ``` - + - Run docker containers. It will take some time to download the latest CVAT release and other required images like postgres, redis, etc. from DockerHub and create containers. - ```sh + ```bash docker-compose up -d ``` @@ -102,7 +102,7 @@ For access from China, read [sources for users from China](#sources-for-users-fr admin panel to assign correct groups to the user. Please use the command below: - ```sh + ```bash docker exec -it cvat bash -ic 'python3 ~/manage.py createsuperuser' ``` @@ -112,7 +112,7 @@ For access from China, read [sources for users from China](#sources-for-users-fr - Google Chrome is the only browser which is supported by CVAT. You need to install it as well. Type commands below in a terminal window: - ```sh + ```bash curl https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - sudo sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list' sudo apt-get update @@ -150,7 +150,7 @@ For access from China, read [sources for users from China](#sources-for-users-fr - Clone _CVAT_ source code from the [GitHub repository](https://github.com/opencv/cvat). - ```sh + ```bash git clone https://github.com/opencv/cvat cd cvat ``` @@ -158,14 +158,14 @@ For access from China, read [sources for users from China](#sources-for-users-fr - Run docker containers. It will take some time to download the latest CVAT release and other required images like postgres, redis, etc. from DockerHub and create containers. - ```sh + ```bash docker-compose up -d ``` - Alternative: if you want to build the images locally with unreleased changes run the following command. It will take some time to build CVAT images. - ```sh + ```bash docker-compose -f docker-compose.yml -f docker-compose.dev.yml build docker-compose up -d ``` @@ -175,13 +175,13 @@ For access from China, read [sources for users from China](#sources-for-users-fr admin panel to assign correct groups to other users. Please use the command below: - ```sh + ```bash winpty docker exec -it cvat bash -ic 'python3 ~/manage.py createsuperuser' ``` If you don't have winpty installed or the above command does not work, you may also try the following: - ```sh + ```bash # enter docker image first docker exec -it cvat /bin/bash # then run @@ -234,14 +234,14 @@ For access from China, read [sources for users from China](#sources-for-users-fr - Run docker containers. It will take some time to download the latest CVAT release and other required images like postgres, redis, etc. from DockerHub and create containers. - ```sh + ```bash docker-compose up -d ``` - Alternative: if you want to build the images locally with unreleased changes run the following command. It will take some time to build CVAT images. - ```sh + ```bash docker-compose -f docker-compose.yml -f docker-compose.dev.yml build docker-compose up -d ``` @@ -251,7 +251,7 @@ For access from China, read [sources for users from China](#sources-for-users-fr admin panel to assign correct groups to other users. Please use the command below: - ```sh + ```bash docker exec -it cvat bash -ic 'python3 ~/manage.py createsuperuser' ``` @@ -295,7 +295,7 @@ dashboard might be very useful to see if the problem is with Traefik configurati You can enable the Traefik dashboard by uncommenting the following lines from `docker-compose.yml` -``` +```yml services: traefik: # Uncomment to get Traefik dashboard @@ -309,7 +309,7 @@ services: ``` and if you are using `docker-compose.https.yml`, also uncomment these lines -``` +```yml services: traefik: command: @@ -350,7 +350,7 @@ docker-compose down If you want to access your instance of CVAT outside of your localhost (on another domain), you should specify the `CVAT_HOST` environment variable, like this: -``` +```bash export CVAT_HOST= ``` @@ -422,14 +422,14 @@ enabling you to use HTTPS protocol to access your website. To enable this, first set the the `CVAT_HOST` (the domain of your website) and `ACME_EMAIL` (contact email for Let's Encrypt) environment variables: -``` +```bash export CVAT_HOST= export ACME_EMAIL= ``` Then, use the `docker-compose.https.yml` file to override the base `docker-compose.yml` file: -``` +```bash docker-compose -f docker-compose.yml -f docker-compose.https.yml up -d ``` @@ -444,14 +444,14 @@ If you stay in China, for installation you need to override the following source [Ubuntu mirroring help](https://mirrors.tuna.tsinghua.edu.cn/help/ubuntu/) Pre-compiled packages: - ``` + ```bash deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal main restricted universe multiverse deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-updates main restricted universe multiverse deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-backports main restricted universe multiverse deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-security main restricted universe multiverse ``` Or source packages: - ``` + ```bash deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal main restricted universe multiverse deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-updates main restricted universe multiverse deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-backports main restricted universe multiverse @@ -461,7 +461,7 @@ If you stay in China, for installation you need to override the following source - [Docker mirror station](https://www.daocloud.io/mirror) Add registry mirrors into `daemon.json` file: - ``` + ```json { "registry-mirrors": [ "http://f1361db2.m.daocloud.io", @@ -476,14 +476,14 @@ If you stay in China, for installation you need to override the following source - For using `pip`: [PyPI mirroring help](https://mirrors.tuna.tsinghua.edu.cn/help/pypi/) - ``` + ```bash pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple ``` - For using `npm`: [npm mirroring help](https://npmmirror.com/) - ``` + ```bash npm config set registry https://registry.npm.taobao.org/ ``` @@ -492,14 +492,14 @@ If you stay in China, for installation you need to override the following source [CVAT repository on gitee.com](https://gitee.com/monkeycc/cvat) - For replace acceleration source `docker.com` run: - ``` + ```bash curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - sudo add-apt-repository \ "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) \ ``` -- For replace acceleration source `google.com` run: - ``` +- For replace acceleration source `google.com` run: + ```bash curl https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - ``` diff --git a/site/content/en/docs/contributing/development-environment.md b/site/content/en/docs/contributing/development-environment.md index 3e344c33..5eb4ad26 100644 --- a/site/content/en/docs/contributing/development-environment.md +++ b/site/content/en/docs/contributing/development-environment.md @@ -9,11 +9,11 @@ description: 'Installing a development environment for different operating syste Ubuntu 18.04 - ```sh + ```bash sudo apt-get update && sudo apt-get --no-install-recommends install -y build-essential curl redis-server python3-dev python3-pip python3-venv python3-tk libldap2-dev libsasl2-dev pkg-config libavformat-dev libavcodec-dev libavdevice-dev libavutil-dev libswscale-dev libswresample-dev libavfilter-dev ``` - ```sh + ```bash # Install Node.js 16 curl -sL https://deb.nodesource.com/setup_16.x | sudo -E bash - sudo apt-get install -y nodejs @@ -21,7 +21,7 @@ description: 'Installing a development environment for different operating syste MacOS 10.15 - ```sh + ```bash brew install git python pyenv redis curl openssl node ``` @@ -32,7 +32,7 @@ description: 'Installing a development environment for different operating syste - Install CVAT on your local host: - ```sh + ```bash git clone https://github.com/openvinotoolkit/cvat cd cvat && mkdir logs keys python3 -m venv .env @@ -51,8 +51,10 @@ description: 'Installing a development environment for different operating syste - Create a super user for CVAT: - ```sh - $ python manage.py createsuperuser + ```bash + python manage.py createsuperuser + ``` + ``` Username (leave blank to use 'django'): *** Email address: *** Password: *** @@ -61,7 +63,7 @@ description: 'Installing a development environment for different operating syste - Install npm packages for UI and start UI debug server (run the following command from CVAT root directory): - ```sh + ```bash npm ci && npm run start:cvat-ui ``` @@ -75,7 +77,7 @@ description: 'Installing a development environment for different operating syste - Open new terminal (Ctrl + Shift + T), run Visual Studio Code from the virtual environment - ```sh + ```bash source .env/bin/activate && code ``` diff --git a/site/content/en/docs/contributing/setup-additional-components.md b/site/content/en/docs/contributing/setup-additional-components.md index 4f59d9b4..abd717ae 100644 --- a/site/content/en/docs/contributing/setup-additional-components.md +++ b/site/content/en/docs/contributing/setup-additional-components.md @@ -31,7 +31,7 @@ nuctl deploy --project-name cvat \
-```bash +``` 20.07.17 12:02:23.247 nuctl (I) Deploying function {"name": ""} 20.07.17 12:02:23.248 nuctl (I) Building {"versionInfo": "Label: 1.4.8, Git commit: 238d4539ac7783896d6c414535d0462b5f4cbcf1, OS: darwin, Arch: amd64, Go version: go1.14.3", "name": ""} 20.07.17 12:02:23.447 nuctl (I) Cleaning up before deployment @@ -59,7 +59,7 @@ nuctl deploy --project-name cvat \
-```bash +``` 20.07.17 12:05:23.377 nuctl (I) Deploying function {"name": ""} 20.07.17 12:05:23.378 nuctl (I) Building {"versionInfo": "Label: 1.4.8, Git commit: 238d4539ac7783896d6c414535d0462b5f4cbcf1, OS: darwin, Arch: amd64, Go version: go1.14.3", "name": ""} 20.07.17 12:05:23.590 nuctl (I) Cleaning up before deployment @@ -87,7 +87,7 @@ nuctl get function
-```bash +``` NAMESPACE | NAME | PROJECT | STATE | NODE PORT | REPLICAS nuclio | openvino.dextr | cvat | ready | 55274 | 1/1 nuclio | openvino.omz.public.yolo-v3-tf | cvat | ready | 57308 | 1/1 @@ -108,7 +108,7 @@ cat /tmp/input.json | nuctl invoke openvino.omz.public.yolo-v3-tf -c 'applicatio
-```bash +``` 20.07.17 12:07:44.519 nuctl.platform.invoker (I) Executing function {"method": "POST", "url": "http://:57308", "headers": {"Content-Type":["application/json"],"X-Nuclio-Log-Level":["info"],"X-Nuclio-Target":["openvino.omz.public.yolo-v3-tf"]}} 20.07.17 12:07:45.275 nuctl.platform.invoker (I) Got response {"status": "200 OK"} 20.07.17 12:07:45.275 nuctl (I) >>> Start of function logs @@ -142,7 +142,7 @@ Server = nuclio ## Run Cypress tests - Install Сypress as described in the [documentation](https://docs.cypress.io/guides/getting-started/installing-cypress.html). - Run cypress tests: -```sh +```bash cd /tests /node_modules/.bin/cypress run --headless --browser chrome ``` diff --git a/site/content/en/docs/faq.md b/site/content/en/docs/faq.md index 424a3f81..f2d7a153 100644 --- a/site/content/en/docs/faq.md +++ b/site/content/en/docs/faq.md @@ -14,13 +14,13 @@ and backup all CVAT volumes. To update CVAT, you should clone or download the new version of CVAT and rebuild the CVAT docker images as usual. -```sh +```bash docker-compose build ``` and run containers: -```sh +```bash docker-compose up -d ``` @@ -32,13 +32,13 @@ Please do not terminate the migration and wait till the process is complete. Make sure there aren't error messages from Elasticsearch: -```sh +```bash docker logs cvat_elasticsearch ``` If you see errors like this: -```sh +```bash lood stage disk watermark [95%] exceeded on [uMg9WI30QIOJxxJNDiIPgQ][uMg9WI3][/usr/share/elasticsearch/data/nodes/0] free: 116.5gb[4%], all indices on this node will be marked read-only ``` @@ -48,20 +48,20 @@ You should free up disk space or change the threshold, to do so check: [Elastics To change the hostname, simply set the `CVAT_HOST` environemnt variable -``` +```bash export CVAT_HOST= ``` NOTE, if you're using `docker-compose` with `sudo` to run CVAT, then please add the `-E` (or `--preserve-env`) flag to preserve the user environment variable which set above to take effect in your docker containers: -``` +```bash sudo -E docker-compose up -d ``` If you want to change the default web application port, change the `ports` part of `traefik` service configuration in `docker-compose.yml` -``` +```yml services: traefik: ... diff --git a/site/content/en/docs/manual/advanced/formats/_index.md b/site/content/en/docs/manual/advanced/formats/_index.md index d8175fdb..6a772d48 100644 --- a/site/content/en/docs/manual/advanced/formats/_index.md +++ b/site/content/en/docs/manual/advanced/formats/_index.md @@ -22,3 +22,7 @@ description: 'List of annotation formats supported by CVAT.' - [VGGFace2](format-vggface2) - [Market-1501](format-market1501) - [ICDAR13/15](format-icdar) +- [Open Images](format-openimages) +- [Cityscapes](format-cityscapes) +- [KITTI](format-kitti) +- [LFW](format-lfw) diff --git a/site/content/en/docs/manual/advanced/formats/format-cityscapes.md b/site/content/en/docs/manual/advanced/formats/format-cityscapes.md index e8b70aba..462c18d7 100644 --- a/site/content/en/docs/manual/advanced/formats/format-cityscapes.md +++ b/site/content/en/docs/manual/advanced/formats/format-cityscapes.md @@ -126,7 +126,7 @@ Original Cityscapes color map: ``` -
+
Upload images when creating a task: diff --git a/site/content/en/docs/manual/advanced/serverless-tutorial.md b/site/content/en/docs/manual/advanced/serverless-tutorial.md index a2a511d2..10c662ad 100644 --- a/site/content/en/docs/manual/advanced/serverless-tutorial.md +++ b/site/content/en/docs/manual/advanced/serverless-tutorial.md @@ -64,12 +64,14 @@ with specific configuration files. In the case it is `docker-compose.serverless. It has necessary instructions how to build and deploy Nuclio platform as a docker container and enable corresponding support in CVAT. -```console -$ docker-compose -f docker-compose.yml -f docker-compose.dev.yml -f components/serverless/docker-compose.serverless.yml up -d --build +```bash +docker-compose -f docker-compose.yml -f docker-compose.dev.yml -f components/serverless/docker-compose.serverless.yml up -d --build ``` -```console -$ docker-compose -f docker-compose.yml -f docker-compose.dev.yml -f components/serverless/docker-compose.serverless.yml ps +```bash +docker-compose -f docker-compose.yml -f docker-compose.dev.yml -f components/serverless/docker-compose.serverless.yml ps +``` +``` Name Command State Ports ------------------------------------------------------------------------------------------------------------- cvat /usr/bin/supervisord Up 8080/tcp @@ -87,8 +89,10 @@ is already installed on your operating system. Run the following command to check that it works. In the beginning you should not have any deployed serverless functions. -```console -$ nuctl get functions +```bash +nuctl get functions +``` +``` No functions found ``` @@ -104,10 +108,12 @@ First step is to deploy [SiamMask][siammask-serverless]. The deployment process can depend on your operating system. On Linux you can use `serverless/deploy_cpu.sh` auxiliary script, but below we are using `nuctl` directly. -```console -$ nuctl create project cvat +```bash +nuctl create project cvat nuctl deploy --project-name cvat --path "./serverless/pytorch/foolwood/siammask/nuclio" --platform local +``` +``` 21.05.07 13:00:22.233 nuctl (I) Deploying function {"name": ""} 21.05.07 13:00:22.233 nuctl (I) Building {"versionInfo": "Label: 1.5.16, Git commit: ae43a6a560c2bec42d7ccfdf6e8e11a1e3cc3774, OS: linux, Arch: amd64, Go version: go1.14.3", "name": ""} 21.05.07 13:00:22.652 nuctl (I) Cleaning up before deployment {"functionName": "pth-foolwood-siammask"} @@ -123,8 +129,10 @@ nuctl deploy --project-name cvat --path "./serverless/pytorch/foolwood/siammask/ 21.05.07 13:00:32.796 nuctl (I) Function deploy complete {"functionName": "pth-foolwood-siammask", "httpPort": 49155} ``` -```console -$ nuctl get functions +```bash +nuctl get functions +``` +``` NAMESPACE | NAME | PROJECT | STATE | NODE PORT | REPLICAS nuclio | pth-foolwood-siammask | cvat | ready | 49155 | 1/1 ``` @@ -168,13 +176,13 @@ command. Inference of the serverless function is optimized for CPU using
-```console -$ serverless/deploy_cpu.sh serverless/openvino/omz/public/yolo-v3-tf/ +```bash +serverless/deploy_cpu.sh serverless/openvino/omz/public/yolo-v3-tf/ ``` -```console +``` Deploying serverless/openvino/omz/public/yolo-v3-tf function... 21.07.12 15:55:17.314 nuctl (I) Deploying function {"name": ""} 21.07.12 15:55:17.314 nuctl (I) Building {"versionInfo": "Label: 1.5.16, Git commit: ae43a6a560c2bec42d7ccfdf6e8e11a1e3cc3774, OS: linux, Arch: amd64, Go version: go1.14.3", "name": ""} @@ -234,13 +242,13 @@ autoscaler does not support the local platform (docker)._
-```console -$ serverless/deploy_gpu.sh serverless/tensorflow/matterport/mask_rcnn +```bash +serverless/deploy_gpu.sh serverless/tensorflow/matterport/mask_rcnn ``` -```console +``` Deploying serverless/tensorflow/matterport/mask_rcnn function... 21.07.12 16:48:48.995 nuctl (I) Deploying function {"name": ""} 21.07.12 16:48:48.995 nuctl (I) Building {"versionInfo": "Label: 1.5.16, Git commit: ae43a6a560c2bec42d7ccfdf6e8e11a1e3cc3774, OS: linux, Arch: amd64, Go version: go1.14.3", "name": ""} @@ -280,9 +288,9 @@ of computer vision research projects and production applications in Facebook. Clone the repository somewhere. I assume that all other experiments will be run from the cloned `detectron2` directory. -```console -$ git clone https://github.com/facebookresearch/detectron2 -$ cd detectron2 +```bash +git clone https://github.com/facebookresearch/detectron2 +cd detectron2 ``` ### Run local experiments @@ -295,18 +303,18 @@ In my case I have Ubuntu 20.04 with python 3.8.5. I installed a virtual environment. Follow [opencv-python][opencv-python-github] installation guide to get the library for demo and visualization. -```console -$ python3 -m venv .detectron2 -$ . .detectron2/bin/activate -$ pip install torch==1.8.1+cpu torchvision==0.9.1+cpu torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html -$ pip install opencv-python +```bash +python3 -m venv .detectron2 +. .detectron2/bin/activate +pip install torch==1.8.1+cpu torchvision==0.9.1+cpu torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html +pip install opencv-python ``` Install the detectron2 library from your local clone (you should be inside detectron2 directory). -```console -$ python -m pip install -e . +```bash +python -m pip install -e . ``` After the library from Facebook AI Research is installed, we can run a couple @@ -314,22 +322,22 @@ of experiments. See the [official tutorial][detectron2-tutorial] for more examples. I decided to experiment with [RetinaNet][retinanet-model-zoo]. First step is to download model weights. -```console -$ curl -O https://dl.fbaipublicfiles.com/detectron2/COCO-Detection/retinanet_R_101_FPN_3x/190397697/model_final_971ab9.pkl +```bash +curl -O https://dl.fbaipublicfiles.com/detectron2/COCO-Detection/retinanet_R_101_FPN_3x/190397697/model_final_971ab9.pkl ``` To run experiments let's download an image with cats from wikipedia. -```console -$ curl -O https://upload.wikimedia.org/wikipedia/commons/thumb/0/0b/Cat_poster_1.jpg/1920px-Cat_poster_1.jpg +```bash +curl -O https://upload.wikimedia.org/wikipedia/commons/thumb/0/0b/Cat_poster_1.jpg/1920px-Cat_poster_1.jpg ``` Finally let's run the DL model inference on CPU. If all is fine, you will see a window with cats and bounding boxes around them with scores. -```console -$ python demo/demo.py --config-file configs/COCO-Detection/retinanet_R_101_FPN_3x.yaml \ - --input 1920px-Cat_poster_1.jpg --opts MODEL.WEIGHTS model_final_971ab9.pkl MODEL.DEVICE cpu +```bash +python demo/demo.py --config-file configs/COCO-Detection/retinanet_R_101_FPN_3x.yaml \ + --input 1920px-Cat_poster_1.jpg --opts MODEL.WEIGHTS model_final_971ab9.pkl MODEL.DEVICE cpu ``` ![Cats detected by RetinaNet R101](/images/detectron2_detected_cats.jpg) @@ -559,13 +567,13 @@ The actual deployment process is described in
-```console -$ ./serverless/deploy_cpu.sh ./serverless/pytorch/facebookresearch/detectron2/retinanet/ +```bash +./serverless/deploy_cpu.sh ./serverless/pytorch/facebookresearch/detectron2/retinanet/ ``` -```console +``` 21.07.21 15:20:31.011 nuctl (I) Deploying function {"name": ""} 21.07.21 15:20:31.011 nuctl (I) Building {"versionInfo": "Label: 1.5.16, Git commit: ae43a6a560c2bec42d7ccfdf6e8e11a1e3cc3774, OS: linux, Arch: amd64, Go version: go1.14.3", "name": ""} 21.07.21 15:20:31.407 nuctl (I) Cleaning up before deployment {"functionName": "pth.facebookresearch.detectron2.retinanet_r101"} @@ -675,8 +683,8 @@ After these changes deploy the serverless function once again. For `serverless/pytorch/facebookresearch/detectron2/retinanet/nuclio/` you should run the command below: -```console -$ serverless/deploy_cpu.sh serverless/pytorch/facebookresearch/detectron2/retinanet +```bash +serverless/deploy_cpu.sh serverless/pytorch/facebookresearch/detectron2/retinanet ``` To debug python code inside a container you have to publish the port (in this @@ -693,15 +701,15 @@ IP address of your host (usually IP address starts from `192.168.`). You will need to confirm that you want to connect to your host computer and enter your password. Keep the terminal open after that. -```console -$ docker exec -it nuclio-nuclio-pth.facebookresearch.detectron2.retinanet_r101 /bin/bash -$ apt update && apt install -y ssh -$ ssh -R 5678:localhost:5678 user@ipaddress +```bash +docker exec -it nuclio-nuclio-pth.facebookresearch.detectron2.retinanet_r101 /bin/bash +apt update && apt install -y ssh +ssh -R 5678:localhost:5678 user@ipaddress ``` See how the latest command looks like in my case: -```console +``` root@2d6cceec8f70:/opt/nuclio# ssh -R 5678:localhost:5678 nmanovic@192.168.50.188 The authenticity of host '192.168.50.188 (192.168.50.188)' can't be established. ECDSA key fingerprint is SHA256:0sD6IWi+FKAhtUXr2TroHqyjcnYRIGLLx/wkGaZeRuo. @@ -758,16 +766,20 @@ First of all need to check that you are using the recommended version of Nuclio framework. In my case it is `1.5.16` but you need to check [the installation manual][cvat-auto-annotation-guide]. -```console -$ nuctl version +```bash +nuctl version +``` +``` Client version: "Label: 1.5.16, Git commit: ae43a6a560c2bec42d7ccfdf6e8e11a1e3cc3774, OS: linux, Arch: amd64, Go version: go1.14.3" ``` Check that Nuclio dashboard is running and its version corresponds to `nuctl`. -```console -$ docker ps --filter NAME=^nuclio$ +```bash +docker ps --filter NAME=^nuclio$ +``` +``` CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 7ab0c076c927 quay.io/nuclio/dashboard:1.5.16-amd64 "/docker-entrypoint.…" 6 weeks ago Up 46 minutes (healthy) 80/tcp, 0.0.0.0:8070->8070/tcp, :::8070->8070/tcp nuclio ``` @@ -775,20 +787,22 @@ CONTAINER ID IMAGE COMMAND Be sure that the model, which doesn't work, is healthy. In my case Inside Outside Guidance is not running. -```console -$ docker ps --filter NAME=iog +```bash +docker ps --filter NAME=iog +``` +``` CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ``` Let's run it. Go to the root of CVAT repository and run the deploying command. -```console -$ serverless/deploy_cpu.sh serverless/pytorch/shiyinzhang/iog +```bash +serverless/deploy_cpu.sh serverless/pytorch/shiyinzhang/iog ```
-```console +``` Deploying serverless/pytorch/shiyinzhang/iog function... 21.07.06 12:49:08.763 nuctl (I) Deploying function {"name": ""} 21.07.06 12:49:08.763 nuctl (I) Building {"versionInfo": "Label: 1.5.16, Git commit: ae43a6a560c2bec42d7ccfdf6e8e11a1e3cc3774, OS: linux, Arch: amd64, Go version: go1.14.3", "name": ""} @@ -834,10 +848,16 @@ assigned by Nuclio. Now the port is used by `openvino-dextr` as we can see in logs. To prove our hypothesis just need to run a couple of docker commands: -```console -$ docker container ls -a | grep iog +```bash +docker container ls -a | grep iog +``` +``` eb0c1ee46630 cvat/pth.shiyinzhang.iog:latest "conda run -n iog pr…" 9 minutes ago Created nuclio-nuclio-pth.shiyinzhang.iog -$ docker inspect eb0c1ee46630 | grep 49154 +``` +```bash +docker inspect eb0c1ee46630 | grep 49154 +``` +``` "Error": "driver failed programming external connectivity on endpoint nuclio-nuclio-pth.shiyinzhang.iog (02384290f91b2216162b1603322dadee426afe7f439d3d090f598af5d4863b2d): Bind for 0.0.0.0:49154 failed: port is already allocated", "HostPort": "49154" ``` @@ -846,15 +866,19 @@ To solve the problem let's just remove the previous container for the function. In this case it is `eb0c1ee46630`. After that the deploying command works as expected. -```console -$ docker container rm eb0c1ee46630 +```bash +docker container rm eb0c1ee46630 +``` +``` eb0c1ee46630 -$ serverless/deploy_cpu.sh serverless/pytorch/shiyinzhang/iog +``` +```bash +serverless/deploy_cpu.sh serverless/pytorch/shiyinzhang/iog ```
-```console +``` Deploying serverless/pytorch/shiyinzhang/iog function... 21.07.06 13:09:52.934 nuctl (I) Deploying function {"name": ""} 21.07.06 13:09:52.934 nuctl (I) Building {"versionInfo": "Label: 1.5.16, Git commit: ae43a6a560c2bec42d7ccfdf6e8e11a1e3cc3774, OS: linux, Arch: amd64, Go version: go1.14.3", "name": ""} @@ -885,19 +909,27 @@ useful to look at logs. Just run a couple of commands like
-```console -$ docker logs cvat +```bash +docker logs cvat +``` +``` 2021-07-06 13:44:54,699 DEBG 'runserver' stderr output: [Tue Jul 06 13:44:54.699431 2021] [wsgi:error] [pid 625:tid 140010969868032] [remote 172.28.0.3:40972] [2021-07-06 13:44:54,699] ERROR django.request: Internal Server Error: /api/v1/lambda/functions/pth.shiyinzhang.iog 2021-07-06 13:44:54,700 DEBG 'runserver' stderr output: [Tue Jul 06 13:44:54.699712 2021] [wsgi:error] [pid 625:tid 140010969868032] [remote 172.28.0.3:40972] ERROR - 2021-07-06 13:44:54,699 - log - Internal Server Error: /api/v1/lambda/functions/pth.shiyinzhang.iog -$ docker container ls --filter name=iog +``` +```bash +docker container ls --filter name=iog +``` +``` CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 3b6ef9a9f3e2 cvat/pth.shiyinzhang.iog:latest "conda run -n iog pr…" 4 hours ago Up 4 hours (healthy) 0.0.0.0:49159->8080/tcp, :::49159->8080/tcp nuclio-nuclio-pth.shiyinzhang.iog -$ docker logs nuclio-nuclio-pth.shiyinzhang.iog +``` +```bash +docker logs nuclio-nuclio-pth.shiyinzhang.iog ```
diff --git a/site/content/en/docs/manual/basics/attach-cloud-storage.md b/site/content/en/docs/manual/basics/attach-cloud-storage.md index 08a29b11..3aa768cb 100644 --- a/site/content/en/docs/manual/basics/attach-cloud-storage.md +++ b/site/content/en/docs/manual/basics/attach-cloud-storage.md @@ -61,7 +61,7 @@ For example, let's take [The Oxford-IIIT Pet Dataset](https://www.robots.ox.ac.u - Unpack the archive into the prepared folder and create a manifest file as described in [prepare manifest file section](/docs/manual/advanced/dataset_manifest/): - ``` + ```bash python /utils/dataset_manifest/create.py --output-dir ``` @@ -349,7 +349,7 @@ I used `aws-cli 1.20.49` `Python 3.7.9` `Windows 10`. You can configure credentials by running `aws configure`. You will need to enter `Access Key ID` and `Secret Access Key` as well as region. -``` +```bash aws configure Access Key ID: Secret Access Key: @@ -357,20 +357,20 @@ Secret Access Key: Copy the content of the bucket to a folder on your computer: -``` +```bash aws s3 cp --recursive ``` After copying the files, you can create a manifest file as described in [preapair manifest file section](/docs/manual/advanced/dataset_manifest/): -``` +```bash python /utils/dataset_manifest/create.py --output-dir ``` When the manifest file is ready, you can upload it to aws s3 bucket. If you gave full write permissions when you created the user, run: -``` +```bash aws s3 cp /manifest.jsonl ``` diff --git a/site/content/en/docs/manual/basics/authorization.md b/site/content/en/docs/manual/basics/authorization.md index 829df269..58ad95ca 100644 --- a/site/content/en/docs/manual/basics/authorization.md +++ b/site/content/en/docs/manual/basics/authorization.md @@ -17,7 +17,9 @@ weight: 1 [Django administration panel](http://localhost:8080/admin) to assign correct groups to the user. Please use the command below to create an admin account: - `docker exec -it cvat bash -ic 'python3 ~/manage.py createsuperuser'` + ```bash + docker exec -it cvat bash -ic 'python3 ~/manage.py createsuperuser' + ``` - If you want to create a non-admin account, you can do that using the link below on the login page. Don't forget to modify permissions for the new user in the