diff --git a/cvat/apps/documentation/static/documentation/images/image051.jpg b/cvat/apps/documentation/static/documentation/images/image051.jpg index 8ad21860..403d0526 100644 Binary files a/cvat/apps/documentation/static/documentation/images/image051.jpg and b/cvat/apps/documentation/static/documentation/images/image051.jpg differ diff --git a/cvat/apps/documentation/static/documentation/images/image131_detrac.jpg b/cvat/apps/documentation/static/documentation/images/image131_detrac.jpg index fc72b9dd..59697973 100644 Binary files a/cvat/apps/documentation/static/documentation/images/image131_detrac.jpg and b/cvat/apps/documentation/static/documentation/images/image131_detrac.jpg differ diff --git a/cvat/apps/documentation/static/documentation/images/image144_detrac.jpg b/cvat/apps/documentation/static/documentation/images/image144_detrac.jpg index 2fe7f183..cbb21fcb 100644 Binary files a/cvat/apps/documentation/static/documentation/images/image144_detrac.jpg and b/cvat/apps/documentation/static/documentation/images/image144_detrac.jpg differ diff --git a/cvat/apps/documentation/static/documentation/images/image194.jpg b/cvat/apps/documentation/static/documentation/images/image194.jpg new file mode 100644 index 00000000..c1097586 Binary files /dev/null and b/cvat/apps/documentation/static/documentation/images/image194.jpg differ diff --git a/cvat/apps/documentation/static/documentation/images/image195.jpg b/cvat/apps/documentation/static/documentation/images/image195.jpg new file mode 100644 index 00000000..0badbb38 Binary files /dev/null and b/cvat/apps/documentation/static/documentation/images/image195.jpg differ diff --git a/cvat/apps/documentation/static/documentation/images/image196_detrac.jpg b/cvat/apps/documentation/static/documentation/images/image196_detrac.jpg new file mode 100644 index 00000000..d752f13e Binary files /dev/null and b/cvat/apps/documentation/static/documentation/images/image196_detrac.jpg differ diff --git a/cvat/apps/documentation/static/documentation/images/image197.jpg b/cvat/apps/documentation/static/documentation/images/image197.jpg new file mode 100644 index 00000000..e42ed1d9 Binary files /dev/null and b/cvat/apps/documentation/static/documentation/images/image197.jpg differ diff --git a/cvat/apps/documentation/user_guide.md b/cvat/apps/documentation/user_guide.md index e84beddf..ef1f2808 100644 --- a/cvat/apps/documentation/user_guide.md +++ b/cvat/apps/documentation/user_guide.md @@ -35,6 +35,7 @@ - [Annotation with cuboids](#annotation-with-cuboids) - [Annotation with tags](#annotation-with-tags) - [Track mode with polygons](#track-mode-with-polygons) + - [Review](#review) - [Automatic annotation](#automatic-annotation) - [Shape grouping](#shape-grouping) - [Filter](#filter) @@ -290,6 +291,7 @@ Go to the [Django administration panel](http://localhost:8080/admin). There you - Duration — is the amount of time the job is being worked. - Assignee is the user who is working on the job. You can start typing an assignee’s name and/or choose the right person out of the dropdown list. + - Reviewer – a user assigned to carry out the review, read more in the [review](#review) section. - `Copy`. By clicking Copy you will copy the job list to the clipboard. The job list contains direct links to jobs. @@ -753,6 +755,9 @@ Button assignment: - [MS COCO](http://cocodataset.org/#format-data) - [YOLO](https://pjreddie.com/darknet/yolo/) - `Open the task` — opens a page with details about the task. +- `Request a review` - calls up the form to submit the job for a review, read more in the [review](#review) section. +- `Finish the job` - changes the status of the job to `completed` and returns to the task page without review. +- `Submit the review` - (available during the review) calls up the form to submit a review, read more in the [review](#review) section. #### Save Work @@ -813,11 +818,10 @@ Open the job info. ![](static/documentation/images/image144_detrac.jpg) -- Job status: `annotation`, `validation` or `completed` task - _Overview_: - `Assinger` - the one to whom the job is assigned. +- `Reviewer` – a user assigned to carry out the review, read more in the [review](#review) section. - `Start Frame` - the number of the first frame in this job. - `End Frame` - the number of the last frame in this job. - `Frames` - the total number of all frames in the job. @@ -860,6 +864,7 @@ Switching between user interface modes. |![](static/documentation/images/image170.jpg)|`Points` |[Annotation with points](#annotation-with-points) | |![](static/documentation/images/image176.jpg)|`Cuboid` |[Annotation with cuboids](#annotation-with-cuboids) | |![](static/documentation/images/image171.jpg)|`Tag` |[Annotation with tags](#annotation-with-tag) | +|![](static/documentation/images/image195.jpg)|`Open an issue` |[Review](#review) (available only in review mode) | **Edit block** - contains tools for editing tracks and shapes. |Icon |Description |Links to section | @@ -1416,6 +1421,38 @@ Polygons in the track mode allow you to mark moving objects more accurately othe There is no need to redraw the polygon every time using `Shift+N`, instead you can simply move the points or edit a part of the polygon by pressing `Shift+Click`. +## Review + +A special mode to check the annotation allows you to point to an object or area in the frame containing an error. +To go into review mode, you need to select `Request a review` in the menu and assign the user to run a check. + +![](static/documentation/images/image194.jpg) + +After that, the job status will be changed to `validation` +and the reviewer will be able to open the task in review mode. +Review mode is a UI mode, there is a special "issue" tool which you can use to identify objects +or areas in the frame and describe the problem. + +- To do this, first click `open an issue` icon on the controls sidebar: + + ![](static/documentation/images/image195.jpg) + +- Then click on an object in the frame to highlight the object or highlight the area by holding the left mouse button +and describe the problem. The object or area will be shaded in red. +- The created issue will appear in the workspace and in the `issues` tab on the objects sidebar. +- After you save the annotation, other users will be able to see the problem, comment on each issue +and change the status of the problem to `resolved`. +- You can use the arrows on the issues tab to navigate the frames that contain problems. + + ![](static/documentation/images/image196_detrac.jpg) + +- Once all the problems are marked, save the annotation, open the menu and select "submit the review". After that you'll see a form containing the verification statistics, here you can give an assessment of the job and choose further actions: + - Accept - changes the status of the job to `completed`. + - Review next – passes the job to another user for re-review. + - Reject - changes the status of the job to `annotation`. + + ![](static/documentation/images/image197.jpg) + ## Automatic annotation Automatic Annotation is used for creating preliminary annotations.