diff --git a/CHANGELOG.md b/CHANGELOG.md index 4c5d1d3c..799d9fc2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,7 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [1.4.0] - Unreleased ### Added -- +- Documentation on mask annotation () ### Changed - @@ -20,7 +20,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - ### Fixed -- +- Export of instance masks with holes () ### Security - diff --git a/cvat/apps/dataset_manager/formats/README.md b/cvat/apps/dataset_manager/formats/README.md index cee31854..3af8cbfd 100644 --- a/cvat/apps/dataset_manager/formats/README.md +++ b/cvat/apps/dataset_manager/formats/README.md @@ -11,6 +11,7 @@ - [How to add a format](#how-to-add) - [Format descriptions](#formats) - [CVAT](#cvat) + - [Datumaro](#datumaro) - [LabelMe](#labelme) - [MOT](#mot) - [MOTS](#mots) @@ -175,7 +176,7 @@ features, so it can be used to make data backups. - [Format specification](/cvat/apps/documentation/xml_format.md) -#### CVAT for images dumper +#### CVAT for images export Downloaded file: a ZIP file of the following structure: @@ -189,7 +190,7 @@ taskname.zip/ - tracks are split by frames -#### CVAT for videos dumper +#### CVAT for videos export Downloaded file: a ZIP file of the following structure: @@ -207,6 +208,17 @@ taskname.zip/ Uploaded file: an XML file or a ZIP file of the structures above +### Datumaro format + +[Datumaro](https://github.com/openvinotoolkit/datumaro/) is a tool, which can +help with complex dataset and annotation transformations, format conversions, +dataset statistics, merging, custom formats etc. It is used as a provider +of dataset support in CVAT, so basically, everything possible in CVAT +is possible in Datumaro too, but Datumaro can offer dataset operations. + +- supported annotations: any 2D shapes, labels +- supported attributes: any + ### [Pascal VOC](http://host.robots.ox.ac.uk/pascal/VOC/) - [Format specification](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/devkit_doc.pdf) @@ -219,9 +231,10 @@ Uploaded file: an XML file or a ZIP file of the structures above - supported attributes: - - `occluded` + - `occluded` (both UI option and a separate attribute) - `truncated` and `difficult` (should be defined for labels as `checkbox` -es) - action attributes (import only, should be defined as `checkbox` -es) + - arbitrary attributes (in the `attributes` secion of XML files) #### Pascal VOC export @@ -229,7 +242,7 @@ Downloaded file: a zip archive of the following structure: ```bash taskname.zip/ -├── JpegImages/ +├── JPEGImages/ │   ├── .jpg │   ├── .jpg │   └── .jpg @@ -499,17 +512,47 @@ zip images.zip -j -@ < train.txt - [Format specification](http://cocodataset.org/#format-data) -#### COCO dumper description +#### COCO export -Downloaded file: single unpacked `json`. +Downloaded file: a zip archive with following structure: + +```bash +archive.zip/ +├── images/ +│ ├── +│ ├── +│ └── ... +└── annotations/ +    └── instances_default.json +``` - supported annotations: Polygons, Rectangles +- supported attributes: + - `is_crowd` (checkbox or integer with values 0 and 1) - + specifies that the instance (an object group) should have an + RLE-encoded mask in the `segmentation` field. All the grouped shapes + are merged into a single mask, the largest one defines all + the object properties + - `score` (number) - the annotation `score` field + - arbitrary attributes - will be stored in the `attributes` annotation section + + +*Note*: there is also a [support for COCO keypoints over Datumaro](https://github.com/openvinotoolkit/cvat/issues/2910#issuecomment-726077582) -#### COCO loader description +1. Install [Datumaro](https://github.com/openvinotoolkit/datumaro) + `pip install datumaro` +1. Export the task in the `Datumaro` format, unzip +1. Export the Datumaro project in `coco` / `coco_person_keypoints` formats + `datum export -f coco -p path/to/project [-- --save-images]` -Uploaded file: single unpacked `*.json` . +This way, one can export CVAT points as single keypoints or +keypoint lists (without the `visibility` COCO flag). -- supported annotations: Polygons, Rectangles (if `segmentation` field is empty) +#### COCO import + +Uploaded file: a single unpacked `*.json` or a zip archive with the structure above (without images). + +- supported annotations: Polygons, Rectangles (if the `segmentation` field is empty) #### How to create a task from MS COCO dataset @@ -559,25 +602,48 @@ image_feature_description = { } ``` -#### TFRecord dumper description +#### TFRecord export Downloaded file: a zip archive with following structure: ```bash taskname.zip/ -├── task2.tfrecord +├── default.tfrecord └── label_map.pbtxt + +# label_map.pbtxt +item { + id: 1 + name: 'label_0' +} +item { + id: 2 + name: 'label_1' +} +... ``` -- supported annotations: Rectangles +- supported annotations: Rectangles, Polygons (as masks, manually over [Datumaro](https://github.com/openvinotoolkit/datumaro/blob/develop/docs/user_manual.md)) -#### TFRecord loader description +How to export masks: +1. Export annotations in `Datumaro` format +1. Apply `polygons_to_masks` and `boxes_to_masks` transforms + ```bash + datum transform -t polygons_to_masks -p path/to/proj -o ptm + datum transform -t boxes_to_masks -p ptm -o btm + ``` +1. Export in the `TF Detection API` format + ```bash + datum export -f tf_detection_api -p btm [-- --save-images] + ``` + +#### TFRecord import Uploaded file: a zip archive of following structure: ```bash taskname.zip/ -└── task2.tfrecord +└── .tfrecord ``` - supported annotations: Rectangles @@ -706,7 +772,7 @@ python create_pascal_tf_record.py --data_dir --set train --y ### [MOT sequence](https://arxiv.org/pdf/1906.04567.pdf) -#### MOT Dumper +#### MOT export Downloaded file: a zip archive of the following structure: @@ -735,7 +801,7 @@ person - supported annotations: Rectangle shapes and tracks - supported attributes: `visibility` (number), `ignored` (checkbox) -#### MOT Loader +#### MOT import Uploaded file: a zip archive of the structure above or: @@ -749,7 +815,7 @@ taskname.zip/ ### [MOTS PNG](https://www.vision.rwth-aachen.de/page/mots) -#### MOTS PNG Dumper +#### MOTS PNG export Downloaded file: a zip archive of the following structure: @@ -773,7 +839,7 @@ person - supported annotations: Rectangle and Polygon tracks -#### MOTS PNG Loader +#### MOTS PNG import Uploaded file: a zip archive of the structure above @@ -781,7 +847,7 @@ Uploaded file: a zip archive of the structure above ### [LabelMe](http://labelme.csail.mit.edu/Release3.0) -#### LabelMe Dumper +#### LabelMe export Downloaded file: a zip archive of the following structure: @@ -793,7 +859,7 @@ taskname.zip/ - supported annotations: Rectangles, Polygons (with attributes) -#### LabelMe Loader +#### LabelMe import Uploaded file: a zip archive of the following structure: @@ -811,7 +877,7 @@ taskname.zip/ ### [ImageNet](http://www.image-net.org) -#### ImageNet Dumper +#### ImageNet export Downloaded file: a zip archive of the following structure: @@ -835,7 +901,7 @@ taskname.zip/ - supported annotations: Labels -#### ImageNet Loader +#### ImageNet import Uploaded file: a zip archive of the structure above @@ -843,7 +909,7 @@ Uploaded file: a zip archive of the structure above ### [CamVid](http://mi.eng.cam.ac.uk/research/projects/VideoRec/CamVid/) -#### CamVid Dumper +#### CamVid export Downloaded file: a zip archive of the following structure: @@ -873,7 +939,7 @@ has own color which corresponds to a label. - supported annotations: Rectangles, Polygons -#### CamVid Loader +#### CamVid import Uploaded file: a zip archive of the structure above @@ -881,7 +947,7 @@ Uploaded file: a zip archive of the structure above ### [WIDER Face](http://shuoyang1213.me/WIDERFACE/) -#### WIDER Face Dumper +#### WIDER Face export Downloaded file: a zip archive of the following structure: @@ -899,19 +965,21 @@ taskname.zip/ ``` - supported annotations: Rectangles (with attributes), Labels -- supported attributes: `blur`, `expression`, `illumination`, - `occluded` (both the annotation property & an attribute), `pose`, `invalid` +- supported attributes: + - `blur`, `expression`, `illumination`, `pose`, `invalid` + - `occluded` (both the annotation property & an attribute) -#### WIDER Face Loader +#### WIDER Face import Uploaded file: a zip archive of the structure above - supported annotations: Rectangles (with attributes), Labels -- supported attributes: `blur`, `expression`, `illumination`, `occluded`, `pose`, `invalid` +- supported attributes: + - `blur`, `expression`, `illumination`, `occluded`, `pose`, `invalid` ### [VGGFace2](https://github.com/ox-vgg/vgg_face2) -#### VGGFace2 Dumper +#### VGGFace2 export Downloaded file: a zip archive of the following structure: @@ -934,7 +1002,7 @@ label1 - supported annotations: Rectangles, Points (landmarks - groups of 5 points) -#### VGGFace2 Loader +#### VGGFace2 import Uploaded file: a zip archive of the structure above @@ -942,7 +1010,7 @@ Uploaded file: a zip archive of the structure above ### [Market-1501](https://www.aitribune.com/dataset/2018051063) -#### Market-1501 Dumper +#### Market-1501 export Downloaded file: a zip archive of the following structure: @@ -970,7 +1038,7 @@ s1 - sequence - supported annotations: Label `market-1501` with atrributes (`query`, `person_id`, `camera_id`) -#### Market-1501 Loader +#### Market-1501 import Uploaded file: a zip archive of the structure above @@ -978,7 +1046,7 @@ Uploaded file: a zip archive of the structure above ### [ICDAR13/15](https://rrc.cvc.uab.es/?ch=2) -#### ICDAR13/15 Dumper +#### ICDAR13/15 export Downloaded file: a zip archive of the following structure: @@ -1027,7 +1095,7 @@ taskname.zip/ - supported annotations: Rectangles and Polygons with label `icdar` and attributes `index`, `text`, `color`, `center` -#### ICDAR13/15 Loader +#### ICDAR13/15 import Uploaded file: a zip archive of the structure above diff --git a/cvat/apps/dataset_manager/formats/coco.py b/cvat/apps/dataset_manager/formats/coco.py index 3ec3ab18..3e4fb223 100644 --- a/cvat/apps/dataset_manager/formats/coco.py +++ b/cvat/apps/dataset_manager/formats/coco.py @@ -19,7 +19,8 @@ def _export(dst_file, task_data, save_images=False): dataset = Dataset.from_extractors(CvatTaskDataExtractor( task_data, include_images=save_images), env=dm_env) with TemporaryDirectory() as temp_dir: - dataset.export(temp_dir, 'coco_instances', save_images=save_images) + dataset.export(temp_dir, 'coco_instances', save_images=save_images, + merge_images=True) make_zip_archive(temp_dir, dst_file) diff --git a/cvat/apps/documentation/static/documentation/images/exported_cls_masks_example.png b/cvat/apps/documentation/static/documentation/images/exported_cls_masks_example.png new file mode 100644 index 00000000..e0dfa58a Binary files /dev/null and b/cvat/apps/documentation/static/documentation/images/exported_cls_masks_example.png differ diff --git a/cvat/apps/documentation/static/documentation/images/exported_inst_masks_example.png b/cvat/apps/documentation/static/documentation/images/exported_inst_masks_example.png new file mode 100644 index 00000000..d0362c80 Binary files /dev/null and b/cvat/apps/documentation/static/documentation/images/exported_inst_masks_example.png differ diff --git a/cvat/apps/documentation/static/documentation/images/label_color_picker.jpg b/cvat/apps/documentation/static/documentation/images/label_color_picker.jpg new file mode 100644 index 00000000..cd031476 Binary files /dev/null and b/cvat/apps/documentation/static/documentation/images/label_color_picker.jpg differ diff --git a/cvat/apps/documentation/static/documentation/images/label_panel_anno_window.jpg b/cvat/apps/documentation/static/documentation/images/label_panel_anno_window.jpg new file mode 100644 index 00000000..4e31455f Binary files /dev/null and b/cvat/apps/documentation/static/documentation/images/label_panel_anno_window.jpg differ diff --git a/cvat/apps/documentation/static/documentation/images/mask_export_example1_cls_mask.png b/cvat/apps/documentation/static/documentation/images/mask_export_example1_cls_mask.png new file mode 100644 index 00000000..19c1f05c Binary files /dev/null and b/cvat/apps/documentation/static/documentation/images/mask_export_example1_cls_mask.png differ diff --git a/cvat/apps/documentation/static/documentation/images/mask_export_example1_editor.png b/cvat/apps/documentation/static/documentation/images/mask_export_example1_editor.png new file mode 100644 index 00000000..a53e4ca1 Binary files /dev/null and b/cvat/apps/documentation/static/documentation/images/mask_export_example1_editor.png differ diff --git a/cvat/apps/documentation/static/documentation/images/mask_export_example1_inst_mask.png b/cvat/apps/documentation/static/documentation/images/mask_export_example1_inst_mask.png new file mode 100644 index 00000000..b32fcc4c Binary files /dev/null and b/cvat/apps/documentation/static/documentation/images/mask_export_example1_inst_mask.png differ diff --git a/cvat/apps/documentation/user_guide.md b/cvat/apps/documentation/user_guide.md index 2bd85467..bc21e2e5 100644 --- a/cvat/apps/documentation/user_guide.md +++ b/cvat/apps/documentation/user_guide.md @@ -29,6 +29,7 @@ - [OpenCV Tools](#opencv-tools) - [Annotation with rectangle by 4 points](#annotation-with-rectangle-by-4-points) - [Annotation with polygons](#annotation-with-polygons) + - [Creating masks](#creating-masks) - [Annotation with polylines](#annotation-with-polylines) - [Annotation with points](#annotation-with-points) - [Points in shape mode](#points-in-shape-mode) @@ -1282,6 +1283,67 @@ To edit a polygon you have to click on it while holding `Shift`, it will open th ![](static/documentation/images/gif007_mapillary_vistas.gif) +### Cutting holes in polygons + +Currently, CVAT does not support cutting transparent holes in polygons. However, +it is poissble to generate holes in exported instance and class masks. +To do this, one needs to define a background class in the task and draw holes +with it as additional shapes above the shapes needed to have holes: + +The editor window: + ![The editor](static/documentation/images/mask_export_example1_editor.png) + +Remember to use z-axis ordering for shapes by \[\-\] and \[\+\, \=\] keys. + +Exported masks: + ![A class mask](static/documentation/images/mask_export_example1_cls_mask.png) ![An instance mask](static/documentation/images/mask_export_example1_inst_mask.png) + +Notice that it is currently impossible to have a single instance number for +internal shapes (they will be merged into the largest one and then covered by +"holes"). + +### Creating masks + +There are several formats in CVAT that can be used to export masks: +- `Segmentation Mask` (PASCAL VOC masks) +- `CamVid` +- `MOTS` +- `ICDAR` +- `COCO` (RLE-encoded instance masks, [guide](https://github.com/openvinotoolkit/cvat/blob/develop/cvat/apps/dataset_manager/formats/README.md#coco)) +- `TFRecord` ([over Datumaro](https://github.com/openvinotoolkit/datumaro/blob/develop/docs/user_manual.md), [guide](https://github.com/openvinotoolkit/cvat/blob/develop/cvat/apps/dataset_manager/formats/README.md#tfrecord)): +- `Datumaro` + +An example of exported masks (in the `Segmentation Mask` format): + + ![A class mask](static/documentation/images/exported_cls_masks_example.png) ![An instance mask](static/documentation/images/exported_inst_masks_example.png) + +Important notices: +- Both boxes and polygons are converted into masks +- Grouped objects are considered as a single instance and exported as a single + mask (label and attributes are taken from the largest object in the group) + +#### Class colors + +All the labels have associated colors, which are used in the generated masks. +These colors can be changed in the task label properties: + + ![](static/documentation/images/label_color_picker.jpg) + +Label colors are also displayed in the annotation window on the right panel, +where you can show or hide specific labels +(only the presented labels are displayed): + + ![](static/documentation/images/label_panel_anno_window.jpg) + +A background class can be: +- A default class, which is implicitly-added, of black color (RGB 0, 0, 0) +- `background` class with any color (has a priority, name is case-insensitive) +- Any class of black color (RGB 0, 0, 0) + +To change backgound color in generated masks (default is black), +change `background` class color to the desired one. + + ## Annotation with polylines It is used for road markup annotation etc. diff --git a/cvat/requirements/base.txt b/cvat/requirements/base.txt index 2d5c8a1f..a6501a4e 100644 --- a/cvat/requirements/base.txt +++ b/cvat/requirements/base.txt @@ -50,4 +50,4 @@ open3d==0.11.2 # --no-binary=pycocotools: workaround for binary incompatibility on numpy 1.20 # of pycocotools and tensorflow 2.4.1 # when pycocotools is installed by wheel in python 3.8+ -datumaro==0.1.7 --no-binary=datumaro --no-binary=pycocotools +datumaro==0.1.8 --no-binary=datumaro --no-binary=pycocotools