You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
156 lines
7.0 KiB
Markdown
156 lines
7.0 KiB
Markdown
---
|
|
title: 'AI Tools'
|
|
linkTitle: 'AI Tools'
|
|
weight: 14
|
|
description: 'Overview of semi-automatic and automatic annotation tools available in CVAT.'
|
|
---
|
|
|
|
The tool is designed for semi-automatic and automatic annotation using DL models.
|
|
The tool is available only if there is a corresponding model.
|
|
For more details about DL models read the [Models](/docs/manual/advanced/models/) section.
|
|
|
|
## Interactors
|
|
|
|
Interactors are used to create a polygon semi-automatically.
|
|
Supported DL models are not bound to the label and can be used for any objects.
|
|
To create a polygon usually you need to use regular or positive points.
|
|
For some kinds of segmentation negative points are available.
|
|
Positive points are the points related to the object.
|
|
Negative points should be placed outside the boundary of the object.
|
|
In most cases specifying positive points alone is enough to build a polygon.
|
|
A list of available out-of-the-box interactors is placed below.
|
|
|
|
- Before you start, select the `magic wand` on the controls sidebar and go to the `Interactors` tab.
|
|
Then select a label for the polygon and a required DL model. To view help about each of the
|
|
models, you can click the `Question mark` icon.
|
|
|
|
|
|
|
|
- Click `Interact` to enter the interaction mode. Depending on the selected model,
|
|
the method of markup will also differ.
|
|
Now you can place positive and/or negative points. The [IOG](#inside-outside-guidance) model also uses a rectangle.
|
|
Left click creates a positive point and right click creates a negative point.
|
|
After placing the required number of points (the number is different depending on the model),
|
|
the request will be sent to the server and when the process is complete a polygon will be created.
|
|
If you are not satisfied with the result, you can set additional points or remove points.
|
|
To delete a point, hover over the point you want to delete, if the point can be deleted,
|
|
it will enlarge and the cursor will turn into a cross, then left-click on the point.
|
|
If you want to postpone the request and create a few more points, hold down `Ctrl` and continue (the `Block`
|
|
button on the top panel will turn blue), the request will be sent after the key is released.
|
|
|
|
|
|
|
|
- In the process of drawing, you can select the number of points in the polygon using the switch.
|
|
|
|
|
|
|
|
- You can use the `Selected opacity` slider in the `Objects sidebar` to change the opacity of the polygon.
|
|
You can read more in the [Objects sidebar](/docs/manual/basics/objects-sidebar/#appearance) section.
|
|
|
|
- To finish interaction, click on the `Done` button on the top panel or press `N` on your keyboard.
|
|
|
|
- When the object is finished, you can edit it like a polygon.
|
|
You can read about editing polygons in the [Annotation with polygons](/docs/manual/advanced/annotation-with-polygons/) section.
|
|
|
|
### Deep extreme cut (DEXTR)
|
|
|
|
This is an optimized version of the original model, introduced at the end of 2017.
|
|
It uses the information about extreme points of an object to get its mask. The mask then converted to a polygon.
|
|
For now this is the fastest interactor on CPU.
|
|
|
|
|
|
|
|
### Feature backpropagating refinement scheme (f-BRS)
|
|
|
|
The model allows to get a mask for an object using positive points
|
|
(should be left-clicked on the foreground), and negative points
|
|
(should be right-clicked on the background, if necessary).
|
|
It is recommended to run the model on GPU, if possible.
|
|
|
|
|
|
|
|
### High Resolution Net (HRNet)
|
|
|
|
The model allows to get a mask for an object using positive points
|
|
(should be left-clicked on the foreground), and negative points
|
|
(should be right-clicked on the background, if necessary).
|
|
It is recommended to run the model on GPU, if possible.
|
|
|
|
|
|
|
|
### Inside-Outside-Guidance
|
|
|
|
The model uses a bounding box and inside/outside points to create a mask.
|
|
First of all, you need to create a bounding box, wrapping the object.
|
|
Then you need to use positive and negative points to say the model where is a foreground,
|
|
and where is a background. Negative points are optional.
|
|
|
|
|
|
|
|
## Detectors
|
|
|
|
Detectors are used to automatically annotate one frame. Supported DL models are suitable only for certain labels.
|
|
|
|
- Before you start, click the `magic wand` on the controls sidebar and select the `Detectors` tab.
|
|
You need to match the labels of the DL model (left column) with the labels in your task (right column).
|
|
Then click `Annotate`.
|
|
|
|
|
|
|
|
- Some of models supports attributes annotation (like facial emotions, for example: ``serverless/openvino/omz/intel/face-detection-0205``).
|
|
In this case you can also match attributes of the DL model with the attributes of a CVAT label.
|
|
|
|
|
|
|
|
- This action will automatically annotates one frame.
|
|
In the [Automatic annotation](/docs/manual/advanced/automatic-annotation/) section you can read
|
|
how to make automatic annotation of all frames.
|
|
|
|
### Mask RCNN
|
|
|
|
The model generates polygons for each instance of an object in the image.
|
|
|
|
### Faster RCNN
|
|
|
|
The model generates bounding boxes for each instance of an object in the image. In this model,
|
|
RPN and Fast R-CNN are combined into a single network.
|
|
|
|
## Trackers
|
|
|
|
Trackers are used to automatically annotate an object using bounding box.
|
|
Supported DL models are not bound to the label and can be used for any objects.
|
|
|
|
- Before you start, select the `magic wand` on the controls sidebar and go to the `Trackers` tab.
|
|
Then select a `Label` and `Tracker` for the object and click `Track`. Then annotate the desired objects with the
|
|
bounding box in the first frame.
|
|
|
|
|
|
|
|
- All annotated objects will be automatically tracked when you move to the next frame.
|
|
For tracking, use `Next` button on the top panel or the `F` button to move on to the next frame.
|
|
|
|
|
|
|
|
- You can enable/disable tracking using `tracker switcher` on sidebar.
|
|
|
|
|
|
|
|
- Trackable objects have indication on canvas with a model indication.
|
|
|
|
|
|
|
|
- You can monitoring the process by the messages appearing at the top.
|
|
If you change one or more objects, before moving to the next frame, you will see a message that
|
|
the objects states initialization is taking place. The objects that you do not change are already on the server
|
|
and therefore do not require initialization. After the objects are initialized, tracking will occur.
|
|
|
|
|
|
|
|
### SiamMask
|
|
|
|
Fast online Object Tracking and Segmentation. Tracker is able to track different objects in one server request.
|
|
Trackable object will be tracked automatically if the previous frame was
|
|
a latest keyframe for the object. Have tracker indication on canvas. `SiamMask` tracker supported CUDA.
|
|
|
|
> If you plan to track simple non-overlapping objects consider using fast client-side [TrackerMIL from OpenCV](/docs/manual/advanced/opencv-tools/#trackermil).
|