Computer Vision Annotation Tool (CVAT) is a web-based tool which helps to annotate video and images for Computer Vision algorithms. It was inspired by [Vatic](http://carlvondrick.com/vatic/) free, online, interactive video annotation tool. CVAT has many powerful features: __interpolation of bounding boxes between key frames, automatic annotation using TensorFlow OD API, shortcuts for most of critical actions, dashboard with a list of annotation tasks, LDAP and basic authorization, etc...__ It was created for and used by a professional data annotation team. UX and UI were optimized especially for computer vision tasks developed by our team.
## Getting started
@ -30,7 +28,7 @@ There you can:
2. Specify mandatory parameters of the task. You have to fill ``Name``, ``Labels`` and ``Select Files`` at least.
2. Specify mandatory parameters of the task. You have to fill ``Name``, ``Labels`` and press ``Select Files`` at least.
@ -52,16 +50,20 @@ There you can:
Default value for an attribute is the first value after "``:``".
For ``select`` and ``radio`` input types the special value is available: ``__undefined__``. Specify this value first if an attribute should be annotated explicity.
For ``select`` and ``radio`` input types the special value is available: ``__undefined__``. Specify this value first if an attribute should be annotated explicitly.
__Bug Tracker__. Specify full URL your bug tracker if you have it.
__Source__. To create huge tasks please use ``shared`` server directory (choose ``Share`` option in the dialog).
__Flip images__. All selected files will be turned around 180.
__Z-Order__. Defines the order on drawn polygons. Check the box for enable layered dislaying.
__Overlap Size__. Use this option to make overlapped segments. The option makes tracks continuous from one segment into another. Use it for interpolation mode.
__Segment size__. Use this option to divide huge dataset on several segments.
__Segment size__. Use this option to divide huge dataset by a few less size segments.
__Image Quality__. Use this option to specify quality of uploaded images. The option makes it faster to load high-quality datasets. Use the value from ``1`` (completely compressed images) to ``95`` (almost not compressed images).
@ -83,24 +85,40 @@ There you can:
3. An image can be moved/shifted by holding left mouse button inside some area without annotated objects. If ``Shift`` key is pressed then all annotated objects are ignored otherwise a highlighted bounding box will be moved instead of the image itself. Usually the functionality is used together with zoom to precisely locate an object of interest.
### Types of Shapes (basic)
There are four shapes you can annotate your images with:
- Choose an object's ``label``. When you created an annotation task you had to specify one or several labels with attributes.
- Choose right ``Shape`` (e.g. box) and ``Label`` (was specified by you while creating the task) beforehand:
- Create a bounding box by clicking on ``Create Track`` button or ``N`` shortcut. Choose left top and right bottom points. Your first bounding box is ready! It is possible to adjust boundaries and location of the bounding box using mouse.
- Create a bounding box by clicking on ``Create Shape`` button or ``N`` shortcut. Choose opposite points. Your first bounding box is ready!
- It is possible to adjust boundaries and location of the bounding box using mouse. In the top right corner size of the box is shown. You can also undo/redo your actions by using ``Ctrl+Z`` / ``Shift+Ctrl+Z Ctrl+Y``.
3. In the list of objects you can see the labeled car. In the side panel you can perform basic operations under the object.
@ -139,7 +157,7 @@ Usage examples:
4. When the annotated object disappears or becomes too small, you need to finish the track. To do that you need to choose ``Outsided Property``.
4. When the annotated object disappears or becomes too small, you need to finish the track. To do that you need to choose ``Outside Property``.
@ -168,30 +186,20 @@ Usage examples:
### Attribute Annotation mode (basics)
Usage examples:
- Edit attributes using keyboard with fast navigation between objects and frames.
1. To enter into ``Attribute Annotation`` mode press ``Shift+Enter`` shortcut. After that it is possible to change attributes using keyboard.
- In this Mode you can edit attributes with fast navigation between objects and frames using keyboard. Press ``Shift+Enter`` shortcut to enter AAMode. After that it is possible to change attributes using keyboard.
2. The active attribute will be red. In this case it is ``Age``.
3. Look at the bottom side panel to see all possible shortcuts to change the attribute. Press ``4`` key on your keyboard to assign ``adult`` value for the attribute.
- The active attribute will be red. In this case it is ``Gender``. Look at the bottom side panel to see all possible shortcuts to change the attribute. Press ``2`` key on your keyboard to assign ``female`` value for the attribute.
4. Press ``Up Arrow``/``Down Arrow`` keys on your keyboard to go to next attribute .
- Press ``Up Arrow``/``Down Arrow`` keys on your keyboard to go to next attribute. In this case after pressing ``Down Arrow`` you will be able to edit ``Age`` attribute.
5. In this case after pressing ``Down Arrow`` you will be able to edit ``Gender`` attribute.
6. Use ``Right Arrow``/``Left Arrow`` keys to move on previous/next image.
- Use ``Right Arrow``/``Left Arrow`` keys to move on previous/next image with annotation.
### Downloading annotations
@ -201,8 +209,9 @@ Usage examples:
3. The annotation will be written into **.xml** file. To find the annotation file go to the directory where your browser saves downloaded files by default. For more information visit [XML annotation format](/documentation/xml_format.html) description.
3. The annotation will be written into **.xml** file. To find the annotation file go to the directory where your browser saves downloaded files by default. For more information visit [.xml format page](./documentation/xml_format.html).
There is ``Player Settings`` which adjusting ``Workspace`` and ``Other Settings``.
In ``Player Settings`` you can:
- Control step of ``C`` and ``V`` shortcuts
- Control speed of ``Space``/Play button
- Show every image in full or zoomed out like previous (reset by default)
- Enable ``Grid`` when you don't need too small annotation. ``Grid`` can have different opacity, color and cells' size — use ``F2`` to tune settings.
- Adjust ``Brightness``/``Contrast``/``Saturation`` of too expose or too dark images using ``F2`` — color settings (changes displaying and not the image itself).
Shortcuts:
- ``Shift+B``/``Alt+B`` for brightness
- ``Shift+C``/``Alt+C`` for contrast
- ``Shift+S``/``Alt+S`` for saturation
- ``F2`` —> ``Reset Color Settings`` for default image
``Other Settings`` contain:
- ``Show All Interpolation Tracks`` checkbox — shows hidden object on side panel for every interpolated object (turned off by default)
- ``AAM Zoom Margin`` slider — defines margins for shape in attribute annotation mode
- ``Enable AutoSaving`` checkbox — turned off by default
- ``Propagate Frames`` input box — allow to choose on how many frames selected object will be copied in by ``Ctrl+B`` (50 by default)
---
### Navigation by frames/images
### Bottom Panel
@ -254,102 +314,139 @@ Go to the first and latest frames.
---
Go to the next/previous frame with a predefined step. Shortcuts: ``v`` — step backward, ``c`` — step forward. By default the step is ``10``.
Go to the next/previous frame with a predefined step. Shortcuts: ``v`` — step backward, ``c`` — step forward. By default the step is ``10`` (change at ``Open Menu`` —> ``Settings`` —> ``Player Step``).
To change the predefined step go to settings (``Open Menu`` —> ``Settings``) and modify ``Player Step`` property.
---
Go to the next/previous frame with step equals to 1. Shortcuts: ``d`` — previous, ``f`` — next.
---
Play the sequence of frames or the set of images. Shortcut: ``Space``.
Play the sequence of frames or the set of images. Shortcut: ``Space`` (change at ``Open Menu`` —> ``Settings`` —> ``Player Speed``).
To adjust player speed go to settings (``Open Menu`` —> ``Settings``) and modify a value of ``Player Speed`` property.
---
Go to specified frame. Press ``~`` to highlight element.
Go to specified frame.
---
__Open Menu__ button
It is the main menu for the annotation tool. It can be used to download, upload and remove annotations. As well it shows statistics about the current annotation task.
### Bottom side panel
---
__Filter__ input box
How to use filters is described in the Advanced guide (below).
### Side panel (list of objects)
---
__History / Undo-Redo panel__
In the side panel you can see the list of available objects on the current frame. An example how the list can look like below:
Use shortcuts for undo/redo actions ``Ctrl+Z`` __/__ ``Ctrl+Shift+Z``/``Ctrl+Y``
### Side panel
#### Objects
In the Side Panel you can see the list of available objects on the current frame. An example how the list can look like below:
You also can see all labels that used on this frame and highlight them by clicking needed label.
A bounding box can be locked to prevent its modification or moving by an accident. Shortcut to lock an object: ``l``.
---
A bounding box can be removed. Shortcut: ``Delete``. A locked bounding box can be deleted using ``Shift+Delete`` shortcut.
__Objects' card__
A shape can be removed. Shortcut: ``Delete``. A locked shape can be deleted using ``Shift+Delete`` shortcut.
---
A bounding box can be **Occluded**. Shortcut: ``q``. Such bounding boxes have dashed boundaries.
A shape can be locked to prevent its modification or moving by an accident. Shortcut to lock an object: ``l``.
---
A shape can be **Occluded**. Shortcut: ``q``. Such shapes have dashed boundaries.
---
The type of a bounding box can be changed by selecting __Label__ property. For instance, it can look like on the figure below:
You can copy and paste this object on this or other frame. ``Ctrl+C``/``Ctrl+V`` shortcuts works under mouse point.
---
You can propagate this object on next X frames. ``Ctrl+B`` shortcut works under mouse point. ``F2`` for change on how many frames to propagate this object.
---
You can change how this objects' annotation is displayed on this frame. It could be Hide, Shows Only Box, Shows Box and Title. ``H`` is for this object, ``T+H`` for all objects on this frame.
---
The type of a shape can be changed by selecting __Label__ property. For instance, it can look like on the figure below:
To change a type of a bounding box using keyboard you need to press ``Shift+<number>``.
To change a type of a highlighted shape using keyboard you need to press ``Shift+<number>``.
### Open Menu
It is the main menu for the annotation tool. It can be used to download, upload and remove annotations. As well it shows statistics about the current annotation task.
### Bottom side panel
- ``Create Shape`` (``N``) — start/stop draw new shape mode
- ``Working Mode`` — Annotation or Interpolation modes. You can't interpolate Polygons/Polylines/Points, but you can propagate them using ``Ctrl+B`` or merge into a track
- ``Poly Shape Size`` — (optional) hard number of dots for creating Polygon/Polyline shapes
### Settings
The menu contains different parameters which can be adjust by the user needs. For example, ``Auto Saving Internal``, ``Player Step``, ``Player Speed``.
#### Fullscreen player mode
Go to ``Open Menu`` —> ``Fullscreen Player``
Exit with ``F11`` or ``Ecs``.
That is how it looks like.
- ``Brightness`` makes it appear that there is more or less light within the image.
- ``Contrast`` controls the difference between dark and light parts of the image
- ``Saturation`` takes away all color or enhance the color.
## Annotation mode (advanced)
Basic operations in the mode was described above.
__occluded__ attribute is used if an object is occluded by another object or it isn't fully visible on the frame. Use ``q`` shortcut to set the property quickly.
__occluded__ attribute is used if an object is occluded by another object or it isn't fully visible on the frame. Use ``Q`` shortcut to set the property quickly.
Example: both cars on the figure below should be labeled as __occluded__.
If a frame contains too many objects and it is difficult to annotate them due to many bounding boxes are placed mostly in the same place when it makes sense to lock them. Bounding boxes for locked objects are transparent and it is easy to annotate new objects. Also it will not be possible to change previously annotated objects by an accident. Shortcut: ``l``.
If a frame contains too many objects and it is difficult to annotate them due to many shapes are placed mostly in the same place when it makes sense to lock them. Shapes for locked objects are transparent and it is easy to annotate new objects. Also it will not be possible to change previously annotated objects by an accident. Shortcut: ``L``.
## Interpolation mode (advanced)
@ -373,31 +470,118 @@ It is possible to handle many objects on the same frame in the mode.
It is more convenient to annotate objects of the same type. For the purpose it is possible to specify a corresponding filter. For example, the following filter will hide all objects except pedestrians: ``//pedestrian``.
It is more convenient to annotate objects of the same type. For the purpose
it is possible to specify a corresponding filter. For example, the following
filter will hide all objects except pedestrians: ``pedestrian``.
To navigate between objects (pedestrians in the case) use the following shortcuts:
- ``Tab`` - go to the next object
- ``Shift+Tab`` - go to the previous object.
- ``Tab`` — go to the next object
- ``Shift+Tab`` — go to the previous object.
By default in the mode objects are zoomed in to full screen. Check
Be sure ``Z-Order`` flag in ``Create task`` dialog is enabled if you want annotate polygons. Z-Order flag defines order of drawing. It is necessary to get right annotation mask without extra work (additional drawing of borders). Z-order can be changed by `+`/`-` which set maximum/minimum z-order respectively.
Before start need to be sure that ``Polygon`` is selected.
Click ``N`` for entering drawing mode. Now you can start your polygon.
You can zoom in/out (on mouse wheel scroll) and move (on mouse wheel press
and mouse move) while drawing. Click ``N`` again for completing the shape.
Also you can set fixed number of points in the field "Poly Shape Size", then
drawing will be stopped automatically. You can drag object after it was drawn
and fix a position of an individual points after finishing the object. You
can add/delete points after finishing.
Below you can see results with opacity and black stroke:
Also if you need annotate small objects, increase ``Image Quality`` to ``95`` in ``Create task`` dialog for annotators convenience.
## Annotation with polylines
It is used for road markup annotation etc.
Before start need to be sure that ``Polyline`` is selected.
By default in the mode objects are zoomed. To disable the functionality uncheck the corresponding setting: ``Open Menu`` —> ``Settings`` —> ``Zoom boxes in Attribute Annotation Mode``.
By default other objects are hidden. To change the behaviour uncheck the corresponding setting: ``Open Menu`` —> ``Setting`` —> ``Hide Other in Attribute Annotation Mode``.
Click ``N`` for entering drawing mode. Now you can start your polyline.
You can zoom in/out (on mouse wheel scroll) and move (on mouse wheel press and
mouse move) while drawing. Click ``N`` again for completing the shape. Also
you can set fixed number of points in the field "Poly Shape Size", then drawing
will be stopped automatically. You can drag object after it was drawn and fix
a position of an individual points after finishing the object. You can
add/delete points after finishing.
## Annotation with points
It is used for face landmarks annotation etc.
Before start need to be sure that ``Points`` is selected.
Click ``N`` for entering drawing mode. Now you can start marking a needed area.
Click ``N`` again for finishing marking an area. Also you can set fixed number
of points in the field "Poly Shape Size", then drawing will be stopped
automatically. Points are automatically grouped — between individual start
and finish all points will be considered linked. You can zoom in/out (on mouse
wheel scroll) and move (on mouse wheel press and mouse move) while drawing.
You can drag object after it was drawn and fix a position of an individual
points after finishing the object. You can add/delete points after finishing.
## Shape grouping
This feature allows to group several shapes.
You may use ``Group Shapes`` button or shortcuts:
- ``G`` — switch group mode
- ``Alt+G`` — close group mode
- ``Shift+G`` — reset group for selected shapes
You may select shapes by click or by area selection.
Grouped shapes will have ``group_id`` filed in dumped annotation.
Also you may switch color distribution from by instance (default) to by group. For it need switch ``Color By Group`` checkbox.
Shapes which haven't ``group_id`` will be highlighted with white color.
## Filter
There are several reasons to use the feature:
1. When use a filter objects which don't correspond to the filter will be hidden. Use ``Settings`` —> ``Hide Filtered Tracks`` or ``K`` shortcut if you want to change the behaviour.
2. Fast navigation between frames which have an object of interest. Use ``Left Arrow/Right Arrow`` keys for the purpose. If the filter is empty the mentioned arrows will go to previous/next frames.
1. When use a filter objects which don't correspond to the filter will be hidden.
2. Fast navigation between frames which have an object of interest. Use ``Left Arrow/Right Arrow`` keys for the purpose. If the filter is empty the mentioned arrows will go to previous/next frames which contain any objects.
To use the functionality it is enough to specify a value inside ``Filter`` text box and defocus the text box (for example, click on the image). After that the filter will be applied.
---
In a trivial case a correct filter should correspond to the template: ``//label[prop operator "value"]``
In a trivial case a correct filter should correspond to the template: ``label[prop operator "value"]``
``label`` is a type of an object (e.g _person, car, face_, etc). If the type isn't important you can use ``*``.
``label`` is a type of an object (e.g _person, car, face_, etc.). If the type isn't important you can use ``*``.
``prop`` is a property which should be filtered. The following items are available:
@ -417,13 +601,13 @@ In a trivial case a correct filter should correspond to the template: ``//label[
Example | Description
-------------------------------|-------------
``//face`` | all faces
``//*[id=4]`` | object with id #4
``//*[type="annotation"]`` | *annotation* objects only
``//car[occluded="true"]`` | cars with *occluded* property
``//*[lock!="true"]`` | all unlocked objects
``//car[attr/parked="true"]`` | parked cars
``//*[attr/*="__undefined__"]``| any objects with ``__undefined__`` value of an attribute
``face`` | all faces
``*[id=4]`` | object with id #4
``*[type="annotation"]`` | *annotation* objects only
``car[occluded="true"]`` | cars with *occluded* property
``*[lock!="true"]`` | all unlocked objects
``car[attr/parked="true"]`` | parked cars
``*[attr/*="__undefined__"]``| any objects with ``__undefined__`` value of an attribute
---
@ -431,47 +615,55 @@ The functionality allows to create more complex conditions. Several filters can
Nikita Manovich
8 years ago
