78 Commits (d18e1db99816c8143d99844f5a5e4e1012b557a8)

Author SHA1 Message Date
zhiltsov-max 0053411f6e
[Datumaro] Add debug output in VOC import (#1662)
* Add debug output in VOC import
6 years ago
zhiltsov-max 696a47b3c5
Add assets importing tests for cvat format (#1652) 6 years ago
zhiltsov-max ba309c882f
[Datumaro] Add DatasetItem attributes (#1639)
* Add DatasetItem attributes

* Update tests

* Update datumaro format
6 years ago
zhiltsov-max 1b413c65a2
Fix image size conversion (#1593)
* Fix image size conversion

* update changelog

Co-authored-by: Nikita Manovich <40690625+nmanovic@users.noreply.github.com>
6 years ago
zhiltsov-max 727fcd50b3
[Datumaro] Remove annotation dir for cvat dataset format (#1637)
* Remove annotation dir for cvat dataset format

* fixes
6 years ago
zhiltsov-max 2044d35566
Add directory check on image save (#1636)
* Add directory check on image save

* Add tests

* change exception type

* update test
6 years ago
zhiltsov-max 306e91c3d7
[Datumaro] Add tests for dataset examples (#1648)
* add dataset examples

* update docs

* update yolo tests

* join voc format test classes

* remplace voc extractor tests with import test

* update tfrecord format tests

* update mot tests

* update labelme tests

* update image dir tests
6 years ago
zhiltsov-max d1b2960b3d
[Datumaro] Add dataset examples (#1640)
* add dataset examples
6 years ago
zhiltsov-max da3fa34ce0
Add tf import checks and tests (#1567)
* Add tf import checks and tests

* implement disabled formats on server

* python 3.5 compatibility

* add checks to dm tests

* fix tests

* Support for disabled formats in UI

* add sorting for formats, mark grey disabled items

* update changelog

* advance package versions
6 years ago
zhiltsov-max ac922e33a5
Add default colors for masks (#1549)
* Use default colors for masks

* do not change labels in voc converter

* update tests

* add backgound color handling

* consider case with existing source colormap

* update changelog

* fix file name

* fix insert

* move color table colors to beginning
6 years ago
zhiltsov-max e2dedb2f83
Use z_order as a class property (#1589)
* Use z_order as a class property

* Fix z_order use in voc

* Update changelog
6 years ago
zhiltsov-max 1200ecf67c
[Datumaro] Simplify log level setting (#1583)
* Simplify loglevel setting

* update changelog
6 years ago
zhiltsov-max 42c5637ddd
[Datumaro] Fix mask to polygons warning (#1581)
* Fix message, add test
* update changelog
6 years ago
zhiltsov-max 4c603504b7
Add VOC grayscale masks test and documentation (#1576)
* Add a test for unpainted masks
* Update format documentation
6 years ago
zhiltsov-max fb380d9855
Include empty images in exported annotations (#1479) 6 years ago
zhiltsov-max 98a9718e63
[Datumaro] Change alignment in mask parsing (#1547) 6 years ago
zhiltsov-max 42990906b9
[Datumaro] Fix coco import conflict with labels (#1548) 6 years ago
zhiltsov-max 5ab549956f
Merge annotations and dataset_manager apps (#1352) 6 years ago
zhiltsov-max c78db45d9a
[Datumaro] Fix duplicating keypoints in COCO export (#1435) 6 years ago
zhiltsov-max 819b81d7f3
Fix label comparison in voc format (#1382) 6 years ago
zhiltsov-max bd659246e9
[Datumaro] Fix COCO keypoint export bug (#1388) 6 years ago
zhiltsov-max afeab69aa3
Save full image paths in coco (#1381) 6 years ago
zhiltsov-max 887c6f0432
Move annotation formats to dataset manager (#1256)
* Move formats to dataset manager

* Unify datataset export and anno export implementations

* Add track_id to TrackedShape, export tracked shapes

* Replace MOT format

* Replace LabelMe format

* Add new formats to dm

* Add dm tests

* Extend TrackedShape

* Enable dm test in CI

* Fix tests

* Add import

* Fix tests

* Fix mot track ids

* Fix mot format

* Update attribute logic in labelme tests

* Use common code in yolo

* Put datumaro in path in settings

* Expect labels file in MOT next to annotations file

* Add MOT format description

* Add import

* Add labelme format description

* Linter fix

* Linter fix2

* Compare attributes ordered

* Update docs

* Update tests
6 years ago
Andrey Zhavoronkov e7808cfb03
Data streaming using chunks (#1007)
Huge feature (200+ commits from different developers). It completely changes layout of data (please expect very long DB migration process if you have a lot of tasks). The primary idea is to send data as zip chunks (e.g. 36 images in one chunk) or encoded video chunks and decode them on the client side. It helps to solve the problem with latency  when you try to view a separate frame in the UI quickly (play mode).
Another important feature of the patch is to provide access to the original images. Thus for annotations the client uses compressed chunks but if you want to export a dataset Datumaro will use original chunks (but video will be decoded with original quality and encoded with maximum/optimal quality in any case).
6 years ago
zhiltsov-max 6a4ccdea1f
[Datumaro] Extract common extractor functionality (#1319)
* Extract common extractor functionality

* Simplify coco extractor

* Fix tfrecord
6 years ago
zhiltsov-max 03817f050b
[Datumaro] Fix image merging (#1301)
* Always merge images for own dataset

* Fix codacy
6 years ago
zhiltsov-max 4f20f46cf4
Fix attributes with spaces in names (#1305) 6 years ago
zhiltsov-max fe862b4abc
[Datumaro] Update LabelMe format (#1296)
* Little refactoring

* Add LabelMe format

* Add usernames

* Update tests

* Add extractor test
6 years ago
zhiltsov-max c91e8957df
[Datumaro] LabelMe format (#1293)
* Little refactoring

* Add LabelMe format
6 years ago
zhiltsov-max 0744c6ae16
[Datumaro] MOT format (#1289)
* Add mot format base

* Add mot format

* Extract common code
6 years ago
zhiltsov-max 8efaf58f11
Add recursive importers (#1290) 6 years ago
zhiltsov-max 9545132097
[Datumaro] Allow empty COCO dataset export (#1272)
* Allow empty dataset export in coco

* Add line to changelog

Co-authored-by: Nikita Manovich <40690625+nmanovic@users.noreply.github.com>
6 years ago
zhiltsov-max 936d3059cf
[Datumaro] Fix frame matching in video annotations import (#1274)
* Add extra frame matching way for videos

* Add line to changelog
6 years ago
Nikita Manovich ec2fa6ee51
Fixed security issues in Datumaro (#1244)
* Fixed security issues reported by bandit.
* Fixed voc_format extractor
* Sorted requirements, added a comment, removed nosec for exec.
6 years ago
zhiltsov-max 2ebca5bb5a
[Datumaro] Dataset format auto detection (#1242)
* Add dataset format detection

* Add auto format detection for import

* Split VOC extractor
6 years ago
Nikita Manovich 24130cda41
[Datumaro] Optimize mask operations (#1232)
* Optimize mask to rle

* Optimize mask operations

* Fix dm format cmdline

* Use RLE masks in datumaro format
6 years ago
Nikita Manovich be5577d013
[Datumaro] Label remapping transform (#1233)
* Add label remapping transform

* Apply transforms before project saving

* Refactor voc converter
6 years ago
zhiltsov-max b3f7f5b8bc
Add tags support for VOC (#1201)
* Extend voc format test with tags

* Add import and export of voc labels

* Fix voc and yolo format version numbers
6 years ago
zhiltsov-max 0d873a3de6
Fix remainder logic for subset splitting (#1222) 6 years ago
zhiltsov-max a058765d29
Add tags to cvat xml (#1200)
* Extend cvat format test

* Add tags to cvat for images

* Add tags to cvat format in dm

* Add import of tags from datumaro
6 years ago
zhiltsov-max da69a40b96
Add random split transform (#1213) 6 years ago
zhiltsov-max 3604f0c5ce
Auto-generate labelmap for voc from task (#1214) 6 years ago
zhiltsov-max 79c30696dd
Move project dir to .datumaro (#1207) 6 years ago
zhiltsov-max 54c277a0c9
Replace COCO implementation (#1195) 6 years ago
zhiltsov-max f2dfcc039b
[Datumaro] Remote project export fixes (#1193)
* Export project with trask name

* Do not expose server paths

* Fix tfrecord mask reading in tf>1.14

* Setuptools compatibility
6 years ago
zhiltsov-max cd8ef2aca4
Replace VOC format support in CVAT with Datumaro (#1167)
* Add image meta reading to voc

* Replace voc support in cvat

* Bump format version

* Materialize lazy transforms in voc export

* Store voc instance id as group id

* Add flat format import

* Add documentation

* Fix format name in doc
6 years ago
zhiltsov-max 8caa1695c5
Replace mask format support with Datumaro (#1163)
* Add box to mask transform

* Fix 'source' labelmap mode in voc converter

* Import groups

* Replace mask format support

* Update mask format documentation

* codacy

* Fix tests

* Fix dataset

* Fix segments grouping

* Merge instances in mask export
6 years ago
zhiltsov-max 80d3f97583
Replace tfrecord format support in CVAT with Datumaro (#1157) 6 years ago
zhiltsov-max b36f402f9c
[Datumaro] Add masks to tfrecord format (#1156)
* Employ transforms and item wrapper

* Add image class and tests

* Add image info support to formats

* Fix cli

* Fix merge and voc converte

* Update remote images extractor

* Codacy

* Remove item name, require path in Image

* Merge images of dataset items

* Update tests

* Add image dir converter

* Update Datumaro format

* Update COCO format with image info

* Update CVAT format with image info

* Update TFrecord format with image info

* Update VOC formar with image info

* Update YOLO format with image info

* Update dataset manager bindings with image info

* Add image name to id transform

* Fix coco export

* Add masks support for tfrecord

* Refactor coco

* Fix comparison

* Remove dead code

* Extract common code for instances
6 years ago
zhiltsov-max a376ee76fd
[Datumaro] Introduce image info (#1140)
* Employ transforms and item wrapper

* Add image class and tests

* Add image info support to formats

* Fix cli

* Fix merge and voc converte

* Update remote images extractor

* Codacy

* Remove item name, require path in Image

* Merge images of dataset items

* Update tests

* Add image dir converter

* Update Datumaro format

* Update COCO format with image info

* Update CVAT format with image info

* Update TFrecord format with image info

* Update VOC formar with image info

* Update YOLO format with image info

* Update dataset manager bindings with image info

* Add image name to id transform

* Fix coco export
6 years ago