57 Commits (cb512fb5848250c13cf1c32ad2d47a99382d3672)

Author SHA1 Message Date
Roman Donchenko 580ae49e5a
Migrate to Docker Compose V2 (#5524)
Also, migrate to the version less Compose file format.

Compose V1 is EOL:
<https://www.docker.com/blog/announcing-compose-v2-general-availability/>.
3 years ago
Maxim Zhiltsov ec3e1f34a4
Better reporting for user limits (#5225)
- Added explanatory messages for actions denied for user limits
- Fixed few rules and checks
- Upgraded OPA version
3 years ago
Maria Khrustaleva e624c5b959
Simplify upload data for task (#5498)
It's possible to specify only the manifest file and filename pattern for
creating task with cloud storage data.
The special characters supported now for the pattern are `*`, `?`,
`[seq]`, `[!seq]`.
Please see
[here](8898a8b264/tests/python/rest_api/test_tasks.py (L686))
for some examples of how to use this functionality.

Co-authored-by: Maxim Zhiltsov <zhiltsov.max35@gmail.com>
3 years ago
Maxim Zhiltsov af65707eee
Mark tests that require external services (#5474)
Related: #5225 

External services are not available when we execute in Helm.

- Added a way to mark REST API tests that require external services to
run
- Changed the way of filtering tests in Helm tests

Currently, we can't execute external service mocks in tests, and we
ignore related tests in the Helm
execution command. But this command may be outdated, because Helm tests
are not executed in each PR.
This PR allows to indicate related tests and filter them out without the
need to synchronize the CI command.
3 years ago
Maxim Zhiltsov 760f40d3d8
Wait for OPA to load rules in tests startup (#5483)
OPA can take some time to load rules, but our tests don't wait for OPA,
and start right after the server is loaded.
Sometimes it works, but in other times the tests may fail because OPA is
still loading the rules.
This PR allows to wait for OPA during the test suite startup.
3 years ago
Maxim Zhiltsov 37b685f47a
Limit test execution time (#5484)
This should protect us from unnoticed hangs, which happen sometimes and
waste CI and developer time.
Currently, the limit is 15s per test, which should be enough in normal
conditions.
3 years ago
Roman Donchenko a6e5813233
Make the Python test log less messy (#5456) 3 years ago
Roman Donchenko fa92ccb987
SDK: Improve the PyTorch adapter layer (#5455)
* Make the extractors return tensors instead of Python data structures.
* Let the user specify custom label IDs.
3 years ago
Andrey Zhavoronkov 980c019427
PostgresSQL DB v15 and health check endpoint (#5312) 3 years ago
Maxim Zhiltsov 6cf67dba9e
Add project proxy tests in SDK (#5431)
Ported from https://github.com/opencv/cvat/pull/4819

- Fixed project APIs in SDK core
- Added tests for projects
3 years ago
Maria Khrustaleva c9f214a894
Unify error handling with the cloud storage (#5389)
Added a general solution for checking bucket and file status instead of
checking in all places. Exception information has become more
user-friendly.
3 years ago
Roman Donchenko 487c60ce2b
SDK: Add an adapter layer that presents a CVAT task as a torchvision dataset (#5417) 3 years ago
Maxim Zhiltsov 82adde42aa
Refactor resource import export tests (#5429)
Extracted some enhancements from
https://github.com/opencv/cvat/pull/4819

- Extracted common s3 manipulations in tests
- Refactored import/export tests to be more clear
3 years ago
Roman Donchenko 0a032b3236
Migrate to using pathlib in most of the SDK (#5435)
For user-facing functions, keep accepting `str` paths to maintain
compatibility and flexibility, but add support for arbitrary path-like
objects. For internal functions (in `downloading.py` and
`uploading.py`), don't bother and require `pathlib.Path`.

The only code that isn't converted is build-time code (e.g. `setup.py`)
and code that came from openapi-generator.
3 years ago
Roman Donchenko 481630e719
Migrate tests/python to from os.path to pathlib (#5426)
`pathlib` improves code readability and type safety. It is already used
in some of the tests; convert all remaining `os.path` usage to `pathlib`
equivalents.
3 years ago
Anastasia Yasakova 3f9ab7cf68
Fix loss of rotation in CVAT format (#5407)
Fix https://github.com/opencv/cvat/issues/4378
3 years ago
Roman Donchenko 192fd72680
Fix creation of tasks with Git repositories via the SDK (#5409)
Fixes #4365
3 years ago
Maria Khrustaleva 9b0d963d1a
Fix REST API tests (#5416)
Fixed REST API tests after merging #5408 and #5396 to develop

Co-authored-by: Boris Sekachev <sekachev.bs@gmail.com>
Co-authored-by: Nikita Manovich <nikita@cvat.ai>
3 years ago
Maria Khrustaleva f405c4acac
Fix missing source tag in project annotations (#5408) 3 years ago
Nikita Manovich 00228ab7bb
Fix several issues with testing system (#5394)
Fix https://github.com/opencv/cvat/issues/5214

1. Stable names for containers (_ vs -)
2. Improve documentation
3 years ago
Kirill Sizov 2ecd8c7b0c
Test webhooks sender: wait for delivery response (#5365) 3 years ago
Roman Donchenko 0a16cfce5b
SDK: add a high-level method to download task data chunks (#5356) 3 years ago
dependabot[bot] 0786d05f6b
Bump pillow from 9.0.1 to 9.3.0 in /tests/python (#5341) 3 years ago
Maria Khrustaleva 08dd27d993
Fix missed token with using social account authentication (#5344) 3 years ago
Kirill Sizov 25503a35d2
Fix export job (#5282) 3 years ago
Roman Donchenko aadfd8814e
Fix TUS offset queries in production environments (#5204)
* Fix TUS offset queries in production environments

Previously, `mod_wsgi` would convert `HEAD` requests into `GET`, which
would be rejected, so clients were unable to resume an upload that failed
midway through.

To make use of this, update the SDK code to enable upload resumption.

* Add a test that forces a chunked TUS upload

* test_can_create_from_backup_in_chunks: make sure the upload is actually chunked
3 years ago
magician 0b7fb04a09
fix pascal_voc import bug (#4647)
* fix  pascal_voc import bug

* Add tests

* Fix black

* Small fixes

* Fix test

* Fix black

* Update Changelog

* Fix Changelog

* Fixes

* Remove unnecessary changes

* Remove unnecessary changes

* Join tests

* Small fix

* Fixes

* Fix pylint

* Remove unused import

* Small fix

Co-authored-by: yasakova-anastasia <yasakova_anastasiya@mail.ru>
3 years ago
Maria Khrustaleva ba74709c40
Add tests for export job dataset (#5160) 3 years ago
Kirill Sizov db6bbbab6a
Fix dataset import for Datumaro format (#4544)
* Datumaro format: add load_data_callback

* add test

* fix test

* fix project dataset uploading for some formats

* Fix black

* Update Changelog

* Update README.md

* Update README.md

Co-authored-by: Maxim Zhiltsov <zhiltsov.max35@gmail.com>

* Fixes

* Remove useless trailing backslashes

* Fix tests

* Fix test

* Join tests

* Small fix

* Fix remark

Co-authored-by: kirill.sizov <kirill.sizov@intel.com>
Co-authored-by: yasakova-anastasia <yasakova_anastasiya@mail.ru>
Co-authored-by: Maxim Zhiltsov <zhiltsov.max35@gmail.com>
3 years ago
Boris Sekachev 43e22c2cce
Updated rego rules, job assignee can resolve an issue (#5167) 3 years ago
Anastasia Yasakova 2311b10b80
Fix an issue with Imagenet dataset import (#4861) 3 years ago
Kirill Sizov ce09f28512
Fix SDK tests: login once per class (#5123) 3 years ago
Andrey Zhavoronkov 570bc3880f
Added user_registered signal (#5007) 3 years ago
Aleksey Alekseev c3dd3499e3
Added Cypress testing for feature: Multiple tasks creating from videos (#5028)
* add tests for try of creating multitasks with images and video

* add tests seccess creating video with Remote source

* reduce time of debounce

* try fix 107, 118 tests

* try fix 107 tests

* fix test with attach video

* disable test with attach video

* fix url to remote videos

* Aborted extra changes

* Updated files

* Refactored case 107

* Fixed almost all tests

* Aborted extra changes

* removed extra files

* Improved tests, added tests for local videos

* Fixed corner case

* Fix ://

* Redesigned a way of getting a file type

* Added html type to button

* Some refactoring

* Removed extra file

* sdk tests: fix path for file share images

Co-authored-by: Boris <sekachev.bs@gmail.com>
Co-authored-by: kirill-sizov <sizow.k.d@gmail.com>
3 years ago
Kirill Sizov dbcfb30cb2
Fix SDK tests (#5077)
* Update SDK tests to use LoginSerializerEx
3 years ago
Maxim Zhiltsov 511f9706d5
Allow trailing slashes in SDK client url (#5057) 3 years ago
Роман Донченко b029bc9da8
cvat-sdk: Fix creating tasks with non-local files (#5058)
* Move the test file share contents to tests/share

This way, it can be reused between tests.

* cvat-sdk: Fix creating tasks with non-local files

Forcing the `Content-Type` for the `upload_data` API call to
`multipart/form-data` does not work, because the current logic for
converting Python values to parts (`ApiClient._convert_body_to_post_params`)
does not encode them in a way that Django REST Framework can understand (it
JSON-encodes each part).

Fortunately, we don't actually need to do that, since when we create a task
with non-local files, we don't need to upload any files, and so we can just
post the original JSON, so do just that.

I couldn't add a test for the remote image case, because CVAT rejects all URLs
with non-public IP addressses. However, I did test this case manually.
3 years ago
Kirill Sizov bcb3f9bda8
Tests for Webhook sender (#5017)
* tests/rest_api/assets: update webhooks in testdb

* tests/rest_api/test_webhook_sender: add tests for project and ping events

* tests/rest_api/test_webhook_sender: add tests for webhook intersection case

* tests/rest_api/test_webhook_sender: add tests for task update events

* tests/rest_api/test_webhook_sender: add tests for task create and job update events

* tests/rest_api/test_webhook_sender: add tests for issue events

* tests/rest_api/test_webhook_sender: add tests for membership events

* tests/rest_api/test_webhook_sender: add tests for organization event

* Fix Pylint warnings

* apps/engine: get rid of sending `create` signals in views

* apps/organizations: get rid of sending `create` signals in views

* tests/rest_api/test_webhooks: fix typo

* apps/engine: get rid of sending  signals in task view

* tests/rest_api: remove debug prints

* apps/engine: fix pylint errors

* tests/rest_api/test_webhooks_sender: add tests for `redelivery` method

* tests: define tag for minio image

* apps/webhooks: remove owner_id from write serializer

* tests/rest_api/test_webhooks: fix code style

* tests/rest_api: added tests for webhook comment events

* tests/rest_api: fix typo

Co-authored-by: Anastasia Yasakova <yasakova_anastasiya@mail.ru>

* tests: fix warnings from black

Co-authored-by: Anastasia Yasakova <yasakova_anastasiya@mail.ru>
3 years ago
Роман Донченко 963e8efadb
Remove `cvat-sdk` from `tests/python/requirements.txt` (#5019) 3 years ago
Maxim Zhiltsov 6654366021
Autoformat python tests (#5021) 3 years ago
Maxim Zhiltsov 2e15025434
Check server version in SDK (#4935) 3 years ago
Kirill Sizov bae7564968
Add webhooks (#4863)
Co-authored-by: “klakhov” <kirill.lakhov@cvat.ai>
Co-authored-by: Boris <sekachev.bs@gmail.com>
Co-authored-by: kirill-sizov <kirill.sizov@intel.com>
3 years ago
Anastasia Yasakova ddcee5b3a9
REST API tests with skeletons (#4987) 3 years ago
Anastasia Yasakova ba308336c6
Fix project export with skeletons (#5004) 3 years ago
Andrey Zhavoronkov e22eac7d9b
Helm tests (#4949) 3 years ago
Maria Khrustaleva 56e8c1eb82
Cypress tests for source & target storage (#4914)
* Draft version

* Implemented import & fixed export && some code cleanup && some fixes &&  updated notifications

* Refactoring && fixed several bugs

* Update licence headers && small fixes

* Update remaining licence headers && small changes

* Fix part of tests

* Fix tests

* Remove unused code

* Fix part of comments

* Some fixes

* Move file download process to job

* Rename methods

* Small fix

* Fix storages configuration for tasks

* Styles

* Remove unused

* Change storage configuration && fix forms reset && some fixes

* Update imports

* Remove extra argument type

* Add catch

* Fix import backup from local

* Import architecture refactoring && some fixes

* Rename props

* Small reorganization of export architecture &&  minor fixes

* Remove unused import

* Small fix && skip error notification when no permissions

* Fix project creating

* Fix part of eslint issues

* Fix eslint

* Fix eslint

* Fix eslint

* eslint

* Fix some eslint issues

* Combine uploadAnnotations and importDataset

* Fix annotation uploading from local

* Update tests

* Fix annotation uploading

* Fix notification

* Update dependencies

* fix

* Update jest tests

* Skip error notification when no permissions

* Update case 91 92 canvas3d tests

* Styles

* Update icons

* eslint

* eslint

* eslint

* eslint

* Create & delete cloud storage

* Common part && export job annotations

* Update tests && add tests for project backup

* Fix typo

* Rename variables

* debug

* Revert timeout

* Fix server host

* Fix test with project backup restore

* small refactoring

* Update issue_2473_import_annotations_frames_dots_in_name
3 years ago
Maxim Zhiltsov 68375ec23e
Add SDK docs (#4928)
- Added auth docs in API schema and SDK ApiClient/Config (fixes #1517)
- Added SDK docs with API, SDK and CLI docs
- Added `develop` branch in the docs
- Allowed unauthorized access to `api/docs`, `api/swagger`, `api/schema` endpoints
- Added `--insecure` env var to control host checks in CLI
- Refactored `build_docs.py` (backported https://github.com/openvinotoolkit/datumaro/pull/589)
- Extracted requirements file for site
3 years ago
Maxim Zhiltsov 4f6e81eab3
Bump sdk and cli versions (#4923) 3 years ago
Maxim Zhiltsov 7218f4e283
Add host schema auto-detection in SDK (#4910) 3 years ago
Maxim Zhiltsov 1dcba5a843
Align server api, sdk and cli versions to 2.1.0 (#4913) 3 years ago