45 Commits (e24bacede2e93b8617aa2d7ae5c56626512d3e01)

Author SHA1 Message Date
Kirill Sizov 2ab66fc32c
Remove limit manager (#5642) 3 years ago
Maxim Zhiltsov 0280b5fb64
Add simple filters for collections (#5575)
Added simple filter queries for collections in the server API. Such
queries should cover simpler cases, while more complex filters should
use the JSON syntax.

- Added simple filters for collection endpoints in API (e.g.
`/tasks?project_id=42&user=myusername`)
- Removed duplicating collection endpoints in API in favor of their full
versions:
  - `/projects/{id}/tasks` -> `/tasks?project_id={id}`
  - `/tasks/{id}/jobs` -> `/jobs?task_id={id}`
  - `/jobs/{id}/issues` -> `/issues?job_id={id}`
  - `/issue/{id}/comments` -> `/comments?issue_id={id}`
  - Corresponding owning objects now return a link to the collection:
    - `/projects/{id}`: `.tasks`
    - `/tasks/{id}`: `.jobs`
    - `/jobs/{id}`: `.issues`
    - `/issue/{id}`: `.comments`
- Fixed errors in the generated server API schema:
- Input and output types in all the basic model methods (`Create`,
`Update`, `Retrieve`, `Delete`)
- Removed invalid separation for the project list operation response
type
- File structure for projects in the dataset cache is updated to store
the new project representation. Old caches will be invalidated.
- Added tests
3 years ago
Kirill Sizov 7b86ed814e
Use 'ParseError' instead of 'BadRequest' (#5625)
When we use `BadRequest` our responses doesn't contain error message, so
it's better to use exceptions from `rest_framework.exceptions`
3 years ago
Kirill Sizov 4e3d81ccae
Fix webhooks (#5611) 3 years ago
ddx-day 0f0913c138
Django social account sso (#5059)
Issue: https://github.com/opencv/cvat/issues/1217

Currently there are a few proposals for SSO authentication to bypass the
current user/password login on the UI. By using Django social accounts
it is also possible to use SSO on the API, retrieving the security token
by passing the code from the OAuth2 workflow. This is an example using
Amazon Cognito, but any other social account could also be added.

### Motivation and context
Currently CVAT has no functionality to log in with SSO. Other current
proposals bypass the current Django framework to add SSO in the UI only,
but still use username and password for the API. Using Django social
accounts integrates SSO with the API as well, allowing it to be used as
an alternative to the username and password, but can also be used
together with other SSO frameworks that are UI only.

### How has this been tested?
Unit tests for SSO manager in cvat-core and integration test with
cvat-sdk for /auth/cognito endpoint.

### Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply.
If an item isn't applicable by a reason then ~~explicitly
strikethrough~~ the whole
line. If you don't do that github will show an incorrect process for the
pull request.
If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [x] I submit my changes into the `develop` branch
- [ ] I have added a description of my changes into
[CHANGELOG](https://github.com/cvat-ai/cvat/blob/develop/CHANGELOG.md)
file
- [x] I have updated the [documentation](
https://github.com/cvat-ai/cvat/blob/develop/README.md#documentation)
accordingly
- [x] I have added tests to cover my changes
- [x] I have linked related issues ([read github docs](

https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))
- [ ] I have increased versions of npm packages if it is necessary
([cvat-canvas](https://github.com/cvat-ai/cvat/tree/develop/cvat-canvas#versioning),

[cvat-core](https://github.com/cvat-ai/cvat/tree/develop/cvat-core#versioning),
[cvat-data](https://github.com/cvat-ai/cvat/tree/develop/cvat-data#versioning)
and
[cvat-ui](https://github.com/cvat-ai/cvat/tree/develop/cvat-ui#versioning))

### License

- [x] I submit _my code changes_ under the same [MIT License](
https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the
project.
  Feel free to contact the maintainers if that's a concern.

Co-authored-by: Melanie Day <mday@EYEDIAGNOSIS.local>
Co-authored-by: Maria Khrustaleva <maria@cvat.ai>
Co-authored-by: Nikita Manovich <nikita@cvat.ai>
3 years ago
Maxim Zhiltsov 31f0578220
Add a way to create task with custom jobs (#5536)
This PR adds an option to specify file to job mapping explicitly during
task creation. This option is incompatible with most other job-related
parameters like `sorting_method` and `frame_step`.

- Added a new task creation parameter (`job_file_mapping`) to set a
custom file to job mapping during task creation
3 years ago
Kirill Sizov 71a0aaf2bb
Improve error messages when limits reached (#5556) 3 years ago
Roman Donchenko 3d9c5add94
SDK: fix string field serialization for multipart/form-data requests (#5479)
Django REST Framework ignores the Content-Type on request body parts, so
it doesn't know that they are JSON-encoded. Instead, it just tries to
decode each part as if it was an `str()`-encoded value.

Change the encoding to match the decoding. The only type this matters
for is `str`, because `json.dumps` and `str` produce different encodings
for `str` values.

Remove `none_type` from the list of encodable types since, to my
knowledge, there's no way to encode a `None` value as a
`multipart/form-data` part in a way that DRF will understand.
3 years ago
Maria Khrustaleva 47860c9d22
Remove previews downloading when task creating with cloud storage data (#5499)
PR removes previews downloading from CS when task creating (skipped in
PR #5478). In addition, I had to change the test to check for the file
name existing in the message when the specified file is not found in the
bucket, because now the preview is no longer downloaded at the stage of
creating a task.
3 years ago
Maxim Zhiltsov 470336e82c
Fix docker command (#5541)
Follow up for #5538 

The previous fix fixed the situation for local environments, but broke
the CI. This PR tries to use another way.
3 years ago
Maxim Zhiltsov d2a4830634
Fix docker command in tests (#5538)
This PR fixes a problem with local running of tests introduced in #5498.
When running tests locally, there can be multiple tags available, while
we need the `dev` tag, which should be created right before tests are
executed.

- Added the missing image tag in the docker run command
3 years ago
Andrey Zhavoronkov 52e3f3c28a
Added tests for preview endpoints (#5529) 3 years ago
Maxim Zhiltsov ec3e1f34a4
Better reporting for user limits (#5225)
- Added explanatory messages for actions denied for user limits
- Fixed few rules and checks
- Upgraded OPA version
3 years ago
Maria Khrustaleva e624c5b959
Simplify upload data for task (#5498)
It's possible to specify only the manifest file and filename pattern for
creating task with cloud storage data.
The special characters supported now for the pattern are `*`, `?`,
`[seq]`, `[!seq]`.
Please see
[here](8898a8b264/tests/python/rest_api/test_tasks.py (L686))
for some examples of how to use this functionality.

Co-authored-by: Maxim Zhiltsov <zhiltsov.max35@gmail.com>
3 years ago
Maxim Zhiltsov af65707eee
Mark tests that require external services (#5474)
Related: #5225 

External services are not available when we execute in Helm.

- Added a way to mark REST API tests that require external services to
run
- Changed the way of filtering tests in Helm tests

Currently, we can't execute external service mocks in tests, and we
ignore related tests in the Helm
execution command. But this command may be outdated, because Helm tests
are not executed in each PR.
This PR allows to indicate related tests and filter them out without the
need to synchronize the CI command.
3 years ago
Maria Khrustaleva c9f214a894
Unify error handling with the cloud storage (#5389)
Added a general solution for checking bucket and file status instead of
checking in all places. Exception information has become more
user-friendly.
3 years ago
Maxim Zhiltsov 82adde42aa
Refactor resource import export tests (#5429)
Extracted some enhancements from
https://github.com/opencv/cvat/pull/4819

- Extracted common s3 manipulations in tests
- Refactored import/export tests to be more clear
3 years ago
Roman Donchenko 481630e719
Migrate tests/python to from os.path to pathlib (#5426)
`pathlib` improves code readability and type safety. It is already used
in some of the tests; convert all remaining `os.path` usage to `pathlib`
equivalents.
3 years ago
Anastasia Yasakova 3f9ab7cf68
Fix loss of rotation in CVAT format (#5407)
Fix https://github.com/opencv/cvat/issues/4378
3 years ago
Maria Khrustaleva 9b0d963d1a
Fix REST API tests (#5416)
Fixed REST API tests after merging #5408 and #5396 to develop

Co-authored-by: Boris Sekachev <sekachev.bs@gmail.com>
Co-authored-by: Nikita Manovich <nikita@cvat.ai>
3 years ago
Maria Khrustaleva f405c4acac
Fix missing source tag in project annotations (#5408) 3 years ago
Nikita Manovich 00228ab7bb
Fix several issues with testing system (#5394)
Fix https://github.com/opencv/cvat/issues/5214

1. Stable names for containers (_ vs -)
2. Improve documentation
3 years ago
Kirill Sizov 2ecd8c7b0c
Test webhooks sender: wait for delivery response (#5365) 3 years ago
Maria Khrustaleva 08dd27d993
Fix missed token with using social account authentication (#5344) 3 years ago
Kirill Sizov 25503a35d2
Fix export job (#5282) 3 years ago
magician 0b7fb04a09
fix pascal_voc import bug (#4647)
* fix  pascal_voc import bug

* Add tests

* Fix black

* Small fixes

* Fix test

* Fix black

* Update Changelog

* Fix Changelog

* Fixes

* Remove unnecessary changes

* Remove unnecessary changes

* Join tests

* Small fix

* Fixes

* Fix pylint

* Remove unused import

* Small fix

Co-authored-by: yasakova-anastasia <yasakova_anastasiya@mail.ru>
3 years ago
Maria Khrustaleva ba74709c40
Add tests for export job dataset (#5160) 3 years ago
Kirill Sizov db6bbbab6a
Fix dataset import for Datumaro format (#4544)
* Datumaro format: add load_data_callback

* add test

* fix test

* fix project dataset uploading for some formats

* Fix black

* Update Changelog

* Update README.md

* Update README.md

Co-authored-by: Maxim Zhiltsov <zhiltsov.max35@gmail.com>

* Fixes

* Remove useless trailing backslashes

* Fix tests

* Fix test

* Join tests

* Small fix

* Fix remark

Co-authored-by: kirill.sizov <kirill.sizov@intel.com>
Co-authored-by: yasakova-anastasia <yasakova_anastasiya@mail.ru>
Co-authored-by: Maxim Zhiltsov <zhiltsov.max35@gmail.com>
3 years ago
Boris Sekachev 43e22c2cce
Updated rego rules, job assignee can resolve an issue (#5167) 3 years ago
Anastasia Yasakova 2311b10b80
Fix an issue with Imagenet dataset import (#4861) 3 years ago
Kirill Sizov ce09f28512
Fix SDK tests: login once per class (#5123) 3 years ago
Andrey Zhavoronkov 570bc3880f
Added user_registered signal (#5007) 3 years ago
Kirill Sizov dbcfb30cb2
Fix SDK tests (#5077)
* Update SDK tests to use LoginSerializerEx
3 years ago
Kirill Sizov bcb3f9bda8
Tests for Webhook sender (#5017)
* tests/rest_api/assets: update webhooks in testdb

* tests/rest_api/test_webhook_sender: add tests for project and ping events

* tests/rest_api/test_webhook_sender: add tests for webhook intersection case

* tests/rest_api/test_webhook_sender: add tests for task update events

* tests/rest_api/test_webhook_sender: add tests for task create and job update events

* tests/rest_api/test_webhook_sender: add tests for issue events

* tests/rest_api/test_webhook_sender: add tests for membership events

* tests/rest_api/test_webhook_sender: add tests for organization event

* Fix Pylint warnings

* apps/engine: get rid of sending `create` signals in views

* apps/organizations: get rid of sending `create` signals in views

* tests/rest_api/test_webhooks: fix typo

* apps/engine: get rid of sending  signals in task view

* tests/rest_api: remove debug prints

* apps/engine: fix pylint errors

* tests/rest_api/test_webhooks_sender: add tests for `redelivery` method

* tests: define tag for minio image

* apps/webhooks: remove owner_id from write serializer

* tests/rest_api/test_webhooks: fix code style

* tests/rest_api: added tests for webhook comment events

* tests/rest_api: fix typo

Co-authored-by: Anastasia Yasakova <yasakova_anastasiya@mail.ru>

* tests: fix warnings from black

Co-authored-by: Anastasia Yasakova <yasakova_anastasiya@mail.ru>
3 years ago
Maxim Zhiltsov 6654366021
Autoformat python tests (#5021) 3 years ago
Maxim Zhiltsov 2e15025434
Check server version in SDK (#4935) 3 years ago
Kirill Sizov bae7564968
Add webhooks (#4863)
Co-authored-by: “klakhov” <kirill.lakhov@cvat.ai>
Co-authored-by: Boris <sekachev.bs@gmail.com>
Co-authored-by: kirill-sizov <kirill.sizov@intel.com>
3 years ago
Anastasia Yasakova ddcee5b3a9
REST API tests with skeletons (#4987) 3 years ago
Anastasia Yasakova ba308336c6
Fix project export with skeletons (#5004) 3 years ago
Andrey Zhavoronkov e22eac7d9b
Helm tests (#4949) 3 years ago
Maxim Zhiltsov 68375ec23e
Add SDK docs (#4928)
- Added auth docs in API schema and SDK ApiClient/Config (fixes #1517)
- Added SDK docs with API, SDK and CLI docs
- Added `develop` branch in the docs
- Allowed unauthorized access to `api/docs`, `api/swagger`, `api/schema` endpoints
- Added `--insecure` env var to control host checks in CLI
- Refactored `build_docs.py` (backported https://github.com/openvinotoolkit/datumaro/pull/589)
- Extracted requirements file for site
3 years ago
Maxim Zhiltsov 53697ecac5
SDK layer 2 - cover RC1 usecases (#4813) 4 years ago
Kirill Sizov b60d3b481a
fix remote image link (#4871) 4 years ago
Maria Khrustaleva 182c39aa8f
Hotfix missing manifest file (#116) 4 years ago
Maxim Zhiltsov 5f58a0f7be
Add 2nd layer of SDK (#19) 4 years ago