Added simple filter queries for collections in the server API. Such
queries should cover simpler cases, while more complex filters should
use the JSON syntax.
- Added simple filters for collection endpoints in API (e.g.
`/tasks?project_id=42&user=myusername`)
- Removed duplicating collection endpoints in API in favor of their full
versions:
- `/projects/{id}/tasks` -> `/tasks?project_id={id}`
- `/tasks/{id}/jobs` -> `/jobs?task_id={id}`
- `/jobs/{id}/issues` -> `/issues?job_id={id}`
- `/issue/{id}/comments` -> `/comments?issue_id={id}`
- Corresponding owning objects now return a link to the collection:
- `/projects/{id}`: `.tasks`
- `/tasks/{id}`: `.jobs`
- `/jobs/{id}`: `.issues`
- `/issue/{id}`: `.comments`
- Fixed errors in the generated server API schema:
- Input and output types in all the basic model methods (`Create`,
`Update`, `Retrieve`, `Delete`)
- Removed invalid separation for the project list operation response
type
- File structure for projects in the dataset cache is updated to store
the new project representation. Old caches will be invalidated.
- Added tests
After this, there's only 12 files with ESLint errors left. I didn't fix
those because they all have at least one
`cypress/no-unnecessary-waiting` error, and I don't have enough Cypress
expertise to fix those.
Issue: https://github.com/opencv/cvat/issues/1217
Currently there are a few proposals for SSO authentication to bypass the
current user/password login on the UI. By using Django social accounts
it is also possible to use SSO on the API, retrieving the security token
by passing the code from the OAuth2 workflow. This is an example using
Amazon Cognito, but any other social account could also be added.
### Motivation and context
Currently CVAT has no functionality to log in with SSO. Other current
proposals bypass the current Django framework to add SSO in the UI only,
but still use username and password for the API. Using Django social
accounts integrates SSO with the API as well, allowing it to be used as
an alternative to the username and password, but can also be used
together with other SSO frameworks that are UI only.
### How has this been tested?
Unit tests for SSO manager in cvat-core and integration test with
cvat-sdk for /auth/cognito endpoint.
### Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply.
If an item isn't applicable by a reason then ~~explicitly
strikethrough~~ the whole
line. If you don't do that github will show an incorrect process for the
pull request.
If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [x] I submit my changes into the `develop` branch
- [ ] I have added a description of my changes into
[CHANGELOG](https://github.com/cvat-ai/cvat/blob/develop/CHANGELOG.md)
file
- [x] I have updated the [documentation](
https://github.com/cvat-ai/cvat/blob/develop/README.md#documentation)
accordingly
- [x] I have added tests to cover my changes
- [x] I have linked related issues ([read github docs](
https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))
- [ ] I have increased versions of npm packages if it is necessary
([cvat-canvas](https://github.com/cvat-ai/cvat/tree/develop/cvat-canvas#versioning),
[cvat-core](https://github.com/cvat-ai/cvat/tree/develop/cvat-core#versioning),
[cvat-data](https://github.com/cvat-ai/cvat/tree/develop/cvat-data#versioning)
and
[cvat-ui](https://github.com/cvat-ai/cvat/tree/develop/cvat-ui#versioning))
### License
- [x] I submit _my code changes_ under the same [MIT License](
https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the
project.
Feel free to contact the maintainers if that's a concern.
Co-authored-by: Melanie Day <mday@EYEDIAGNOSIS.local>
Co-authored-by: Maria Khrustaleva <maria@cvat.ai>
Co-authored-by: Nikita Manovich <nikita@cvat.ai>
This PR adds an option to specify file to job mapping explicitly during
task creation. This option is incompatible with most other job-related
parameters like `sorting_method` and `frame_step`.
- Added a new task creation parameter (`job_file_mapping`) to set a
custom file to job mapping during task creation
Django REST Framework ignores the Content-Type on request body parts, so
it doesn't know that they are JSON-encoded. Instead, it just tries to
decode each part as if it was an `str()`-encoded value.
Change the encoding to match the decoding. The only type this matters
for is `str`, because `json.dumps` and `str` produce different encodings
for `str` values.
Remove `none_type` from the list of encodable types since, to my
knowledge, there's no way to encode a `None` value as a
`multipart/form-data` part in a way that DRF will understand.
PR removes previews downloading from CS when task creating (skipped in
PR #5478). In addition, I had to change the test to check for the file
name existing in the message when the specified file is not found in the
bucket, because now the preview is no longer downloaded at the stage of
creating a task.
This will let users to run their PyTorch code without network access,
provided that they have already cached the data.
### How has this been tested?
<!-- Please describe in detail how you tested your changes.
Include details of your testing environment, and the tests you ran to
see how your change affects other areas of the code, etc. -->
Unit tests.
Turns out that #5456 had a nasty side effect. Session hooks are called
when pytest is run with `--collect-only` (even though no tests are
actually run in this case), and Visual Studio Code periodically runs
`pytest --collect-only` in order to learn what tests exist in the
project. As a result, it keeps restarting the services and restoring the
database in the background.
Work around this by skipping all logic in the hooks if `--collect-only`
is in the options.
Extracted from https://github.com/opencv/cvat/pull/5083
- Added a default arg for task data uploading
- Added an option to wait for the data processing in task data uploading
- Moved data splitting by requests for TUS closer to the point of use
This PR fixes a problem with local running of tests introduced in #5498.
When running tests locally, there can be multiple tags available, while
we need the `dev` tag, which should be created right before tests are
executed.
- Added the missing image tag in the docker run command
Bumps [json5](https://github.com/json5/json5) from 2.2.1 to 2.2.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/json5/json5/releases">json5's
releases</a>.</em></p>
<blockquote>
<h2>v2.2.2</h2>
<ul>
<li>Fix: Properties with the name <code>__proto__</code> are added to
objects and arrays.
(<a
href="https://github-redirect.dependabot.com/json5/json5/issues/199">#199</a>)
This also fixes a prototype pollution vulnerability reported by
Jonathan Gregson! (<a
href="https://github-redirect.dependabot.com/json5/json5/issues/295">#295</a>).</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/json5/json5/blob/main/CHANGELOG.md">json5's
changelog</a>.</em></p>
<blockquote>
<h3>v2.2.2 [<a
href="https://github.com/json5/json5/tree/v2.2.2">code</a>, <a
href="https://github.com/json5/json5/compare/v2.2.1...v2.2.2">diff</a>]</h3>
<ul>
<li>Fix: Properties with the name <code>__proto__</code> are added to
objects and arrays.
(<a
href="https://github-redirect.dependabot.com/json5/json5/issues/199">#199</a>)
This also fixes a prototype pollution vulnerability reported by
Jonathan Gregson! (<a
href="https://github-redirect.dependabot.com/json5/json5/issues/295">#295</a>).</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="14f8cb186e"><code>14f8cb1</code></a>
2.2.2</li>
<li><a
href="10cc7ca916"><code>10cc7ca</code></a>
docs: update CHANGELOG for v2.2.2</li>
<li><a
href="7774c10979"><code>7774c10</code></a>
fix: add <strong>proto</strong> to objects and arrays</li>
<li><a
href="edde30abd8"><code>edde30a</code></a>
Readme: slight tweak to intro</li>
<li><a
href="97286f8bd5"><code>97286f8</code></a>
Improve example in readme</li>
<li><a
href="d720b4fe4a"><code>d720b4f</code></a>
Improve readme (e.g. explain JSON5 better!) (<a
href="https://github-redirect.dependabot.com/json5/json5/issues/291">#291</a>)</li>
<li><a
href="910ce25914"><code>910ce25</code></a>
docs: fix spelling of Aseem</li>
<li><a
href="2aab4dd2a7"><code>2aab4dd</code></a>
test: require tap as t in cli tests</li>
<li><a
href="6d426865ce"><code>6d42686</code></a>
test: remove mocha syntax from tests</li>
<li><a
href="4798b9dbde"><code>4798b9d</code></a>
docs: update installation and usage for modules</li>
<li>Additional commits viewable in <a
href="https://github.com/json5/json5/compare/v2.2.1...v2.2.2">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the
[Security Alerts page](https://github.com/opencv/cvat/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
It's possible to specify only the manifest file and filename pattern for
creating task with cloud storage data.
The special characters supported now for the pattern are `*`, `?`,
`[seq]`, `[!seq]`.
Please see
[here](8898a8b264/tests/python/rest_api/test_tasks.py (L686))
for some examples of how to use this functionality.
Co-authored-by: Maxim Zhiltsov <zhiltsov.max35@gmail.com>
Related: #5225
External services are not available when we execute in Helm.
- Added a way to mark REST API tests that require external services to
run
- Changed the way of filtering tests in Helm tests
Currently, we can't execute external service mocks in tests, and we
ignore related tests in the Helm
execution command. But this command may be outdated, because Helm tests
are not executed in each PR.
This PR allows to indicate related tests and filter them out without the
need to synchronize the CI command.
OPA can take some time to load rules, but our tests don't wait for OPA,
and start right after the server is loaded.
Sometimes it works, but in other times the tests may fail because OPA is
still loading the rules.
This PR allows to wait for OPA during the test suite startup.
This should protect us from unnoticed hangs, which happen sometimes and
waste CI and developer time.
Currently, the limit is 15s per test, which should be enough in normal
conditions.
Currently, `server.py` runs as PID 1, which means that it won't be
terminated by a `SIGTERM` signal unless it explicitly handles it (which
it doesn't). So when Docker tries to shut the container down, it sends
the server a `SIGTERM`, which gets ignored, and then sits there for 10
seconds before sending it a `SIGKILL`.
To work around this, enable the built-in Docker init program, which
forwards signals to the Python server. Since the Python server is no
longer PID 1, `SIGTERM` will now shut it down immediately.
The `init` option is supported starting from the Compose format version
3.7, so bump the version.
Extracted some enhancements from
https://github.com/opencv/cvat/pull/4819
- Extracted common s3 manipulations in tests
- Refactored import/export tests to be more clear
For user-facing functions, keep accepting `str` paths to maintain
compatibility and flexibility, but add support for arbitrary path-like
objects. For internal functions (in `downloading.py` and
`uploading.py`), don't bother and require `pathlib.Path`.
The only code that isn't converted is build-time code (e.g. `setup.py`)
and code that came from openapi-generator.
`pathlib` improves code readability and type safety. It is already used
in some of the tests; convert all remaining `os.path` usage to `pathlib`
equivalents.
Fixed REST API tests after merging #5408 and #5396 to develop
Co-authored-by: Boris Sekachev <sekachev.bs@gmail.com>
Co-authored-by: Nikita Manovich <nikita@cvat.ai>
Kirill Lakhov
Boris Sekachev
Kirill Sizov
Maxim Zhiltsov
Roman Donchenko
ddx-day
Maria Khrustaleva
Andrey Zhavoronkov
dependabot[bot]
Anastasia Yasakova
