135 Commits (912e47e56c772eb6c2fb5b32f898b029a985fdfc)

Author SHA1 Message Date
Kirill Sizov b263d871f5
[Snyk] Security upgrade django from 3.2.16 to 3.2.17 (#5665)
<p>This PR was automatically created by Snyk using the credentials of a
real user.</p><br /><h3>Snyk has created this PR to fix one or more
vulnerable packages in the `pip` dependencies of this project.</h3>



#### Changes included in this PR

- Changes to the following files to upgrade the vulnerable dependencies
to a fixed version:
    - cvat/requirements/base.txt


<details>
<summary>⚠️ <b>Warning</b></summary>

```
Jinja2 3.1.2 requires MarkupSafe, which is not installed.
h5py 3.6.0 requires numpy, which is not installed.
azure-storage-blob 12.13.0 requires cryptography, which is not installed.

```
</details>


#### Vulnerabilities that will be fixed





##### By pinning:
Severity | Priority Score (*) | Issue | Upgrade | Breaking Change |
Exploit Maturity

:-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:-------------------------
![medium
severity](https://res.cloudinary.com/snyk/image/upload/w_20,h_20/v1561977819/icon/m.png
"medium severity") | **551/1000** <br/> **Why?** Recently disclosed, Has
a fix available, CVSS 5.3 | Denial of Service (DoS)
<br/>[SNYK-PYTHON-DJANGO-3266406](https://snyk.io/vuln/SNYK-PYTHON-DJANGO-3266406)
| `django:` <br> `3.2.16 -> 3.2.17` <br> | No | No Known Exploit

(*) Note that the real score may have changed since the PR was raised.




Some vulnerabilities couldn't be fully fixed and so Snyk will still find
them when the project is tested again. This may be because the
vulnerability existed within more than one direct dependency, but not
all of the affected dependencies could be upgraded.


Check the changes in this PR to ensure they won't cause issues with your
project.



------------



**Note:** *You are seeing this because you or someone else with access
to this repository has authorized Snyk to open fix PRs.*

For more information: <img
src="https://api.segment.io/v1/pixel/track?data=eyJ3cml0ZUtleSI6InJyWmxZcEdHY2RyTHZsb0lYd0dUcVg4WkFRTnNCOUEwIiwiYW5vbnltb3VzSWQiOiJiZTIyZjVmOS03Y2IzLTRiYmEtODdjZS0xOGY1ZDIzZmFkNmUiLCJldmVudCI6IlBSIHZpZXdlZCIsInByb3BlcnRpZXMiOnsicHJJZCI6ImJlMjJmNWY5LTdjYjMtNGJiYS04N2NlLTE4ZjVkMjNmYWQ2ZSJ9fQ=="
width="0" height="0"/>
🧐 [View latest project
report](https://app.snyk.io/org/cvat/project/4bbc4b80-3fb9-4009-a7bb-51016d44946b?utm_source&#x3D;github&amp;utm_medium&#x3D;referral&amp;page&#x3D;fix-pr)

🛠 [Adjust project
settings](https://app.snyk.io/org/cvat/project/4bbc4b80-3fb9-4009-a7bb-51016d44946b?utm_source&#x3D;github&amp;utm_medium&#x3D;referral&amp;page&#x3D;fix-pr/settings)

📚 [Read more about Snyk's upgrade and patch
logic](https://support.snyk.io/hc/en-us/articles/360003891078-Snyk-patches-to-fix-vulnerabilities)

[//]: #
(snyk:metadata:{"prId":"be22f5f9-7cb3-4bba-87ce-18f5d23fad6e","prPublicId":"be22f5f9-7cb3-4bba-87ce-18f5d23fad6e","dependencies":[{"name":"django","from":"3.2.16","to":"3.2.17"}],"packageManager":"pip","projectPublicId":"4bbc4b80-3fb9-4009-a7bb-51016d44946b","projectUrl":"https://app.snyk.io/org/cvat/project/4bbc4b80-3fb9-4009-a7bb-51016d44946b?utm_source=github&utm_medium=referral&page=fix-pr","type":"auto","patch":[],"vulns":["SNYK-PYTHON-DJANGO-3266406"],"upgrade":[],"isBreakingChange":false,"env":"prod","prType":"fix","templateVariants":["updated-fix-title","pr-warning-shown","priorityScore"],"priorityScoreList":[551]})

---

**Learn how to fix vulnerabilities with free interactive lessons:**

🦉 [Denial of Service
(DoS)](https://learn.snyk.io/lessons/redos/javascript/?loc&#x3D;fix-pr)

---------

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
Co-authored-by: Nikita Manovich <nikita@cvat.ai>
3 years ago
Kirill Sizov 4a45321788
Fix Helm tests (#5579) 3 years ago
Roman Donchenko d5f3f93e81
Allow the documentation website to be indexed by search engines (#5623)
Our HTML documentation template slaps a `<META NAME="ROBOTS"
CONTENT="NOINDEX, NOFOLLOW">` on every page unless the `HUGO_ENV`
environment variable is set to `production`, thereby excluding the
entire documentation website from web search results. Set the variable
to fix it.

This behavior is inherited from Docsy, although they have changed it
since: <https://github.com/google/docsy/pull/653>.
3 years ago
Roman Donchenko 64fc53e3d8
Update issue and pull request templates (#5620)
* Move the reproduction recipe above the expected & actual behavior.
Otherwise it's a bit awkward to fill out the template, because you have
to describe what happens before you describe how to get there.

* Fix references to the old `cvat-ai/cvat` GitHub project.

* Update/remove broken links.

* Fix wording.

<!-- Raised an issue to propose your change
(https://github.com/cvat-ai/cvat/issues).
It helps to avoid duplication of efforts from multiple independent
contributors.
Discuss your ideas with maintainers to be sure that changes will be
approved and merged.
Read the
[CONTRIBUTION](https://github.com/cvat-ai/cvat/blob/develop/CONTRIBUTING.md)
guide. -->

<!-- Provide a general summary of your changes in the Title above -->

### Motivation and context
<!-- Why is this change required? What problem does it solve? If it
fixes an open
issue, please link to the issue here. Describe your changes in detail,
add
screenshots. -->
Just fixing some things that keep annoying me when I create issues/PRs.

### How has this been tested?
<!-- Please describe in detail how you tested your changes.
Include details of your testing environment, and the tests you ran to
see how your change affects other areas of the code, etc. -->

### Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply.
If an item isn't applicable by a reason then ~~explicitly
strikethrough~~ the whole
line. If you don't do that github will show an incorrect process for the
pull request.
If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [x] I submit my changes into the `develop` branch
- ~~[ ] I have added a description of my changes into
[CHANGELOG](https://github.com/cvat-ai/cvat/blob/develop/CHANGELOG.md)
file~~
- ~~[ ] I have updated the [documentation](
https://github.com/cvat-ai/cvat/blob/develop/README.md#documentation)
accordingly~~
- ~~[ ] I have added tests to cover my changes~~
- ~~[ ] I have linked related issues ([read github docs](

https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))~~
- ~~[ ] I have increased versions of npm packages if it is necessary
([cvat-canvas](https://github.com/cvat-ai/cvat/tree/develop/cvat-canvas#versioning),

[cvat-core](https://github.com/cvat-ai/cvat/tree/develop/cvat-core#versioning),
[cvat-data](https://github.com/cvat-ai/cvat/tree/develop/cvat-data#versioning)
and
[cvat-ui](https://github.com/cvat-ai/cvat/tree/develop/cvat-ui#versioning))~~

### License

- [x] I submit _my code changes_ under the same [MIT License](
https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the
project.
  Feel free to contact the maintainers if that's a concern.
3 years ago
Roman Donchenko 14bb0b74af
Fix a CI issue that causes unit tests to run against the wrong version of CVAT (#5612)
There seems to be a bug somewhere in the Docker ecosystem (it's probably
either Docker Compose, Docker Buildx or BuildKit) that causes `docker
compose build` to ignore base images that are already present in the
system, and instead fetch them from Docker Hub, if there's a custom
Buildx builder configured. There's a bug report here:
<https://github.com/docker/compose/issues/9939>.

This bug means that when the build pipeline builds the `cvat_ci` image,
it's based on the latest release of `cvat/server` from Docker Hub
instead of the version that we just built. Consequently, we run the unit
tests against that release instead of the development version.

Fortunately, we don't actually need to set up a Buildx builder in most
jobs (including the `unit_testing` job), so just don't do that.

Also, use `cvat/server:local` as the base image in `Dockerfile.ci`. This
will prevent a similar bug from reoccurring in the future, since the
`local` tag should never be uploaded to Docker Hub.
3 years ago
Roman Donchenko 330f123785
Fix some typos (#5601) 3 years ago
ddx-day 0f0913c138
Django social account sso (#5059)
Issue: https://github.com/opencv/cvat/issues/1217

Currently there are a few proposals for SSO authentication to bypass the
current user/password login on the UI. By using Django social accounts
it is also possible to use SSO on the API, retrieving the security token
by passing the code from the OAuth2 workflow. This is an example using
Amazon Cognito, but any other social account could also be added.

### Motivation and context
Currently CVAT has no functionality to log in with SSO. Other current
proposals bypass the current Django framework to add SSO in the UI only,
but still use username and password for the API. Using Django social
accounts integrates SSO with the API as well, allowing it to be used as
an alternative to the username and password, but can also be used
together with other SSO frameworks that are UI only.

### How has this been tested?
Unit tests for SSO manager in cvat-core and integration test with
cvat-sdk for /auth/cognito endpoint.

### Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply.
If an item isn't applicable by a reason then ~~explicitly
strikethrough~~ the whole
line. If you don't do that github will show an incorrect process for the
pull request.
If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [x] I submit my changes into the `develop` branch
- [ ] I have added a description of my changes into
[CHANGELOG](https://github.com/cvat-ai/cvat/blob/develop/CHANGELOG.md)
file
- [x] I have updated the [documentation](
https://github.com/cvat-ai/cvat/blob/develop/README.md#documentation)
accordingly
- [x] I have added tests to cover my changes
- [x] I have linked related issues ([read github docs](

https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))
- [ ] I have increased versions of npm packages if it is necessary
([cvat-canvas](https://github.com/cvat-ai/cvat/tree/develop/cvat-canvas#versioning),

[cvat-core](https://github.com/cvat-ai/cvat/tree/develop/cvat-core#versioning),
[cvat-data](https://github.com/cvat-ai/cvat/tree/develop/cvat-data#versioning)
and
[cvat-ui](https://github.com/cvat-ai/cvat/tree/develop/cvat-ui#versioning))

### License

- [x] I submit _my code changes_ under the same [MIT License](
https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the
project.
  Feel free to contact the maintainers if that's a concern.

Co-authored-by: Melanie Day <mday@EYEDIAGNOSIS.local>
Co-authored-by: Maria Khrustaleva <maria@cvat.ai>
Co-authored-by: Nikita Manovich <nikita@cvat.ai>
3 years ago
Maria Khrustaleva b00bc653ff
Social account authentication tests (#5444)
Depends on #5349
Related #5432
Added tests for social account authentication functionality: cypress
test with dummy auth server
3 years ago
Andrey Zhavoronkov 81f2b90368
Updated installation guide and workflows (#5578) 3 years ago
Kirill Sizov 7df2b2fded
Add search cache workflow (#5560)
Added separate workflow that search cache, it will allow us to remove
duplication of this logic in `main.yml`, `full.yml` and `schedule.yml`

! This solution won't work if PR changes search-cache.yml, but this
workflow is supposed to almost never change
3 years ago
Roman Donchenko 580ae49e5a
Migrate to Docker Compose V2 (#5524)
Also, migrate to the version less Compose file format.

Compose V1 is EOL:
<https://www.docker.com/blog/announcing-compose-v2-general-availability/>.
3 years ago
Maxim Zhiltsov ec3e1f34a4
Better reporting for user limits (#5225)
- Added explanatory messages for actions denied for user limits
- Fixed few rules and checks
- Upgraded OPA version
3 years ago
Andrey Zhavoronkov aa4980eea5
Added missed imagePullPolicy for all CVAT deployments (#5504) 3 years ago
Andrey Zhavoronkov 321ba4f21d
Run Helm tests for all PRs (#5506) 3 years ago
Nikita Manovich 207116705f
v2.4.0.alpha (#5508)
Co-authored-by: Maxim Zhiltsov <zhiltsov.max35@gmail.com>
Co-authored-by: Kirill Sizov <kirill.sizov@cvat.ai>
3 years ago
Maxim Zhiltsov af65707eee
Mark tests that require external services (#5474)
Related: #5225 

External services are not available when we execute in Helm.

- Added a way to mark REST API tests that require external services to
run
- Changed the way of filtering tests in Helm tests

Currently, we can't execute external service mocks in tests, and we
ignore related tests in the Helm
execution command. But this command may be outdated, because Helm tests
are not executed in each PR.
This PR allows to indicate related tests and filter them out without the
need to synchronize the CI command.
3 years ago
Roman Donchenko 487c60ce2b
SDK: Add an adapter layer that presents a CVAT task as a torchvision dataset (#5417) 3 years ago
Roman Donchenko 5441c4ee67
Fix isort version output in the workflow (#5436)
`isort --version` prints a large banner, which, after shell processing,
is collapsed into one line, and the result is an unreadable mess. Use
`--version-number` instead, which prints just the number.
3 years ago
Maxim Zhiltsov 4e97c243d6
Refactor OPA tests (#5373)
- OPA tests are now autogenerated
- Updated invocation docs
3 years ago
Andrey Zhavoronkov 6f6db4d554
Helm: fix secrets for non-default release name (#5403) 3 years ago
Nikita Manovich a3bf41439d
Fix CI due to cypress verify works too long (#5320)
https://docs.cypress.io/guides/guides/command-line#cypress-verify
3 years ago
Boris Sekachev e5d01359aa
Removed code duplications, added checks to CI (#5253) 3 years ago
Andrey Zhavoronkov 49bdef01f1
IAM OPA bundle for dev environment (#5190) 3 years ago
Kirill Sizov 05598e0830
extend main workflow (#5213) 3 years ago
Kirill Sizov 12a295af72
Extend logs for CI workflows (#5179) 3 years ago
Kirill Sizov 4cd43a27c8
Refactor CI (#5060)
* workflows: add docker image scan for CI workflow

* enter debug mode

* update mod-wsgi dependency

* add Trivy image scanner for cvat/ui image

* add trivyignore file

* update trivyignore file

* try scan without trivyignore

* remove trivy scanner

* add workflow that updates yarn.lock file for PRs from Snyk

* remove extra empty lines

* revert changes for mod-wsgi version

* remove deprecated set-output for workflows

* update upload-artifact action

* update upload-artifact action

* test another github action for getting changed files

* fix typo

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* revert changes for bandit linter

* remove debug changes

* remove debug changes

* remove debug changes

* remove debug changes

* exit debug mode
3 years ago
Andrey Zhavoronkov 469217cc55
Added support for webhooks in Helm charts (#5118)
* added webhook support

* added webhook tests

* update_version supports helm values

* fix

* debug

* fix

* fix remote file test

* increase minikube resources

* fix syntax

* Update helm.yml

* fixed comment

* Update .github/workflows/helm.yml

Co-authored-by: Kirill Sizov <kirill.sizov@cvat.ai>

Co-authored-by: Kirill Sizov <kirill.sizov@cvat.ai>
3 years ago
Kirill Sizov ce09f28512
Fix SDK tests: login once per class (#5123) 3 years ago
Kirill Sizov ad4635e020
Update Github Actions (#5136)
* update actions/checkout version

* update actions/setup-python version

* update actoins/setup-node version
3 years ago
Kirill Sizov 6958971482
Improve report information for Black linter (#5124) 3 years ago
Kirill Sizov dbcfb30cb2
Fix SDK tests (#5077)
* Update SDK tests to use LoginSerializerEx
3 years ago
Maxim Zhiltsov 426f8e3ef0
Extend SDK layer 1 docs (#5011) 3 years ago
Maxim Zhiltsov 6654366021
Autoformat python tests (#5021) 3 years ago
Kirill Sizov 6274bd11b6
workflows: update value of status for `waiting server` step (#5026) 3 years ago
Kirill Sizov 20bf2883f0
Full check: use SHA instead of branch name (#5009) 3 years ago
Andrey Zhavoronkov e22eac7d9b
Helm tests (#4949) 3 years ago
Maria Khrustaleva 56e8c1eb82
Cypress tests for source & target storage (#4914)
* Draft version

* Implemented import & fixed export && some code cleanup && some fixes &&  updated notifications

* Refactoring && fixed several bugs

* Update licence headers && small fixes

* Update remaining licence headers && small changes

* Fix part of tests

* Fix tests

* Remove unused code

* Fix part of comments

* Some fixes

* Move file download process to job

* Rename methods

* Small fix

* Fix storages configuration for tasks

* Styles

* Remove unused

* Change storage configuration && fix forms reset && some fixes

* Update imports

* Remove extra argument type

* Add catch

* Fix import backup from local

* Import architecture refactoring && some fixes

* Rename props

* Small reorganization of export architecture &&  minor fixes

* Remove unused import

* Small fix && skip error notification when no permissions

* Fix project creating

* Fix part of eslint issues

* Fix eslint

* Fix eslint

* Fix eslint

* eslint

* Fix some eslint issues

* Combine uploadAnnotations and importDataset

* Fix annotation uploading from local

* Update tests

* Fix annotation uploading

* Fix notification

* Update dependencies

* fix

* Update jest tests

* Skip error notification when no permissions

* Update case 91 92 canvas3d tests

* Styles

* Update icons

* eslint

* eslint

* eslint

* eslint

* Create & delete cloud storage

* Common part && export job annotations

* Update tests && add tests for project backup

* Fix typo

* Rename variables

* debug

* Revert timeout

* Fix server host

* Fix test with project backup restore

* small refactoring

* Update issue_2473_import_annotations_frames_dots_in_name
3 years ago
Maxim Zhiltsov 68375ec23e
Add SDK docs (#4928)
- Added auth docs in API schema and SDK ApiClient/Config (fixes #1517)
- Added SDK docs with API, SDK and CLI docs
- Added `develop` branch in the docs
- Allowed unauthorized access to `api/docs`, `api/swagger`, `api/schema` endpoints
- Added `--insecure` env var to control host checks in CLI
- Refactored `build_docs.py` (backported https://github.com/openvinotoolkit/datumaro/pull/589)
- Extracted requirements file for site
3 years ago
Kirill Sizov 0b6b758a56
Fix Nightly build (#4941) 3 years ago
Kirill Sizov a50d38f9e9
Fix docker tag (#4940) 3 years ago
Kirill Sizov 26c0a6509a
Workflows: fix typo (#4939)
Co-authored-by: kirill-sizov <kirill.sizov@intel.com>
3 years ago
Kirill Sizov 860b0d96a6
Update workflow configs (#4930) 3 years ago
Nikita Manovich 3957892954
Fix workflows after cvat container was renamed (#4848) 3 years ago
Maxim Zhiltsov 53697ecac5
SDK layer 2 - cover RC1 usecases (#4813) 4 years ago
kirill-sizov 2813503858 fix container name 4 years ago
Andrey Zhavoronkov 5e2eda7601
Extend helm to support Serverless functions and Analytics (#110) 4 years ago
Maxim Zhiltsov 5f58a0f7be
Add 2nd layer of SDK (#19) 4 years ago
Kirill Sizov 3a88677eb4
Workflow dispatch for full check (#92)
* add workflow_dispatch trigger
* full.yml: add inputs for workflow dispatch trigger
4 years ago
Kirill Sizov 776a549fb9
Add workflow_dispatch trigger for full check (#91)
add workflow_dispatch trigger
4 years ago
Kirill Sizov 6dd662b8e7
Hotfix for publishing dev versions of DockerHub images (#80)
Rename dockerhub workspace
4 years ago