This PR adds an option to specify file to job mapping explicitly during
task creation. This option is incompatible with most other job-related
parameters like `sorting_method` and `frame_step`.
- Added a new task creation parameter (`job_file_mapping`) to set a
custom file to job mapping during task creation
Django REST Framework ignores the Content-Type on request body parts, so
it doesn't know that they are JSON-encoded. Instead, it just tries to
decode each part as if it was an `str()`-encoded value.
Change the encoding to match the decoding. The only type this matters
for is `str`, because `json.dumps` and `str` produce different encodings
for `str` values.
Remove `none_type` from the list of encodable types since, to my
knowledge, there's no way to encode a `None` value as a
`multipart/form-data` part in a way that DRF will understand.
This will let users to run their PyTorch code without network access,
provided that they have already cached the data.
### How has this been tested?
<!-- Please describe in detail how you tested your changes.
Include details of your testing environment, and the tests you ran to
see how your change affects other areas of the code, etc. -->
Unit tests.
Extracted from https://github.com/opencv/cvat/pull/5083
- Added a default arg for task data uploading
- Added an option to wait for the data processing in task data uploading
- Moved data splitting by requests for TUS closer to the point of use
You have to use the `import_status` action in order to query the input
status. Otherwise, the `/api/projects/{id}/dataset/` endpoint initiates
a dataset export. Currently, `import_dataset` inadvertently monitors the
status of that export, not the original import.
For user-facing functions, keep accepting `str` paths to maintain
compatibility and flexibility, but add support for arbitrary path-like
objects. For internal functions (in `downloading.py` and
`uploading.py`), don't bother and require `pathlib.Path`.
The only code that isn't converted is build-time code (e.g. `setup.py`)
and code that came from openapi-generator.
* Fix TUS offset queries in production environments
Previously, `mod_wsgi` would convert `HEAD` requests into `GET`, which
would be rejected, so clients were unable to resume an upload that failed
midway through.
To make use of this, update the SDK code to enable upload resumption.
* Add a test that forces a chunked TUS upload
* test_can_create_from_backup_in_chunks: make sure the upload is actually chunked
tqdm does not necessarily redraw the progress bar after every update,
and without the call to `refresh` it might not update it to the final state.
I discovered this while writing a test in which the progress bar is updated
in chunks, and the last chunk is smaller than the previous ones.
* Move the test file share contents to tests/share
This way, it can be reused between tests.
* cvat-sdk: Fix creating tasks with non-local files
Forcing the `Content-Type` for the `upload_data` API call to
`multipart/form-data` does not work, because the current logic for
converting Python values to parts (`ApiClient._convert_body_to_post_params`)
does not encode them in a way that Django REST Framework can understand (it
JSON-encodes each part).
Fortunately, we don't actually need to do that, since when we create a task
with non-local files, we don't need to upload any files, and so we can just
post the original JSON, so do just that.
I couldn't add a test for the remote image case, because CVAT rejects all URLs
with non-public IP addressses. However, I did test this case manually.
- Added auth docs in API schema and SDK ApiClient/Config (fixes#1517)
- Added SDK docs with API, SDK and CLI docs
- Added `develop` branch in the docs
- Allowed unauthorized access to `api/docs`, `api/swagger`, `api/schema` endpoints
- Added `--insecure` env var to control host checks in CLI
- Refactored `build_docs.py` (backported https://github.com/openvinotoolkit/datumaro/pull/589)
- Extracted requirements file for site
Maxim Zhiltsov
Roman Donchenko
Maria Khrustaleva
Andrey Zhavoronkov
Kirill Sizov