wangchunlin
/
cvat
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Fixed codacy issues

Browse Source
main
Maya 5 years ago
parent cf406f51dc
commit 39c2d17586
1 changed files with 256 additions and 230 deletions
Show Stats Download Patch File Download Diff File

486
cvat/apps/documentation/mounting_cloud_storages.md
Unescape Escape View File

@ -2,26 +2,26 @@
- [AWS S3 bucket](#aws-s3-bucket-as-filesystem)
- [Ubuntu 20.04](#aws_s3_ubuntu_2004)
- [Mount](#aws_s3_mount)
- [Automounting](#aws_s3_automounting)
- [Automatically mount](#aws_s3_automatically_mount)
- [Using /etc/fstab](#aws_s3_using_fstab)
- [Using systemd](#aws_s3_using_systemd)
- [Cheking](#aws_s3_cheking)
- [Check](#aws_s3_check)
- [Unmount](#aws_s3_unmount_filesystem)
- [Azure container](#microsoft-azure-container-as-filesystem)
- [Ubuntu 20.04](#azure_ubuntu_2004)
- [Mount](#azure_mount)
- [Automounting](#azure_automounting)
- [Using /etc/fstab]('#azure_using_fstab')
- [Automatically mount](#azure_automatically_mount)
- [Using /etc/fstab](#azure_using_fstab)
- [Using systemd](#azure_using_systemd)
- [Cheking](#azure_cheking)
- [Check](#azure_check)
- [Unmount](#azure_unmount_filesystem)
- [Google Drive](#google-drive-as-filesystem)
- [Ubuntu 20.04](#google_drive_ubuntu_2004)
- [Mount](#google_drive_mount)
- [Automounting](#google_drive_automounting)
- [Using /etc/fstab]('#google_drive_using_fstab')
- [Automatically mount](#google_drive_automatically_mount)
- [Using /etc/fstab](#google_drive_using_fstab)
- [Using systemd](#google_drive_using_systemd)
- [Cheking](#google_drive_cheking)
- [Check](#google_drive_check)
- [Unmount](#google_drive_unmount_filesystem)
# Mounting cloud storage
@ -29,86 +29,86 @@
### <a name="aws_s3_ubuntu_2004">Ubuntu 20.04</a>
#### <a name="aws_s3_mount">Mount</a>
1. Install s3fs
1. Install s3fs:
```bash
sudo apt install s3fs
```
```bash
sudo apt install s3fs
```
2. Enter your credentials in a file `${HOME}/.passwd-s3fs` and set owner-only permissions:
1. Enter your credentials in a file `${HOME}/.passwd-s3fs` and set owner-only permissions:
```bash
echo ACCESS_KEY_ID:SECRET_ACCESS_KEY > ${HOME}/.passwd-s3fs
chmod 600 ${HOME}/.passwd-s3fs
```
```bash
echo ACCESS_KEY_ID:SECRET_ACCESS_KEY > ${HOME}/.passwd-s3fs
chmod 600 ${HOME}/.passwd-s3fs
```
3. Uncomment `user_allow_other` in the `/etc/fuse.conf` file: `sudo nano /etc/fuse.conf`
4. Run s3fs, replace `bucket_name`, `mount_point`:
1. Uncomment `user_allow_other` in the `/etc/fuse.conf` file: `sudo nano /etc/fuse.conf`
1. Run s3fs, replace `bucket_name`, `mount_point`:
```bash
s3fs <bucket_name> <mount_point> -o allow_other
```
```bash
s3fs <bucket_name> <mount_point> -o allow_other
```
For more details see [here](https://github.com/s3fs-fuse/s3fs-fuse).
#### <a name="aws_s3_automounting">Automounting</a>
Follow the 3 mounting steps above.
#### <a name="aws_s3_automatically_mount">Automatically mount</a>
Follow the first 3 mounting steps above.
##### <a name="aws_s3_using_fstab">Using fstab</a>
1. Create a shell script named aws_s3_fuse(e.g in /usr/bin, as root) with this content
(replace `user_name` on whose behalf the disk will be mounted, `backet_name`, `mount_point`, `/path/to/.passwd-s3fs`):
1. Create a bash script named aws_s3_fuse(e.g in /usr/bin, as root) with this content
(replace `user_name` on whose behalf the disk will be mounted, `backet_name`, `mount_point`, `/path/to/.passwd-s3fs`):
```bash
#!/bin/bash
sudo -u <user_name> s3fs <backet_name> <mount_point> -o passwd_file=/path/to/.passwd-s3fs -o allow_other
exit 0
```
```bash
#!/bin/bash
sudo -u <user_name> s3fs <backet_name> <mount_point> -o passwd_file=/path/to/.passwd-s3fs -o allow_other
exit 0
```
2. Give it the exec permission:
1. Give it the execution permission:
```bash
sudo chmod +x /usr/bin/aws_s3_fuse
```
```bash
sudo chmod +x /usr/bin/aws_s3_fuse
```
3. Edit `/etc/fstab` adding a line like this, replace `mount_point`):
1. Edit `/etc/fstab` adding a line like this, replace `mount_point`):
```
/absolute/path/to/aws_s3_fuse <mount_point> fuse allow_other,user,_netdev 0 0
```
```bash
/absolute/path/to/aws_s3_fuse <mount_point> fuse allow_other,user,_netdev 0 0
```
##### <a name="aws_s3_using_systemd">Using systemd</a>
1. Create unit file `sudo nano /etc/systemd/system/s3fs.service`.
Replace `user_name`, `bucket_name`, `mount_point`, `/path/to/.passwd-s3fs`
1. Create unit file `sudo nano /etc/systemd/system/s3fs.service`
(replace `user_name`, `bucket_name`, `mount_point`, `/path/to/.passwd-s3fs`):
```
[Unit]
Description=FUSE filesystem over AWS S3 bucket
After=network.target
[Service]
Environment="MOUNT_POINT=<mount_point>"
User=<user_name>
Group=<user_name>
ExecStart=s3fs <bucket_name> ${MOUNT_POINT} -o passwd_file=/path/to/.passwd-s3fs -o allow_other
ExecStop=fusermount -u ${MOUNT_POINT}
Restart=always
Type=forking
[Install]
WantedBy=multi-user.target
```
```bash
[Unit]
Description=FUSE filesystem over AWS S3 bucket
After=network.target
2. Update the system configurations, enable unit autorun when the system boots.
[Service]
Environment="MOUNT_POINT=<mount_point>"
User=<user_name>
Group=<user_name>
ExecStart=s3fs <bucket_name> ${MOUNT_POINT} -o passwd_file=/path/to/.passwd-s3fs -o allow_other
ExecStop=fusermount -u ${MOUNT_POINT}
Restart=always
Type=forking
```bash
sudo systemctl daemon-reload
sudo systemctl enable s3fs.service
sudo systemctl start s3fs.service
```
[Install]
WantedBy=multi-user.target
```
1. Update the system configurations, enable unit autorun when the system boots, mount the bucket:
#### <a name="aws_s3_checking">Cheking</a>
```bash
sudo systemctl daemon-reload
sudo systemctl enable s3fs.service
sudo systemctl start s3fs.service
```
#### <a name="aws_s3_check">Check</a>
A file `/etc/mtab` contains records of currently mounted filesystems.
```bash
cat /etc/mtab | grep 's3fs'
@ -119,122 +119,130 @@ cat /etc/mtab | grep 's3fs'
fusermount -u <mount_point>
```
## Microsoft Azure container as filesystem
### <a name="azure_ubuntu_2004">Ubuntu 20.04</a>
#### <a name="azure_mount">Mount</a>
1. Set up the Microsoft package repository.(More [here](https://docs.microsoft.com/en-us/windows-server/administration/Linux-Package-Repository-for-Microsoft-Software#configuring-the-repositories))
If you used [systemd](#aws_s3_using_systemd) to mount a bucket:
```bash
wget https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
sudo apt-get update
sudo systemctl stop s3fs.service
sudo systemctl disable s3fs.service
```
2. Install `blobfuse` and `fuse`
```bash
sudo apt-get install blobfuse fuse
```
For more details see [here](https://github.com/Azure/azure-storage-fuse/wiki/1.-Installation)
## Microsoft Azure container as filesystem
### <a name="azure_ubuntu_2004">Ubuntu 20.04</a>
#### <a name="azure_mount">Mount</a>
1. Set up the Microsoft package repository.(More [here](https://docs.microsoft.com/en-us/windows-server/administration/Linux-Package-Repository-for-Microsoft-Software#configuring-the-repositories))
3. Create enviroments(replace `account_name`, `account_key`, `mount_point`)
```bash
wget https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
sudo apt-get update
```
```bash
export AZURE_STORAGE_ACCOUNT=<account_name>
export AZURE_STORAGE_ACCESS_KEY=<account_key>
MOUNT_POINT=<mount_point>
```
1. Install `blobfuse` and `fuse`:
4. Create a folder for cache:
```bash
sudo mkdir -p /mnt/blobfusetmp
```
```bash
sudo apt-get install blobfuse fuse
```
For more details see [here](https://github.com/Azure/azure-storage-fuse/wiki/1.-Installation)
5. Make sure the file must be owned by the user who mounts the container:
```bash
sudo chown <user> /mnt/blobfusetmp
```
1. Create enviroments(replace `account_name`, `account_key`, `mount_point`):
6. Create the mount point, if it doesn't exists:
```bash
mkdir -p ${MOUNT_POINT}
```
7. Uncomment `user_allow_other` in the `/etc/fuse.conf` file: `sudo nano /etc/fuse.conf`
8. Mount container(replace `your_container`):
```bash
blobfuse ${MOUNT_POINT} --container-name=<your_container> --tmp-path=/mnt/blobfusetmp -o allow_other
```
```bash
export AZURE_STORAGE_ACCOUNT=<account_name>
export AZURE_STORAGE_ACCESS_KEY=<account_key>
MOUNT_POINT=<mount_point>
```
#### <a name="azure_automounting">Automounting/a<>
Follow the 3 mounting steps above.
##### <a name="azure_using_fstab">Using fstab</a>
1. Create a folder for cache:
```bash
sudo mkdir -p /mnt/blobfusetmp
```
1. Create configuration file `connection.cfg` with same content, change accountName,
select one from accountKey or sasToken and replace with your value
1. Make sure the file must be owned by the user who mounts the container:
```bash
sudo chown <user> /mnt/blobfusetmp
```
```
accountName <account-name-here>
# Please provide either an account key or a SAS token, and delete the other line.
accountKey <account-key-here-delete-next-line>
#change authType to specify only 1
sasToken <shared-access-token-here-delete-previous-line>
authType <MSI/SAS/SPN/Key/empty>
containerName <insert-container-name-here>
```
1. Create the mount point, if it doesn't exists:
```bash
mkdir -p ${MOUNT_POINT}
```
2. create `azure_fuse`(e.g in /usr/bin, as root) with content below
(replace `user_name` on whose behalf the disk will be mounted, `mount_point`, `/path/to/blobfusetmp`,`/path/to/connection.cfg`):
1. Uncomment `user_allow_other` in the `/etc/fuse.conf` file: `sudo nano /etc/fuse.conf`
1. Mount container(replace `your_container`):
```
#!/bin/bash
sudo -u <user_name> blobfuse <mount_point> --tmp-path=/path/to/blobfusetmp --config-file=/path/to/connection.cfg -o allow_other
exit 0
```bash
blobfuse ${MOUNT_POINT} --container-name=<your_container> --tmp-path=/mnt/blobfusetmp -o allow_other
```
```
#### <a name="azure_automatically_mount">Automatically mount</a>
Follow the first 7 mounting steps above.
##### <a name="azure_using_fstab">Using fstab</a>
3. Give it the exec permission:
1. Create configuration file `connection.cfg` with same content, change accountName,
select one from accountKey or sasToken and replace with your value:
```bash
accountName <account-name-here>
# Please provide either an account key or a SAS token, and delete the other line.
accountKey <account-key-here-delete-next-line>
#change authType to specify only 1
sasToken <shared-access-token-here-delete-previous-line>
authType <MSI/SAS/SPN/Key/empty>
containerName <insert-container-name-here>
```
1. Create a bash script named `azure_fuse`(e.g in /usr/bin, as root) with content below
(replace `user_name` on whose behalf the disk will be mounted, `mount_point`, `/path/to/blobfusetmp`,`/path/to/connection.cfg`):
```bash
#!/bin/bash
sudo -u <user_name> blobfuse <mount_point> --tmp-path=/path/to/blobfusetmp --config-file=/path/to/connection.cfg -o allow_other
exit 0
```
1. Give it the execution permission:
```bash
sudo chmod +x /usr/bin/azure_fuse
```
1. Edit `/etc/fstab` with the blobfuse script. Add the following line(replace paths):
```bash
sudo chmod +x /usr/bin/azure_fuse
/absolute/path/to/azure_fuse </path/to/desired/mountpoint> fuse allow_other,user,_netdev
```
4. Edit `/etc/fstab` with the blobfuse script. Add the following line(replace paths):
`/absolute/path/to/mount.sh </path/to/desired/mountpoint> fuse _netdev,allow_other`
##### <a name="azure_using_systemd">Using systemd</a>
1. Create unit file `sudo nano /etc/systemd/system/blobfuse.service`.
Replace `user_name`, `mount_point`, `container_name`,`/path/to/connection.cfg`
```
[Unit]
Description=FUSE filesystem over Azure container
After=network.target
[Service]
Environment="MOUNT_POINT=<mount_point>"
User=<user_name>
Group=<user_name>
ExecStart=blobfuse ${MOUNT_POINT} --container-name=<container_name> --tmp-path=/mnt/blobfusetmp --config-file=/path/to/connection.cfg -o allow_other
ExecStop=fusermount -u ${MOUNT_POINT}
Restart=always
Type=forking
[Install]
WantedBy=multi-user.target
```
2. Update the system configurations, enable unit autorun when the system boots:
```bash
sudo systemctl daemon-reload
sudo systemctl enable blobfuse.service
sudo systemctl blobfuse s3fs.service
```
Or for more detail [see here](https://github.com/Azure/azure-storage-fuse/tree/master/systemd)
#### <a name="azure_cheking">Cheking</a>
1. Create unit file `sudo nano /etc/systemd/system/blobfuse.service`.
(replace `user_name`, `mount_point`, `container_name`,`/path/to/connection.cfg`):
```bash
[Unit]
Description=FUSE filesystem over Azure container
After=network.target
[Service]
Environment="MOUNT_POINT=<mount_point>"
User=<user_name>
Group=<user_name>
ExecStart=blobfuse ${MOUNT_POINT} --container-name=<container_name> --tmp-path=/mnt/blobfusetmp --config-file=/path/to/connection.cfg -o allow_other
ExecStop=fusermount -u ${MOUNT_POINT}
Restart=always
Type=forking
[Install]
WantedBy=multi-user.target
```
1. Update the system configurations, enable unit autorun when the system boots, mount the container:
```bash
sudo systemctl daemon-reload
sudo systemctl enable blobfuse.service
sudo systemctl start blobfuse.service
```
Or for more detail [see here](https://github.com/Azure/azure-storage-fuse/tree/master/systemd)
#### <a name="azure_check">Check</a>
A file `/etc/mtab` contains records of currently mounted filesystems.
```bash
cat /etc/mtab | grep 'blobfuse'
@ -245,6 +253,13 @@ cat /etc/mtab | grep 'blobfuse'
fusermount -u <mount_point>
```
If you used [systemd](#azure_using_systemd) to mount a container:
```bash
sudo systemctl stop blobfuse.service
sudo systemctl disable blobfuse.service
```
If you have any mounting problems, check out the [answers](https://github.com/Azure/azure-storage-fuse/wiki/3.-Troubleshoot-FAQ)
to common problems
@ -255,99 +270,103 @@ To mount a google drive as a filesystem in user space(FUSE)
you can use [google-drive-ocamlfuse](https://github.com/astrada/google-drive-ocamlfuse)
To do this follow the instructions below:
1. Install google-drive-ocamlfuse
1. Install google-drive-ocamlfuse:
```
sudo add-apt-repository ppa:alessandro-strada/ppa
sudo apt-get update
sudo apt-get install google-drive-ocamlfuse
```
```bash
sudo add-apt-repository ppa:alessandro-strada/ppa
sudo apt-get update
sudo apt-get install google-drive-ocamlfuse
```
2. Run `google-drive-ocamlfuse` without parameters
1. Run `google-drive-ocamlfuse` without parameters:
```bash
google-drive-ocamlfuse
```
This command will create the default application directory (~/.gdfuse/default), containing the configuration file config
(see the [wiki](https://github.com/astrada/google-drive-ocamlfuse/wiki) page for more details about configuration).
And it will start a web browser to obtain authorization to access your Google Drive.
This will let you modify default configuration before mounting the filesystem.
```bash
google-drive-ocamlfuse
```
Then you can choose a local directory to mount your Google Drive (e.g.: ~/GoogleDrive).
This command will create the default application directory (~/.gdfuse/default),
containing the configuration file config (see the [wiki](https://github.com/astrada/google-drive-ocamlfuse/wiki)
page for more details about configuration).
And it will start a web browser to obtain authorization to access your Google Drive.
This will let you modify default configuration before mounting the filesystem.
3. Create the mount point, if it doesn't exist(replace mount_point):
Then you can choose a local directory to mount your Google Drive (e.g.: ~/GoogleDrive).
```bash
mountpoint="<mount_point>"
mkdir -p $mountpoint
```
1. Create the mount point, if it doesn't exist(replace mount_point):
4. Uncomment `user_allow_other` in the `/etc/fuse.conf` file: `sudo nano /etc/fuse.conf`
```bash
mountpoint="<mount_point>"
mkdir -p $mountpoint
```
5. Mount the filesystem
```bash
google-drive-ocamlfuse -o allow_other $mountpoint
```
1. Uncomment `user_allow_other` in the `/etc/fuse.conf` file: `sudo nano /etc/fuse.conf`
1. Mount the filesystem:
```bash
google-drive-ocamlfuse -o allow_other $mountpoint
```
#### <a name="google_drive_automounting">Automounting</a>
Follow the 3 mounting steps above.
#### <a name="google_drive_automatically_mount">Automatically mount</a>
Follow the first 4 mounting steps above.
##### <a name="google_drive_using_fstab">Using fstab</a>
1. Create a shell script named gdfuse(e.g in /usr/bin, as root) with this content
(replace `user_name` on whose behalf the disk will be mounted, `label`, `mount_point`):
1. Create a bash script named gdfuse(e.g in /usr/bin, as root) with this content
(replace `user_name` on whose behalf the disk will be mounted, `label`, `mount_point`):
```bash
#!/bin/bash
sudo -u <user_name> google-drive-ocamlfuse -o allow_other -label <label> <mount_point>
exit 0
```bash
#!/bin/bash
sudo -u <user_name> google-drive-ocamlfuse -o allow_other -label <label> <mount_point>
exit 0
```
```
2. Give it the exec permission:
```bash
sudo chmod +x /usr/bin/gdfuse
```
1. Give it the execution permission:
3. Edit `/etc/fstab` adding a line like this, replace `mount_point`):
```
/absolute/path/to/gdfuse <mount_point> fuse allow_other,user,_netdev 0 0
```
```bash
sudo chmod +x /usr/bin/gdfuse
```
1. Edit `/etc/fstab` adding a line like this, replace `mount_point`):
```bash
/absolute/path/to/gdfuse <mount_point> fuse allow_other,user,_netdev 0 0
```
For more details see [here](https://github.com/astrada/google-drive-ocamlfuse/wiki/Automounting)
For more details see [here](https://github.com/astrada/google-drive-ocamlfuse/wiki/Automounting)
##### <a name="google_drive_using_systemd">Using systemd</a>
1. Create unit file `sudo nano /etc/systemd/system/google-drive-ocamlfuse.service`.
Replace `user_name`, `label`(default `label=default`), `mount_point`.
1. Create unit file `sudo nano /etc/systemd/system/google-drive-ocamlfuse.service`.
(replace `user_name`, `label`(default `label=default`), `mount_point`):
```
[Unit]
Description=FUSE filesystem over Google Drive
After=network.target
[Service]
Environment="MOUNT_POINT=<mount_point>"
User=<user_name>
Group=<user_name>
ExecStart=google-drive-ocamlfuse -label <label> ${MOUNT_POINT}
ExecStop=fusermount -u ${MOUNT_POINT}
Restart=always
Type=forking
[Install]
WantedBy=multi-user.target
```
```bash
[Unit]
Description=FUSE filesystem over Google Drive
After=network.target
2. Update the system configurations, enable unit autorun when the system boots.
[Service]
Environment="MOUNT_POINT=<mount_point>"
User=<user_name>
Group=<user_name>
ExecStart=google-drive-ocamlfuse -label <label> ${MOUNT_POINT}
ExecStop=fusermount -u ${MOUNT_POINT}
Restart=always
Type=forking
```bash
sudo systemctl daemon-reload
sudo systemctl enable google-drive-ocamlfuse.service
sudo systemctl start google-drive-ocamlfuse.service
```
[Install]
WantedBy=multi-user.target
```
1. Update the system configurations, enable unit autorun when the system boots, mount the drive:
For more details see [here](https://github.com/astrada/google-drive-ocamlfuse/wiki/Automounting)
```bash
sudo systemctl daemon-reload
sudo systemctl enable google-drive-ocamlfuse.service
sudo systemctl start google-drive-ocamlfuse.service
```
#### <a name="google_drive_cheking">Cheking</a>
For more details see [here](https://github.com/astrada/google-drive-ocamlfuse/wiki/Automounting)
#### <a name="google_drive_check">Check</a>
A file `/etc/mtab` contains records of currently mounted filesystems.
```bash
cat /etc/mtab | grep 'google-drive-ocamlfuse'
@ -357,3 +376,10 @@ cat /etc/mtab | grep 'google-drive-ocamlfuse'
```bash
fusermount -u <mount_point>
```
If you used [systemd](#google_drive_using_systemd) to mount a drive:
```bash
sudo systemctl stop google-drive-ocamlfuse.service
sudo systemctl disable google-drive-ocamlfuse.service
```

Write Preview
Loading…
Cancel
Save