Added automount using /etc/fstab

5 years ago · cf406f51dc
parent 0bf3153265
commit cf406f51dc
1 changed files with 129 additions and 76 deletions
--- a/cvat/apps/documentation/mounting_cloud_storages.md
+++ b/cvat/apps/documentation/mounting_cloud_storages.md
@ -1,68 +1,83 @@
 - [Mounting cloud storage](#mounting-cloud-storage)
  - [AWS S3 bucket](#aws-s3-bucket-as-filesystem)
-    - [Ubuntu 20.04](#ubuntu-20.04)
-      - [Mount](#mount)
-      - [Cheking](#cheking)
-      - [Automounting](#automounting)
-        - [Using /etc/fstab]('#using-fstab')
-        - [Using systemd](#using-systemd)
-      - [Unmount](#unmount-filesystem)
+    - [Ubuntu 20.04](#aws_s3_ubuntu_2004)
+      - [Mount](#aws_s3_mount)
+      - [Automounting](#aws_s3_automounting)
+        - [Using /etc/fstab](#aws_s3_using_fstab)
+        - [Using systemd](#aws_s3_using_systemd)
+      - [Cheking](#aws_s3_cheking)
+      - [Unmount](#aws_s3_unmount_filesystem)
  - [Azure container](#microsoft-azure-container-as-filesystem)
-    - [Ubuntu 20.04](#ubuntu-20.04)
-      - [Mount](#mount)
-      - [Cheking](#cheking)
-      - [Automounting](#automounting)
-        - [Using /etc/fstab]('#using-fstab')
-        - [Using systemd](#using-systemd)
-      - [Unmount](#unmount-filesystem)
+    - [Ubuntu 20.04](#azure_ubuntu_2004)
+      - [Mount](#azure_mount)
+      - [Automounting](#azure_automounting)
+        - [Using /etc/fstab]('#azure_using_fstab')
+        - [Using systemd](#azure_using_systemd)
+      - [Cheking](#azure_cheking)
+      - [Unmount](#azure_unmount_filesystem)
  - [Google Drive](#google-drive-as-filesystem)
-    - [Ubuntu 20.04](#ubuntu-20.04)
-      - [Mount](#mount)
-      - [Cheking](#cheking)
-      - [Automounting](#automounting)
-        - [Using /etc/fstab]('#using-fstab')
-        - [Using systemd](#using-systemd)
-      - [Unmount](#unmount-filesystem)
-  -
+    - [Ubuntu 20.04](#google_drive_ubuntu_2004)
+      - [Mount](#google_drive_mount)
+      - [Automounting](#google_drive_automounting)
+        - [Using /etc/fstab]('#google_drive_using_fstab')
+        - [Using systemd](#google_drive_using_systemd)
+      - [Cheking](#google_drive_cheking)
+      - [Unmount](#google_drive_unmount_filesystem)
+
 # Mounting cloud storage
 ## AWS S3 bucket as filesystem
-### Ubuntu 20.04
-#### Mount
+### <a name="aws_s3_ubuntu_2004">Ubuntu 20.04</a>
+#### <a name="aws_s3_mount">Mount</a>

 1. Install s3fs
+
 ```bash
 sudo apt install s3fs
 ```

 2. Enter your credentials in a file  `${HOME}/.passwd-s3fs`  and set owner-only permissions:
+
 ```bash
 echo ACCESS_KEY_ID:SECRET_ACCESS_KEY > ${HOME}/.passwd-s3fs
 chmod 600 ${HOME}/.passwd-s3fs
 ```

 3. Uncomment `user_allow_other` in the `/etc/fuse.conf` file: `sudo nano /etc/fuse.conf`
-
 4. Run s3fs, replace `bucket_name`, `mount_point`:
+
 ```bash
 s3fs <bucket_name> <mount_point> -o allow_other
 ```

 For more details see [here](https://github.com/s3fs-fuse/s3fs-fuse).

-#### Cheking
-A file `/etc/mtab` contains records of currently mounted filesystems.
+#### <a name="aws_s3_automounting">Automounting</a>
+Follow the 3 mounting steps above.
+
+##### <a name="aws_s3_using_fstab">Using fstab</a>
+
+1. Create a shell script named aws_s3_fuse(e.g in /usr/bin, as root) with this content
+(replace `user_name` on whose behalf the disk will be mounted, `backet_name`, `mount_point`, `/path/to/.passwd-s3fs`):
+
 ```bash
-cat /etc/mtab | grep 's3fs'
+#!/bin/bash
+sudo -u <user_name> s3fs <backet_name> <mount_point> -o passwd_file=/path/to/.passwd-s3fs -o allow_other
+exit 0
 ```

-#### Automounting
-##### Using fstab
-TODO
+2. Give it the exec permission:
+
+```bash
+sudo chmod +x /usr/bin/aws_s3_fuse
 ```
-cat /etc/mtab | grep s3fs >> /etc/fstab
-doesn't work
+
+3. Edit `/etc/fstab` adding a line like this, replace `mount_point`):
+
+```
+/absolute/path/to/aws_s3_fuse  <mount_point>     fuse    allow_other,user,_netdev     0       0
 ```
-##### Using systemd
+
+##### <a name="aws_s3_using_systemd">Using systemd</a>

 1. Create unit file `sudo nano /etc/systemd/system/s3fs.service`.
 Replace `user_name`, `bucket_name`, `mount_point`, `/path/to/.passwd-s3fs`
@ -73,7 +88,7 @@ Description=FUSE filesystem over AWS S3 bucket
 After=network.target

 [Service]
-Environment=MOUNT_POINT=<mount_point>
+Environment="MOUNT_POINT=<mount_point>"
 User=<user_name>
 Group=<user_name>
 ExecStart=s3fs <bucket_name> ${MOUNT_POINT} -o passwd_file=/path/to/.passwd-s3fs -o allow_other
@ -86,20 +101,27 @@ WantedBy=multi-user.target
 ```

 2. Update the system configurations, enable unit autorun when the system boots.
+
 ```bash
 sudo systemctl daemon-reload
 sudo systemctl enable s3fs.service
+sudo systemctl start s3fs.service
 ```

+#### <a name="aws_s3_checking">Cheking</a>
+A file `/etc/mtab` contains records of currently mounted filesystems.
+```bash
+cat /etc/mtab | grep 's3fs'
+```

-#### Unmount filesystem
+#### <a name="aws_s3_unmount_filesystem">Unmount filesystem</a>
 ```bash
 fusermount -u <mount_point>
 ```

 ## Microsoft Azure container as filesystem
-### Ubuntu 20.04
-#### Mount
+### <a name="azure_ubuntu_2004">Ubuntu 20.04</a>
+#### <a name="azure_mount">Mount</a>
 1. Set up the Microsoft package repository.(More [here](https://docs.microsoft.com/en-us/windows-server/administration/Linux-Package-Repository-for-Microsoft-Software#configuring-the-repositories))

 ```bash
@ -107,45 +129,51 @@ wget https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.
 sudo dpkg -i packages-microsoft-prod.deb
 sudo apt-get update
 ```
+
 2. Install `blobfuse` and `fuse`
+
 ```bash
 sudo apt-get install blobfuse fuse
 ```
 For more details see [here](https://github.com/Azure/azure-storage-fuse/wiki/1.-Installation)

 3. Create enviroments(replace `account_name`, `account_key`, `mount_point`)
+
 ```bash
 export AZURE_STORAGE_ACCOUNT=<account_name>
 export AZURE_STORAGE_ACCESS_KEY=<account_key>
 MOUNT_POINT=<mount_point>
 ```

-3. Create a folder for cache:
+4. Create a folder for cache:
 ```bash
 sudo mkdir -p /mnt/blobfusetmp
 ```

-4. Make sure the file must be owned by the user who mounts the container:
+5. Make sure the file must be owned by the user who mounts the container:
 ```bash
 sudo chown <user> /mnt/blobfusetmp
 ```

-5. Create the mount point, if it doesn't exists:
+6. Create the mount point, if it doesn't exists:
 ```bash
 mkdir -p ${MOUNT_POINT}
 ```

-6. Uncomment `user_allow_other` in the `/etc/fuse.conf` file: `sudo nano /etc/fuse.conf`
+7. Uncomment `user_allow_other` in the `/etc/fuse.conf` file: `sudo nano /etc/fuse.conf`
+8. Mount container(replace `your_container`):

-7. Mount container(replace `your_container`):
 ```bash
 blobfuse ${MOUNT_POINT} --container-name=<your_container> --tmp-path=/mnt/blobfusetmp -o allow_other
 ```

-#### Automounting
+#### <a name="azure_automounting">Automounting/a<>
+Follow the 3 mounting steps above.
+##### <a name="azure_using_fstab">Using fstab</a>
+
+1. Create configuration file `connection.cfg` with same content, change accountName,
+select one from accountKey or sasToken and replace with your value

-##### Using fstab
-1. Create configuration file `connection.cfg` with same content, change accountName, select one from accountKey or sasToken and replace with your value
 ```
 accountName <account-name-here>
 # Please provide either an account key or a SAS token, and delete the other line.
@ -156,19 +184,26 @@ authType <MSI/SAS/SPN/Key/empty>
 containerName <insert-container-name-here>
 ```

-2. create `mount.sh` with content below:
+2. create `azure_fuse`(e.g in /usr/bin, as root) with content below
+(replace `user_name` on whose behalf the disk will be mounted, `mount_point`, `/path/to/blobfusetmp`,`/path/to/connection.cfg`):
+
 ```
 #!/bin/bash
-BLOBFS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
-cd $BLOBFS_DIR/build
-./blobfuse $1 --tmp-path=/mnt/blobfusetmp --use-attr-cache=true -o attr_timeout=240
-o entry_timeout=240 -o negative_timeout=120 -o allow_other --config-file=../connection.cfg
+sudo -u <user_name> blobfuse <mount_point> --tmp-path=/path/to/blobfusetmp  --config-file=/path/to/connection.cfg -o allow_other
+exit 0
+
 ```

-4. Edit `/etc/fstab` with the blobfuse script. Add the following line:
-`/<path_to_blobfuse>/mount.sh </path/to/desired/mountpoint> fuse _netdev,allow_other`
+3. Give it the exec permission:
+```bash
+sudo chmod +x /usr/bin/azure_fuse
+```
+
+4. Edit `/etc/fstab` with the blobfuse script. Add the following line(replace paths):
+`/absolute/path/to/mount.sh </path/to/desired/mountpoint> fuse _netdev,allow_other`
+
+##### <a name="azure_using_systemd">Using systemd</a>

-##### Using systemd
 1. Create unit file `sudo nano /etc/systemd/system/blobfuse.service`.
 Replace `user_name`, `mount_point`, `container_name`,`/path/to/connection.cfg`

@ -178,7 +213,7 @@ Description=FUSE filesystem over Azure container
 After=network.target

 [Service]
-Environment=MOUNT_POINT=<mount_point>
+Environment="MOUNT_POINT=<mount_point>"
 User=<user_name>
 Group=<user_name>
 ExecStart=blobfuse ${MOUNT_POINT} --container-name=<container_name> --tmp-path=/mnt/blobfusetmp --config-file=/path/to/connection.cfg -o allow_other
@ -191,14 +226,21 @@ WantedBy=multi-user.target
 ```

 2. Update the system configurations, enable unit autorun when the system boots:
+
 ```bash
 sudo systemctl daemon-reload
 sudo systemctl enable blobfuse.service
+sudo systemctl blobfuse s3fs.service
 ```
-
 Or for more detail [see here](https://github.com/Azure/azure-storage-fuse/tree/master/systemd)

-#### Unmount the filesystem
+#### <a name="azure_cheking">Cheking</a>
+A file `/etc/mtab` contains records of currently mounted filesystems.
+```bash
+cat /etc/mtab | grep 'blobfuse'
+```
+
+#### <a name="azure_unmount_filesystem">Unmount filesystem</a>
 ```bash
 fusermount -u <mount_point>
 ```
@ -207,12 +249,14 @@ If you have any mounting problems, check out the [answers](https://github.com/Az
 to common problems

 ## Google Drive as filesystem
-### Ubuntu 20.04
-#### Mount
+### <a name="google_drive_ubuntu_2004">Ubuntu 20.04</a>
+#### <a name="google_drive_mount">Mount</a>
 To mount a google drive as a filesystem in user space(FUSE)
 you can use [google-drive-ocamlfuse](https://github.com/astrada/google-drive-ocamlfuse)
 To do this follow the instructions below:
+
 1. Install google-drive-ocamlfuse
+
 ```
 sudo add-apt-repository ppa:alessandro-strada/ppa
 sudo apt-get update
@ -220,7 +264,8 @@ sudo apt-get install google-drive-ocamlfuse
 ```

 2. Run `google-drive-ocamlfuse` without parameters
-```
+
+```bash
 google-drive-ocamlfuse
 ```
 This command will create the default application directory (~/.gdfuse/default), containing the configuration file config
@ -231,6 +276,7 @@ This will let you modify default configuration before mounting the filesystem.
 Then you can choose a local directory to mount your Google Drive (e.g.: ~/GoogleDrive).

 3. Create the mount point, if it doesn't exist(replace mount_point):
+
 ```bash
 mountpoint="<mount_point>"
 mkdir -p $mountpoint
@ -243,34 +289,33 @@ mkdir -p $mountpoint
 google-drive-ocamlfuse -o allow_other $mountpoint
 ```

-#### Cheking
-A file `/etc/mtab` contains records of currently mounted filesystems.
-```bash
-cat /etc/mtab | grep 'google-drive-ocamlfuse'
-```
+#### <a name="google_drive_automounting">Automounting</a>
+Follow the 3 mounting steps above.
+##### <a name="google_drive_using_fstab">Using fstab</a>
+
+1. Create a shell script named gdfuse(e.g in /usr/bin, as root) with this content
+(replace `user_name` on whose behalf the disk will be mounted, `label`, `mount_point`):

-#### Automounting
-##### Using fstab
-1. Create a shell script named gdfuse in /usr/bin (as root) with this content:
 ```bash
 #!/bin/bash
-
-su $USERNAME -l -c "google-drive-ocamlfuse -label $1 $*"
+sudo -u <user_name> google-drive-ocamlfuse -o allow_other -label <label> <mount_point>
 exit 0
+
 ```
 2. Give it the exec permission:
 ```bash
 sudo chmod +x /usr/bin/gdfuse
 ```

-3. Edit `/etc/fstab` adding a line like this, replace `label`(dafualt `label=default`):
+3. Edit `/etc/fstab` adding a line like this, replace `mount_point`):
 ```
-gdfuse#<label>  /home/$USERNAME/gdrive     fuse    uid=1000,gid=1000,allow_other,user,_netdev     0       0
+/absolute/path/to/gdfuse  <mount_point>     fuse    allow_other,user,_netdev     0       0
 ```

-For more details see [instructions](https://github.com/astrada/google-drive-ocamlfuse/wiki/Automounting)
+For more details see [here](https://github.com/astrada/google-drive-ocamlfuse/wiki/Automounting)
+
+##### <a name="google_drive_using_systemd">Using systemd</a>

-##### Using systemd
 1. Create unit file `sudo nano /etc/systemd/system/google-drive-ocamlfuse.service`.
 Replace `user_name`, `label`(default `label=default`), `mount_point`.

@ -280,7 +325,7 @@ Description=FUSE filesystem over Google Drive
 After=network.target

 [Service]
-Environment=MOUNT_POINT=<mount_point>
+Environment="MOUNT_POINT=<mount_point>"
 User=<user_name>
 Group=<user_name>
 ExecStart=google-drive-ocamlfuse -label <label> ${MOUNT_POINT}
@ -293,14 +338,22 @@ WantedBy=multi-user.target
 ```

 2. Update the system configurations, enable unit autorun when the system boots.
+
 ```bash
 sudo systemctl daemon-reload
 sudo systemctl enable google-drive-ocamlfuse.service
+sudo systemctl start google-drive-ocamlfuse.service
 ```

 For more details see [here](https://github.com/astrada/google-drive-ocamlfuse/wiki/Automounting)

-#### Unmount the filesystem
+#### <a name="google_drive_cheking">Cheking</a>
+A file `/etc/mtab` contains records of currently mounted filesystems.
+```bash
+cat /etc/mtab | grep 'google-drive-ocamlfuse'
+```
+
+#### <a name="google_drive_unmount_filesystem">Unmount filesystem</a>
 ```bash
 fusermount -u <mount_point>
 ```