(Simple Storage Service) is a third-party service for object storage that offers scalability, data availability, security, and industry-leading performance in the object storage industry. It means any customer and any industry can use it to stored and protect any amount of data for various cases, such as website data, enterprise application, IoT device data and big data analytic.
NEO Object Storage service has S3 industry compatibility from Amazon Web Service. For more information, please see the page: https://www.biznetgio.com/neo/cloud/products/neo-flex-storage
This part will explain how to implement S3 service using FUSE (File System in User Space) or known as S3FS which can be used for mounting Amazon S3 bucket or Object Storage bucket as a drive/ hard drive on the Operating System. On the file system which is mounted to the S3FS, we can use Unix commands like cp, mv and ls like locally connected disks. For the implementation of OS Debian 9 and Ubuntu 16.04 have same process because they are in the same Linux distro.
- Create Object Storage on Neo Cloud, for how to create it, you can refer to the page https://kb.biznetgio.com/neo/konfigurasi/membuat-neo-object-storage-pertama
- After finishing creating object storage, the next step is noticed S3 Endpoint, Access Key dan Secret Key.
- Next step is to start to install S3FS.
- Install the packages and dependencies needed to run S3FS.
#sudo apt-get install automake autotools-dev fuse g++ git libcurl4-gnutls-dev libfuse-dev libssl-dev libxml2-dev make pkg-config
- Download dan Compile Fuse.
#cd /usr/src
#git clone https://github.com/s3fs-fuse/s3fs-fuse.git
#cd s3fs-fuse
#./autogen.sh
#./configure
#make
#make install
- Create file for saving Access Key dan Secret Key.
#nano /etc/passwd-s3fs
Between Access Key and Secret Key add sign “:”
run chmod at file passwd-s3fs
#chmod 640 /etc/passed-s3fs
- After installation and configuration finish, next mounting Object Storage to the Operation System.
- Make directory for mounting object storage.
#mkdir /mys3bucket
- Mounting the Object Storage
#s3fs cobabikin_35 /mys3bucket/ -o passwd_file=/etc/passwd-s3fs -o url=https://nos.wjv-1.neo.id -o use_path_request_style –o nonempty –o allow_other
- Check Object storage has been successfully mounted with the df -h command
- To see Object Storage contents, first go into the directory mounted before
- By default, when an instance or server restart, object storage is not automatically mounted to the directory, for automatically mounted add parameter at file /etc/fstab.
- Open file fstab
#nano /etc/fstab
If at the time of the mounting process there is no error, you can try entering the directory and see the contents.
After successfully mounted to the directory and Object Storage can be access like a disk/hard drive, next you can use the directory already mounted object storage for backup file using command like copy file or rsync command and cron job for doing a backup command automatically.
- rsync command
#rsync -aur --progress/sumber/ /tujuan/
Information :
a : copy file and add to destination directory
u : add file and skip if file already exist
r : recursive mode
--progress : view process while rsync running
/sumber/ : file or directory want to backup
/tujuan/ : destination directory
- Make a script file for doing backup automatically. For example, put it at directory/opt.
#cd /opt/
#nano script-backup.sh
- Add permission execute.
#chmod +x /opt/script-backup.sh
- Do testing
#sh script-backup.sh
If no error found can continued for make cron job
- create cron job
#crontab -e
Open cron job tab and choose familiar editor.
You can add command for run cron job and can be adjusted with your directory :
Information:
Digit 1 is for run cron job that it run every 1 AM / 01:00
/bin/sh for run script .sh
/opt/script-backup.sh for script file already we make