OpenSDS-Integration-with-Kubernetes-Flexvolume

Prerequisite

ubuntu

  • Version information

    root@proxy:~# cat /etc/issue
Ubuntu 16.04.2 LTS \n \l

docker

  • Version information

    root@proxy:~# docker version
Client:
 Version:      1.12.6
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   78d1802
 Built:        Tue Jan 31 23:35:14 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.6
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   78d1802
 Built:        Tue Jan 31 23:35:14 2017
 OS/Arch:      linux/amd64

golang

  • Version information

    root@proxy:~# go version
go version go1.9.2 linux/amd64

  • You can install golang by executing commands blow:

    wget https://storage.googleapis.com/golang/go1.9.2.linux-amd64.tar.gz
tar -C /usr/local -xzf go1.9.2.linux-amd64.tar.gz
export PATH=$PATH:/usr/local/go/bin
export GOPATH=$HOME/gopath

kubernetes local cluster

  • Version information

    root@proxy:~# kubectl version
Client Version: version.Info{Major:"1", Minor:"9+", GitVersion:"v1.9.0-beta.0-dirty", GitCommit:"a0fb3baa71f1559fd42d1acd9cbdd8a55ab4dfff", GitTreeState:"dirty", BuildDate:"2017-12-13T09:22:09Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"9+", GitVersion:"v1.9.0-beta.0-dirty", GitCommit:"a0fb3baa71f1559fd42d1acd9cbdd8a55ab4dfff", GitTreeState:"dirty", BuildDate:"2017-12-13T09:22:09Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}

  • You can startup the k8s local cluster by executing commands blow:

    cd $HOME
git clone https://github.com/kubernetes/kubernetes.git
cd $HOME/kubernetes
git checkout v1.9.0
make
echo alias kubectl='$HOME/kubernetes/cluster/kubectl.sh' >> /etc/profile
RUNTIME_CONFIG=settings.k8s.io/v1alpha1=true AUTHORIZATION_MODE=Node,RBAC hack/local-up-cluster.sh -O

NOTE:

Due to opensds using etcd as the database which is same with kubernetes so you should startup kubernetes firstly.

opensds local cluster

Testing steps

  • Load some ENVs which is setted before.

    source /etc/profile

  • Download nbp source code.

    using git clone

    git clone https://github.com/opensds/nbp.git  $GOPATH/src/github.com/opensds/nbp

    or using go get

    go get -v  github.com/opensds/nbp/...

  • Build the FlexVolume.

    cd $GOPATH/src/github.com/opensds/nbp/flexvolume
go build -o opensds ./cmd/flex-plugin/

    FlexVolume plugin binary is on the current directory.

  • Copy the OpenSDS FlexVolume binary file to k8s kubelet volume-plugin-dir.if you don't specify the volume-plugin-dir, you can execute commands blow:

    mkdir -p /usr/libexec/kubernetes/kubelet-plugins/volume/exec/opensds.io~opensds/
cp $GOPATH/src/github.com/opensds/nbp/flexvolume/opensds /usr/libexec/kubernetes/kubelet-plugins/volume/exec/opensds.io~opensds/

    NOTE:

    OpenSDS FlexVolume will get the opensds api endpoint from the environment variable `OPENSDS_ENDPOINT`, if you don't specify it, the FlexVolume will use the default vaule: `http://127.0.0.1:50040`. if you want to specify the `OPENSDS_ENDPOINT` executing command `export OPENSDS_ENDPOINT=http://ip:50040` and restart the k8s local cluster.

  • Build the provisioner docker image.

    cd $GOPATH/src/github.com/opensds/nbp/opensds-provisioner
make container

  • Create service account, role and bind them.

    cd $GOPATH/src/github.com/opensds/nbp/opensds-provisioner/examples
kubectl create -f serviceaccount.yaml
kubectl create -f clusterrole.yaml
kubectl create -f clusterrolebinding.yaml

  • Change the opensds endpoint IP in pod-provisioner.yamlThe IP (192.168.56.106) should be replaced with the OpenSDS osdslet actual endpoint IP.

    kind: Pod
apiVersion: v1
metadata:
  name: opensds-provisioner
spec:
  serviceAccount: opensds-provisioner
  containers:
    - name: opensds-provisioner
      image: opensdsio/opensds-provisioner
      securityContext:
      args:
        - "-endpoint=http://192.168.56.106:50040" # should be replaced
      imagePullPolicy: "IfNotPresent"

  • Create provisioner pod.

    kubectl create -f pod-provisioner.yaml

    Execute kubectl get pod to check if the opensds-provisioner is ok.

    root@nbp:~/go/src/github.com/opensds/nbp/opensds-provisioner/examples# kubectl get pod
NAME                  READY     STATUS    RESTARTS   AGE
opensds-provisioner   1/1       Running   0          42m

  • You can use the following cammands to test the OpenSDS FlexVolume and Proversioner functions.

    Create storage class.

    kubectl create -f sc.yaml              # Create StorageClass

    Execute kubectl get sc to check if the storage class is ok.

    root@nbp:~/go/src/github.com/opensds/nbp/opensds-provisioner/examples# kubectl get sc
NAME                 PROVISIONER               AGE
opensds              opensds/nbp-provisioner   46m
standard (default)   kubernetes.io/host-path   49m

    Create PVC.

    kubectl create -f pvc.yaml             # Create PVC

    Execute kubectl get pvc to check if the pvc is ok.

    root@nbp:~/go/src/github.com/opensds/nbp/opensds-provisioner/examples# kubectl get pvc
NAME          STATUS    VOLUME                                 CAPACITY   ACCESS MODES   STORAGECLASS   AGE
opensds-pvc   Bound     731da41e-c9ee-4180-8fb3-d1f6c7f65378   1Gi        RWO            opensds        48m

    Create busybox pod.

    kubectl create -f pod-application.yaml # Create busybox pod and mount the block storage.

    Execute kubectl get pod to check if the busybox pod is ok.

    root@nbp:~/go/src/github.com/opensds/nbp/opensds-provisioner/examples# kubectl get pod
NAME                  READY     STATUS    RESTARTS   AGE
busy-pod              1/1       Running   0          49m
opensds-provisioner   1/1       Running   0          50m

    Execute the findmnt|grep opensds to confirm whether the volume has been provided. If there is some thing that goes wrong, you can check the log files in directory /var/log/opensds.

Clean up steps

kubectl delete -f pod-application.yaml
kubectl delete -f pvc.yaml
kubectl delete -f sc.yaml

kubectl delete -f pod-provisioner.yaml
kubectl delete -f clusterrolebinding.yaml
kubectl delete -f clusterrole.yaml
kubectl delete -f serviceaccount.yaml