fabric-samples/test-network-kind

Kubernetes Test Network

This directory includes a set of kubernetes deployment manifests, scripts, and configuration files suitable for running the Hyperledger Fabric test network on a local KIND cluster.

This is currently an experimental branch. No attempt has been made to optimize or streamline the actual deployment to kubernetes - no helm charts, operators, kustomization overlays, etc. are involved at this early genesis. This is merely a set of kube manifests suitable for replicating the test network on Kubernetes.

Areas for Improvement

• Introduce fabctl as a bridge between objectives running locally and activities running remotely (network.sh equivalent, e.g. see fabric-hyper-kube)
• Provide simple scripts or CLI driver routines (e.g. network.sh up -> kubectl apply ...)
• cryptogen -> Configure a CA
• couchdb state database
• KIND is only one path to a Kube. Check that we are also in good shape with minikube, IBM Fyre, IKS, aws, OCP, azure, etc.
• Use kustomize, helm, operator, etc. etc. to properly integrate and install.
• The manifests directly pull 2.3.2 fabric images and have an imagePullPolicy: Always. Find a better technique to pull :latest tag from docker hub or the kind control plane.
• The fabric config files (2.3.2) are also hard-wired into the /config folder. It would be nice if this project could use the fab release archive (or better - directly from git), and override the stanzas in core.yaml (e.g. externalBuilder)
• Publish fabric-ccs-builder image to docker hub
• Publish asset-transfer-basic and external chaincode sample images to docker hub.
• The peer deployments currently mount the chaincode application bundle into a volume at launch time. This is wrong - chaincode bundles must come AFTER the peers have been deployed, and should not force a peer pod restart.
• Pick out the CC_PACKAGE_ID from peer chaincode install and load into a configmap / k8s secret / env
• Configure multiple pvc - one per network node, rather than one shared volume for all network elements.
• Configure the Fabric REST sample - needs attention in configuring connection profiles, pems, CAs, and signing keys.

Prerequisites

• Docker
• kubectl
• KIND
• fabric-ccs-builder docker image

Fabric CCS Builder

Smart contracts running on Kubernetes rely extensively on the Chaincode as a Service deployment pattern. This test network uses the fabric-ccs-builder image release, build, and detect binaries, copied into the peer pods via an init container at deployment time. Before starting the test network, build the ccs image locally and push to the KIND control plane:

git clone https://github.com/hyperledgendary/fabric-ccs-builder.git /tmp/fabric-ccs-builder

docker build -t hyperledgendary/fabric-ccs-builder /tmp/fabric-ccs-builder

Test Network

Kube

Create a Kubernetes cluster and load docker images into the KIND control plane.

kind create cluster

kind load docker-image hyperledgendary/fabric-ccs-builder

Create a dedicated namespace and persistent volume for the test-network:

kubectl create -f kube/pv-fabric.yaml
kubectl create -f kube/ns-test-network.yaml
kubectl -n test-network create -f kube/pvc-fabric.yaml

Network Config

kubectl -n test-network create configmap fabric-config --from-file=config/
kubectl -n test-network create configmap chaincode-config --from-file=chaincode/

Channel Artifacts

kubectl -n test-network create -f kube/debug.yaml
kubectl -n test-network create -f kube/job-crypto-config.yaml
kubectl -n test-network create -f kube/job-orderer-genesis.yaml
kubectl -n test-network create -f kube/job-create-channel-config.yaml
kubectl -n test-network create -f kube/job-update-org1-anchor-peers.yaml
kubectl -n test-network create -f kube/job-update-org2-anchor-peers.yaml

(Wait for these jobs to complete. It can take a few seconds for images to be pulled from docker hub.)

Orderers

kubectl -n test-network apply -f kube/orderer1.yaml
kubectl -n test-network apply -f kube/orderer2.yaml
kubectl -n test-network apply -f kube/orderer3.yaml

Peers

kubectl -n test-network apply -f kube/org1-peer1.yaml
kubectl -n test-network apply -f kube/org1-peer2.yaml
kubectl -n test-network apply -f kube/org2-peer1.yaml
kubectl -n test-network apply -f kube/org2-peer2.yaml

Create mychannel

kubectl -n test-network exec deploy/org1-peer1 -i -t -- /bin/sh

export CORE_PEER_MSPCONFIGPATH=/var/hyperledger/fabric/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp

peer channel \
  create \
  -c mychannel \
  -o orderer1:6050 \
  -f /var/hyperledger/fabric/channel-artifacts/mychannel.tx \
  --outputBlock /var/hyperledger/fabric/channel-artifacts/mychannel.block \
  --tls \
  --cafile /var/hyperledger/fabric/crypto-config/ordererOrganizations/example.com/orderers/orderer1.example.com/tls/ca.crt 

peer channel \
  update \
  -o orderer1:6050 \
  -c mychannel \
  -f /var/hyperledger/fabric/channel-artifacts/Org1MSPanchors.tx \
  --tls \
  --cafile /var/hyperledger/fabric/crypto-config/ordererOrganizations/example.com/orderers/orderer1.example.com/tls/ca.crt

exit

Join Peers

kubectl \
  -n test-network \
  exec deploy/org1-peer1 \
  -i -t -- \
  /bin/sh -c 'CORE_PEER_MSPCONFIGPATH=/var/hyperledger/fabric/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp peer channel join -b /var/hyperledger/fabric/channel-artifacts/mychannel.block'

kubectl \
  -n test-network \
  exec deploy/org1-peer2 \
  -i -t -- \
  /bin/sh -c 'CORE_PEER_MSPCONFIGPATH=/var/hyperledger/fabric/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp peer channel join -b /var/hyperledger/fabric/channel-artifacts/mychannel.block'

kubectl \
  -n test-network \
  exec deploy/org2-peer1 \
  -i -t -- \
  /bin/sh -c 'CORE_PEER_MSPCONFIGPATH=/var/hyperledger/fabric/crypto-config/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp peer channel join -b /var/hyperledger/fabric/channel-artifacts/mychannel.block'

kubectl \
  -n test-network \
  exec deploy/org2-peer2 \
  -i -t -- \
  /bin/sh -c 'CORE_PEER_MSPCONFIGPATH=/var/hyperledger/fabric/crypto-config/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp peer channel join -b /var/hyperledger/fabric/channel-artifacts/mychannel.block'

Chaincode

Install

kubectl -n test-network exec deploy/org1-peer1 -i -t -- /bin/sh
export CORE_PEER_MSPCONFIGPATH=/var/hyperledger/fabric/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp

peer lifecycle \
  chaincode install \
  /var/hyperledger/fabric/chaincode/asset-transfer-basic.tgz 

exit

Launch External Chaincode

• Determine CHAINCODE_ID from install command and load as a config map / env entry in the cc deployment spec.
• Use an insecure docker registry to build and deploy chaincode images without Docker hub or the kind control plane.

docker build \
  -t hyperledger/asset-transfer-basic \
  ../asset-transfer-basic/chaincode-external
  
kind load docker-image hyperledger/asset-transfer-basic

kubectl -n test-network apply -f kube/cc-asset-transfer-basic.yaml 

Approve and Commit

kubectl -n test-network exec deploy/org1-peer1 -i -t -- /bin/sh
export FABRIC_LOGGING_SPEC=INFO
export CORE_PEER_MSPCONFIGPATH=/var/hyperledger/fabric/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp
export CC_PACKAGE_ID=basic_1.0:d730a5ce916e120f2a2509ee33527a0df68cadac678f5eb196737ad10ba42da9

peer lifecycle \
  chaincode approveformyorg \
  -o orderer1:6050 \
  --channelID mychannel \
  --name basic \
  --version 1 \
  --package-id $CC_PACKAGE_ID \
  --sequence 1 \
  --tls \
  --cafile /var/hyperledger/fabric/crypto-config/ordererOrganizations/example.com/orderers/orderer1.example.com/msp/tlscacerts/tlsca.example.com-cert.pem 

peer lifecycle \
  chaincode commit \
  -o orderer1:6050 \
  --channelID mychannel \
  --name basic \
  --version 1 \
  --sequence 1 \
  --tls \
  --cafile /var/hyperledger/fabric/crypto-config/ordererOrganizations/example.com/orderers/orderer1.example.com/msp/tlscacerts/tlsca.example.com-cert.pem 

Query

(run on org1-peer1)

peer chaincode \
  invoke \
  -o orderer1:6050 \
  -C mychannel \
  -n basic \
  -c '{"Args":["CreateAsset","1","blue","35","tom","1000"]}' \
  --tls \
  --cafile /var/hyperledger/fabric/crypto-config/ordererOrganizations/example.com/orderers/orderer1.example.com/tls/ca.crt \

sleep 2

peer chaincode \
  query \
  -C mychannel \
  -n basic \
  -c '{"Args":["ReadAsset","1"]}'

exit

Reset Network

kubectl -n test-network delete deployment --all 
kubectl -n test-network delete pod --all
kubectl -n test-network delete service --all
kubectl -n test-network delete configmap --all 
kubectl -n test-network delete secret --all 
kubectl -n test-network create -f kube/job-scrub-fabric-volume.yaml
kubectl -n test-network wait --for=condition=complete --timeout=60s job/job-scrub-fabric-volume
kubectl -n test-network delete job --all

GOTO Config

or ...

kind delete cluster

GOTO Kube