diff --git a/test-network-kind/README.md b/test-network-kind/README.md index 7b1de71e..260c0f41 100644 --- a/test-network-kind/README.md +++ b/test-network-kind/README.md @@ -2,37 +2,35 @@ # Kubernetes Test Network This directory includes a set of kubernetes deployment manifests, scripts, and configuration files suitable -for running the Hyperledger Fabric test network in a local KIND (Kubernetes in Docker) cluster. +for running the Hyperledger Fabric test network on a local [KIND](https://kind.sigs.k8s.io/docs/user/quick-start/#installation) +cluster. This is currently an experimental branch. No attempt has been made to optimize or streamline the actual deployment to kubernetes - no helm charts, operators, kustomization overlays, etc. are involved at this -early genesis. This is merely a set of kube manifests suitable for replicating the test network on a local -KIND cluster. +early genesis. This is merely a set of kube manifests suitable for replicating the test network on +Kubernetes. ## Areas for Improvement -- [ ] Introduce `fabctl` as a bridge between objectives running locally and activities running remotely (`network.sh` equivalent.) -- [ ] Provide simple scripts or CLI driver routines (e.g. `network.sh up` equivalent) -- [ ] crypto-config --> Configure a CA +- [ ] Introduce `fabctl` as a bridge between objectives running locally and activities running remotely (`network.sh` equivalent, e.g. see [fabric-hyper-kube](https://github.com/hyperledgendary/fabric-hyper-kube)) +- [ ] Provide simple scripts or CLI driver routines (e.g. `network.sh up` -> `kubectl apply ...`) +- [ ] `cryptogen` -> Configure a CA - [ ] couchdb state database - [ ] KIND is only one path to a Kube. Check that we are also in good shape with minikube, IBM Fyre, IKS, aws, OCP, azure, etc. - [ ] Use kustomize, ~helm~, operator, etc. etc. to properly integrate and install. -- [ ] The manifests directly pull 2.3.2 fabric images and have an imagePullPolicy: Always. Find a better technique to pull :latest tag from docker hub, or `kind load docker-image ...` +- [ ] The manifests directly pull 2.3.2 fabric images and have an imagePullPolicy: Always. Find a better technique to pull :latest tag from docker hub or the kind control plane. - [ ] The fabric config files (2.3.2) are also hard-wired into the /config folder. It would be nice if this project could use the fab release archive (or better - directly from git), and override the stanzas in core.yaml (e.g. externalBuilder) -- [ ] Pick ONE technique for running binaries in k8s (kube jobs, tekton, argo, exec, controller, etc.) - See `fabctl` above -- [ ] Pick ONE technique for copying local config files into the volume mounts (currently configmap+volume mount) -- [ ] Pick ONE technique for managing `peer` CLI _connection profiles_ - see `fabctl` above - [ ] Publish [fabric-ccs-builder](https://github.com/hyperledgendary/fabric-ccs-builder) image to docker hub - [ ] Publish [asset-transfer-basic](../asset-transfer-basic/chaincode-external) and external chaincode sample images to docker hub. -- [ ] The peer deployments currently mount the chaincode application bundle into a volume at launch time. This is wrong - chaincode bundles come AFTER the peers have been deployed. +- [ ] The peer deployments currently mount the chaincode application bundle into a volume at launch time. This is wrong - chaincode bundles must come AFTER the peers have been deployed, and should not force a peer pod restart. - [ ] Pick out the CC_PACKAGE_ID from `peer chaincode install` and load into a configmap / k8s secret / env - [ ] Configure multiple pvc - one per network node, rather than one shared volume for all network elements. - [ ] Configure the Fabric REST sample - needs attention in configuring connection profiles, pems, CAs, and signing keys. ## Prerequisites -- [Docker](docker.io) +- [Docker](https://www.docker.com) - [kubectl](https://kubernetes.io/docs/tasks/tools/) - [KIND](https://kind.sigs.k8s.io/docs/user/quick-start/#installation) - [fabric-ccs-builder](#fabric-ccs-builder) docker image @@ -42,7 +40,7 @@ KIND cluster. Smart contracts running on Kubernetes rely extensively on the [Chaincode as a Service](https://hyperledger-fabric.readthedocs.io/en/latest/cc_service.html) deployment pattern. This test network uses the [fabric-ccs-builder](https://github.com/jkneubuh/fabric-ccs-builder/tree/feature/docker-bundle) image `release`, `build`, and `detect` binaries, copied into the peer pods via an init container at -deployment time. Before starting the test network, build the ccs image locally: +deployment time. Before starting the test network, build the ccs image locally and push to the KIND control plane: ```shell git clone https://github.com/hyperledgendary/fabric-ccs-builder.git /tmp/fabric-ccs-builder @@ -50,30 +48,35 @@ git clone https://github.com/hyperledgendary/fabric-ccs-builder.git /tmp/fabric- docker build -t hyperledgendary/fabric-ccs-builder /tmp/fabric-ccs-builder ``` -## Test Network (PROTO / SCRATCH) +## Test Network -### KIND +### Kube +Create a Kubernetes cluster and [load docker images](https://kind.sigs.k8s.io/docs/user/quick-start/#loading-an-image-into-your-cluster) into the KIND control plane. ```shell -bin/make-kind-with-reg.sh +kind create cluster kind load docker-image hyperledgendary/fabric-ccs-builder ``` -### Config - +Create a dedicated namespace and persistent volume for the test-network: ```shell kubectl create -f kube/pv-fabric.yaml kubectl create -f kube/ns-test-network.yaml kubectl -n test-network create -f kube/pvc-fabric.yaml +``` + +### Network Config + +```shell kubectl -n test-network create configmap fabric-config --from-file=config/ kubectl -n test-network create configmap chaincode-config --from-file=chaincode/ -kubectl -n test-network create -f kube/debug.yaml ``` ### Channel Artifacts ```shell +kubectl -n test-network create -f kube/debug.yaml kubectl -n test-network create -f kube/job-crypto-config.yaml kubectl -n test-network create -f kube/job-orderer-genesis.yaml kubectl -n test-network create -f kube/job-create-channel-config.yaml @@ -175,14 +178,14 @@ exit ### Launch External Chaincode - [ ] Determine `CHAINCODE_ID` from install command and load as a config map / env entry in the cc deployment spec. -- [ ] This project uses a docker registry running at `localhost:5000` to support rapid development cycles and avoid docker hub. +- [ ] Use an [insecure docker registry](bin/make-kind-with-reg.sh) to build and deploy chaincode images without Docker hub or the kind control plane. ```shell docker build \ - -t localhost:5000/hyperledger/asset-transfer-basic \ + -t hyperledger/asset-transfer-basic \ ../asset-transfer-basic/chaincode-external -docker push localhost:5000/hyperledger/asset-transfer-basic +kind load docker-image hyperledger/asset-transfer-basic ``` ```shell @@ -193,6 +196,7 @@ kubectl -n test-network apply -f kube/cc-asset-transfer-basic.yaml ```shell kubectl -n test-network exec deploy/org1-peer1 -i -t -- /bin/sh +export FABRIC_LOGGING_SPEC=INFO export CORE_PEER_MSPCONFIGPATH=/var/hyperledger/fabric/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp export CC_PACKAGE_ID=basic_1.0:d730a5ce916e120f2a2509ee33527a0df68cadac678f5eb196737ad10ba42da9 @@ -231,30 +235,35 @@ peer chaincode \ --tls \ --cafile /var/hyperledger/fabric/crypto-config/ordererOrganizations/example.com/orderers/orderer1.example.com/tls/ca.crt \ +sleep 2 + peer chaincode \ query \ -C mychannel \ -n basic \ -c '{"Args":["ReadAsset","1"]}' + +exit ``` ### Reset Network -```shell -kubectl -n test-network exec deploy/debug -t -- rm -rf /var/hyperledger/fabric -kubectl delete namespace test-network -kubectl delete pv fabric -``` -or +```shell +kubectl -n test-network delete deployment --all +kubectl -n test-network delete pod --all +kubectl -n test-network delete service --all +kubectl -n test-network delete configmap --all +kubectl -n test-network delete secret --all +kubectl -n test-network create -f kube/job-scrub-fabric-volume.yaml +kubectl -n test-network wait --for=condition=complete --timeout=60s job/job-scrub-fabric-volume +kubectl -n test-network delete job --all +``` +[GOTO Config](#network-config) + +or ... ```shell kind delete cluster ``` +[GOTO Kube](#kube) -## REST Sample Application - -TODO: -- fabric-rest-sample docker image, build and push to localhost:5000 registry -- construct fabric sample deployment descriptor (cert.pem, p_k, tlsCA, connection profile, etc.) -- sample notes with pointers to crypto spec assets -- weft to synthesize connection profile diff --git a/test-network-kind/kube/cc-asset-transfer-basic.yaml b/test-network-kind/kube/cc-asset-transfer-basic.yaml index 77b2b52e..8760949b 100644 --- a/test-network-kind/kube/cc-asset-transfer-basic.yaml +++ b/test-network-kind/kube/cc-asset-transfer-basic.yaml @@ -20,8 +20,8 @@ spec: spec: containers: - name: main - image: localhost:5000/hyperledger/asset-transfer-basic - imagePullPolicy: Always + image: hyperledger/asset-transfer-basic + imagePullPolicy: IfNotPresent env: - name: CHAINCODE_SERVER_ADDRESS value: 0.0.0.0:9999 diff --git a/test-network-kind/kube/job-scrub-fabric-volume.yaml b/test-network-kind/kube/job-scrub-fabric-volume.yaml new file mode 100644 index 00000000..1ad56bd0 --- /dev/null +++ b/test-network-kind/kube/job-scrub-fabric-volume.yaml @@ -0,0 +1,32 @@ +# +# Copyright IBM Corp. All Rights Reserved. +# +# SPDX-License-Identifier: Apache-2.0 +# +--- +apiVersion: batch/v1 +kind: Job +metadata: + name: job-scrub-fabric-volume +spec: + backoffLimit: 0 + completions: 1 + template: + metadata: + name: scrub-fabric-volume + spec: + restartPolicy: "Never" + containers: + - name: main + image: alpine + command: + - sh + - -c + - "rm -rvf /var/hyperledger/fabric/*" + volumeMounts: + - name: fabric-volume + mountPath: /var/hyperledger/fabric + volumes: + - name: fabric-volume + persistentVolumeClaim: + claimName: fabric \ No newline at end of file