Update the test-network-kind README; removes the local docker registry; updated 'clean' instructions

Signed-off-by: Josh Kneubuhl <jkneubuh@us.ibm.com>
This commit is contained in:
Josh Kneubuhl 2021-09-07 13:44:42 -04:00
parent 608be95aab
commit 099fd4dea9
3 changed files with 78 additions and 37 deletions

View file

@ -2,37 +2,35 @@
# Kubernetes Test Network
This directory includes a set of kubernetes deployment manifests, scripts, and configuration files suitable
for running the Hyperledger Fabric test network in a local KIND (Kubernetes in Docker) cluster.
for running the Hyperledger Fabric test network on a local [KIND](https://kind.sigs.k8s.io/docs/user/quick-start/#installation)
cluster.
This is currently an experimental branch. No attempt has been made to optimize or streamline the actual
deployment to kubernetes - no helm charts, operators, kustomization overlays, etc. are involved at this
early genesis. This is merely a set of kube manifests suitable for replicating the test network on a local
KIND cluster.
early genesis. This is merely a set of kube manifests suitable for replicating the test network on
Kubernetes.
## Areas for Improvement
- [ ] Introduce `fabctl` as a bridge between objectives running locally and activities running remotely (`network.sh` equivalent.)
- [ ] Provide simple scripts or CLI driver routines (e.g. `network.sh up` equivalent)
- [ ] crypto-config --> Configure a CA
- [ ] Introduce `fabctl` as a bridge between objectives running locally and activities running remotely (`network.sh` equivalent, e.g. see [fabric-hyper-kube](https://github.com/hyperledgendary/fabric-hyper-kube))
- [ ] Provide simple scripts or CLI driver routines (e.g. `network.sh up` -> `kubectl apply ...`)
- [ ] `cryptogen` -> Configure a CA
- [ ] couchdb state database
- [ ] KIND is only one path to a Kube. Check that we are also in good shape with minikube, IBM Fyre, IKS, aws, OCP, azure, etc.
- [ ] Use kustomize, ~helm~, operator, etc. etc. to properly integrate and install.
- [ ] The manifests directly pull 2.3.2 fabric images and have an imagePullPolicy: Always. Find a better technique to pull :latest tag from docker hub, or `kind load docker-image ...`
- [ ] The manifests directly pull 2.3.2 fabric images and have an imagePullPolicy: Always. Find a better technique to pull :latest tag from docker hub or the kind control plane.
- [ ] The fabric config files (2.3.2) are also hard-wired into the /config folder. It would be nice if this project could use the fab release archive (or better - directly from git), and override the stanzas in core.yaml (e.g. externalBuilder)
- [ ] Pick ONE technique for running binaries in k8s (kube jobs, tekton, argo, exec, controller, etc.) - See `fabctl` above
- [ ] Pick ONE technique for copying local config files into the volume mounts (currently configmap+volume mount)
- [ ] Pick ONE technique for managing `peer` CLI _connection profiles_ - see `fabctl` above
- [ ] Publish [fabric-ccs-builder](https://github.com/hyperledgendary/fabric-ccs-builder) image to docker hub
- [ ] Publish [asset-transfer-basic](../asset-transfer-basic/chaincode-external) and external chaincode sample images to docker hub.
- [ ] The peer deployments currently mount the chaincode application bundle into a volume at launch time. This is wrong - chaincode bundles come AFTER the peers have been deployed.
- [ ] The peer deployments currently mount the chaincode application bundle into a volume at launch time. This is wrong - chaincode bundles must come AFTER the peers have been deployed, and should not force a peer pod restart.
- [ ] Pick out the CC_PACKAGE_ID from `peer chaincode install` and load into a configmap / k8s secret / env
- [ ] Configure multiple pvc - one per network node, rather than one shared volume for all network elements.
- [ ] Configure the Fabric REST sample - needs attention in configuring connection profiles, pems, CAs, and signing keys.
## Prerequisites
- [Docker](docker.io)
- [Docker](https://www.docker.com)
- [kubectl](https://kubernetes.io/docs/tasks/tools/)
- [KIND](https://kind.sigs.k8s.io/docs/user/quick-start/#installation)
- [fabric-ccs-builder](#fabric-ccs-builder) docker image
@ -42,7 +40,7 @@ KIND cluster.
Smart contracts running on Kubernetes rely extensively on the [Chaincode as a Service](https://hyperledger-fabric.readthedocs.io/en/latest/cc_service.html)
deployment pattern. This test network uses the [fabric-ccs-builder](https://github.com/jkneubuh/fabric-ccs-builder/tree/feature/docker-bundle)
image `release`, `build`, and `detect` binaries, copied into the peer pods via an init container at
deployment time. Before starting the test network, build the ccs image locally:
deployment time. Before starting the test network, build the ccs image locally and push to the KIND control plane:
```shell
git clone https://github.com/hyperledgendary/fabric-ccs-builder.git /tmp/fabric-ccs-builder
@ -50,30 +48,35 @@ git clone https://github.com/hyperledgendary/fabric-ccs-builder.git /tmp/fabric-
docker build -t hyperledgendary/fabric-ccs-builder /tmp/fabric-ccs-builder
```
## Test Network (PROTO / SCRATCH)
## Test Network
### KIND
### Kube
Create a Kubernetes cluster and [load docker images](https://kind.sigs.k8s.io/docs/user/quick-start/#loading-an-image-into-your-cluster) into the KIND control plane.
```shell
bin/make-kind-with-reg.sh
kind create cluster
kind load docker-image hyperledgendary/fabric-ccs-builder
```
### Config
Create a dedicated namespace and persistent volume for the test-network:
```shell
kubectl create -f kube/pv-fabric.yaml
kubectl create -f kube/ns-test-network.yaml
kubectl -n test-network create -f kube/pvc-fabric.yaml
```
### Network Config
```shell
kubectl -n test-network create configmap fabric-config --from-file=config/
kubectl -n test-network create configmap chaincode-config --from-file=chaincode/
kubectl -n test-network create -f kube/debug.yaml
```
### Channel Artifacts
```shell
kubectl -n test-network create -f kube/debug.yaml
kubectl -n test-network create -f kube/job-crypto-config.yaml
kubectl -n test-network create -f kube/job-orderer-genesis.yaml
kubectl -n test-network create -f kube/job-create-channel-config.yaml
@ -175,14 +178,14 @@ exit
### Launch External Chaincode
- [ ] Determine `CHAINCODE_ID` from install command and load as a config map / env entry in the cc deployment spec.
- [ ] This project uses a docker registry running at `localhost:5000` to support rapid development cycles and avoid docker hub.
- [ ] Use an [insecure docker registry](bin/make-kind-with-reg.sh) to build and deploy chaincode images without Docker hub or the kind control plane.
```shell
docker build \
-t localhost:5000/hyperledger/asset-transfer-basic \
-t hyperledger/asset-transfer-basic \
../asset-transfer-basic/chaincode-external
docker push localhost:5000/hyperledger/asset-transfer-basic
kind load docker-image hyperledger/asset-transfer-basic
```
```shell
@ -193,6 +196,7 @@ kubectl -n test-network apply -f kube/cc-asset-transfer-basic.yaml
```shell
kubectl -n test-network exec deploy/org1-peer1 -i -t -- /bin/sh
export FABRIC_LOGGING_SPEC=INFO
export CORE_PEER_MSPCONFIGPATH=/var/hyperledger/fabric/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp
export CC_PACKAGE_ID=basic_1.0:d730a5ce916e120f2a2509ee33527a0df68cadac678f5eb196737ad10ba42da9
@ -231,30 +235,35 @@ peer chaincode \
--tls \
--cafile /var/hyperledger/fabric/crypto-config/ordererOrganizations/example.com/orderers/orderer1.example.com/tls/ca.crt \
sleep 2
peer chaincode \
query \
-C mychannel \
-n basic \
-c '{"Args":["ReadAsset","1"]}'
exit
```
### Reset Network
```shell
kubectl -n test-network exec deploy/debug -t -- rm -rf /var/hyperledger/fabric
kubectl delete namespace test-network
kubectl delete pv fabric
```
or
```shell
kubectl -n test-network delete deployment --all
kubectl -n test-network delete pod --all
kubectl -n test-network delete service --all
kubectl -n test-network delete configmap --all
kubectl -n test-network delete secret --all
kubectl -n test-network create -f kube/job-scrub-fabric-volume.yaml
kubectl -n test-network wait --for=condition=complete --timeout=60s job/job-scrub-fabric-volume
kubectl -n test-network delete job --all
```
[GOTO Config](#network-config)
or ...
```shell
kind delete cluster
```
[GOTO Kube](#kube)
## REST Sample Application
TODO:
- fabric-rest-sample docker image, build and push to localhost:5000 registry
- construct fabric sample deployment descriptor (cert.pem, p_k, tlsCA, connection profile, etc.)
- sample notes with pointers to crypto spec assets
- weft to synthesize connection profile

View file

@ -20,8 +20,8 @@ spec:
spec:
containers:
- name: main
image: localhost:5000/hyperledger/asset-transfer-basic
imagePullPolicy: Always
image: hyperledger/asset-transfer-basic
imagePullPolicy: IfNotPresent
env:
- name: CHAINCODE_SERVER_ADDRESS
value: 0.0.0.0:9999

View file

@ -0,0 +1,32 @@
#
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
---
apiVersion: batch/v1
kind: Job
metadata:
name: job-scrub-fabric-volume
spec:
backoffLimit: 0
completions: 1
template:
metadata:
name: scrub-fabric-volume
spec:
restartPolicy: "Never"
containers:
- name: main
image: alpine
command:
- sh
- -c
- "rm -rvf /var/hyperledger/fabric/*"
volumeMounts:
- name: fabric-volume
mountPath: /var/hyperledger/fabric
volumes:
- name: fabric-volume
persistentVolumeClaim:
claimName: fabric