Update the test-network-kind README; removes the local docker registry; updated 'clean' instructions

Signed-off-by: Josh Kneubuhl <jkneubuh@us.ibm.com>

test-network-kind/README.md

@@ -2,37 +2,35 @@
 # Kubernetes Test Network 
 
 This directory includes a set of kubernetes deployment manifests, scripts, and configuration files suitable 
-for running the Hyperledger Fabric test network in a local KIND (Kubernetes in Docker) cluster.
+for running the Hyperledger Fabric test network on a local [KIND](https://kind.sigs.k8s.io/docs/user/quick-start/#installation) 
+cluster.
 
 This is currently an experimental branch.  No attempt has been made to optimize or streamline the actual 
 deployment to kubernetes - no helm charts, operators, kustomization overlays, etc. are involved at this 
-early genesis.  This is merely a set of kube manifests suitable for replicating the test network on a local 
+early genesis.  This is merely a set of kube manifests suitable for replicating the test network on 
-KIND cluster.
+Kubernetes. 
 
 
 ## Areas for Improvement
 
-- [ ] Introduce `fabctl` as a bridge between objectives running locally and activities running remotely (`network.sh` equivalent.)
+- [ ] Introduce `fabctl` as a bridge between objectives running locally and activities running remotely (`network.sh` equivalent, e.g. see [fabric-hyper-kube](https://github.com/hyperledgendary/fabric-hyper-kube))
-- [ ] Provide simple scripts or CLI driver routines (e.g. `network.sh up` equivalent)
+- [ ] Provide simple scripts or CLI driver routines (e.g. `network.sh up` -> `kubectl apply ...`)
-- [ ] crypto-config --> Configure a CA
+- [ ] `cryptogen` -> Configure a CA
 - [ ] couchdb state database
 - [ ] KIND is only one path to a Kube.  Check that we are also in good shape with minikube, IBM Fyre, IKS, aws, OCP, azure, etc.
 - [ ] Use kustomize, ~helm~, operator, etc. etc. to properly integrate and install. 
-- [ ] The manifests directly pull 2.3.2 fabric images and have an imagePullPolicy: Always.  Find a better technique to pull :latest tag from docker hub, or `kind load docker-image ...`  
+- [ ] The manifests directly pull 2.3.2 fabric images and have an imagePullPolicy: Always.  Find a better technique to pull :latest tag from docker hub or the kind control plane. 
 - [ ] The fabric config files (2.3.2) are also hard-wired into the /config folder.  It would be nice if this project could use the fab release archive (or better - directly from git), and override the stanzas in core.yaml (e.g. externalBuilder)
-- [ ] Pick ONE technique for running binaries in k8s (kube jobs, tekton, argo, exec, controller, etc.) - See `fabctl` above
-- [ ] Pick ONE technique for copying local config files into the volume mounts (currently configmap+volume mount)
-- [ ] Pick ONE technique for managing `peer` CLI _connection profiles_ - see `fabctl` above
 - [ ] Publish [fabric-ccs-builder](https://github.com/hyperledgendary/fabric-ccs-builder) image to docker hub 
 - [ ] Publish [asset-transfer-basic](../asset-transfer-basic/chaincode-external) and external chaincode sample images to docker hub.
-- [ ] The peer deployments currently mount the chaincode application bundle into a volume at launch time.  This is wrong - chaincode bundles come AFTER the peers have been deployed. 
+- [ ] The peer deployments currently mount the chaincode application bundle into a volume at launch time.  This is wrong - chaincode bundles must come AFTER the peers have been deployed, and should not force a peer pod restart.
 - [ ] Pick out the CC_PACKAGE_ID from `peer chaincode install` and load into a configmap / k8s secret / env 
 - [ ] Configure multiple pvc - one per network node, rather than one shared volume for all network elements.
 - [ ] Configure the Fabric REST sample - needs attention in configuring connection profiles, pems, CAs, and signing keys.
 
 ## Prerequisites
 
-- [Docker](docker.io)
+- [Docker](https://www.docker.com)
 - [kubectl](https://kubernetes.io/docs/tasks/tools/) 
 - [KIND](https://kind.sigs.k8s.io/docs/user/quick-start/#installation)
 - [fabric-ccs-builder](#fabric-ccs-builder) docker image
@@ -42,7 +40,7 @@ KIND cluster.
 Smart contracts running on Kubernetes rely extensively on the [Chaincode as a Service](https://hyperledger-fabric.readthedocs.io/en/latest/cc_service.html)
 deployment pattern.  This test network uses the [fabric-ccs-builder](https://github.com/jkneubuh/fabric-ccs-builder/tree/feature/docker-bundle)
 image `release`, `build`, and `detect` binaries, copied into the peer pods via an init container at 
-deployment time.  Before starting the test network, build the ccs image locally:
+deployment time.  Before starting the test network, build the ccs image locally and push to the KIND control plane:
 
 ```shell
 git clone https://github.com/hyperledgendary/fabric-ccs-builder.git /tmp/fabric-ccs-builder
@@ -50,30 +48,35 @@ git clone https://github.com/hyperledgendary/fabric-ccs-builder.git /tmp/fabric-
 docker build -t hyperledgendary/fabric-ccs-builder /tmp/fabric-ccs-builder
 ```
 
-## Test Network (PROTO / SCRATCH)
+## Test Network
 
-### KIND
+### Kube
 
+Create a Kubernetes cluster and [load docker images](https://kind.sigs.k8s.io/docs/user/quick-start/#loading-an-image-into-your-cluster) into the KIND control plane.  
 ```shell
-bin/make-kind-with-reg.sh
+kind create cluster
 
 kind load docker-image hyperledgendary/fabric-ccs-builder
 ```
 
-### Config
+Create a dedicated namespace and persistent volume for the test-network:
-
 ```shell
 kubectl create -f kube/pv-fabric.yaml
 kubectl create -f kube/ns-test-network.yaml
 kubectl -n test-network create -f kube/pvc-fabric.yaml
+```
+
+### Network Config
+
+```shell
 kubectl -n test-network create configmap fabric-config --from-file=config/
 kubectl -n test-network create configmap chaincode-config --from-file=chaincode/
-kubectl -n test-network create -f kube/debug.yaml
 ```
 
 ### Channel Artifacts
 
 ```shell
+kubectl -n test-network create -f kube/debug.yaml
 kubectl -n test-network create -f kube/job-crypto-config.yaml
 kubectl -n test-network create -f kube/job-orderer-genesis.yaml
 kubectl -n test-network create -f kube/job-create-channel-config.yaml
@@ -175,14 +178,14 @@ exit
 ### Launch External Chaincode
 
 - [ ] Determine `CHAINCODE_ID` from install command and load as a config map / env entry in the cc deployment spec.
-- [ ] This project uses a docker registry running at `localhost:5000` to support rapid development cycles and avoid docker hub.
+- [ ] Use an [insecure docker registry](bin/make-kind-with-reg.sh) to build and deploy chaincode images without Docker hub or the kind control plane.
 
 ```shell
 docker build \
-  -t localhost:5000/hyperledger/asset-transfer-basic \
+  -t hyperledger/asset-transfer-basic \
   ../asset-transfer-basic/chaincode-external
   
-docker push localhost:5000/hyperledger/asset-transfer-basic
+kind load docker-image hyperledger/asset-transfer-basic
 ```
 
 ```shell
@@ -193,6 +196,7 @@ kubectl -n test-network apply -f kube/cc-asset-transfer-basic.yaml
 
 ```shell
 kubectl -n test-network exec deploy/org1-peer1 -i -t -- /bin/sh
+export FABRIC_LOGGING_SPEC=INFO
 export CORE_PEER_MSPCONFIGPATH=/var/hyperledger/fabric/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp
 export CC_PACKAGE_ID=basic_1.0:d730a5ce916e120f2a2509ee33527a0df68cadac678f5eb196737ad10ba42da9
 
@@ -231,30 +235,35 @@ peer chaincode \
   --tls \
   --cafile /var/hyperledger/fabric/crypto-config/ordererOrganizations/example.com/orderers/orderer1.example.com/tls/ca.crt \
 
+sleep 2
+
 peer chaincode \
   query \
   -C mychannel \
   -n basic \
   -c '{"Args":["ReadAsset","1"]}'
+
+exit
 ```
 
 ### Reset Network
-```shell
-kubectl -n test-network exec deploy/debug -t -- rm -rf /var/hyperledger/fabric
-kubectl delete namespace test-network 
-kubectl delete pv fabric
-```
 
-or 
+```shell 
+kubectl -n test-network delete deployment --all 
+kubectl -n test-network delete pod --all
+kubectl -n test-network delete service --all
+kubectl -n test-network delete configmap --all 
+kubectl -n test-network delete secret --all 
+kubectl -n test-network create -f kube/job-scrub-fabric-volume.yaml
+kubectl -n test-network wait --for=condition=complete --timeout=60s job/job-scrub-fabric-volume
+kubectl -n test-network delete job --all
+```
+[GOTO Config](#network-config)
+
+or ... 
 ```shell
 kind delete cluster
 ```
+[GOTO Kube](#kube)
 
 
-## REST Sample Application 
-
-TODO: 
-- fabric-rest-sample docker image, build and push to localhost:5000 registry 
-- construct fabric sample deployment descriptor (cert.pem, p_k, tlsCA, connection profile, etc.)
-- sample notes with pointers to crypto spec assets  
-- weft to synthesize connection profile

test-network-kind/kube/cc-asset-transfer-basic.yaml

@@ -20,8 +20,8 @@ spec:
     spec:
       containers:
       - name: main
-        image: localhost:5000/hyperledger/asset-transfer-basic
+        image: hyperledger/asset-transfer-basic
-        imagePullPolicy: Always
+        imagePullPolicy: IfNotPresent
         env:
         - name: CHAINCODE_SERVER_ADDRESS
           value: 0.0.0.0:9999

test-network-kind/kube/job-scrub-fabric-volume.yaml

@@ -0,0 +1,32 @@
+#
+# Copyright IBM Corp. All Rights Reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+#
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: job-scrub-fabric-volume
+spec:
+  backoffLimit: 0
+  completions: 1
+  template:
+    metadata:
+      name: scrub-fabric-volume
+    spec:
+      restartPolicy: "Never"
+      containers:
+        - name: main
+          image: alpine
+          command:
+            - sh
+            - -c
+            - "rm -rvf /var/hyperledger/fabric/*"
+          volumeMounts:
+            - name: fabric-volume
+              mountPath: /var/hyperledger/fabric
+      volumes:
+        - name: fabric-volume
+          persistentVolumeClaim:
+            claimName: fabric