Updates the README and documentation guide for the Kube test network's usage of Nginx ingress (#697)

* Update test-network-k8s README and docs with new Nginx Ingress

Signed-off-by: Josh Kneubuhl <jkneubuh@us.ibm.com>

* Update cert paths in application_connection target

Signed-off-by: Josh Kneubuhl <jkneubuh@us.ibm.com>
This commit is contained in:
jkneubuh 2022-04-04 04:25:59 -04:00 committed by GitHub
parent 99e7ae8536
commit 40bfa49a80
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
7 changed files with 107 additions and 145 deletions

View file

@ -60,7 +60,6 @@ Tear down the cluster:
## [Detailed Guides](docs/README.md)
- [`./network`](docs/NETWORK.md)
- [Working with Kubernetes](docs/KUBERNETES.md)
- [Certificate Authorities](docs/CA.md)
- [Launching the Test Network](docs/TEST_NETWORK.md)

View file

@ -20,7 +20,7 @@ $ ./network up
Launching network "test-network":
...
✅ - Initializing TLS certificate Issuers ...
✅ - Launching ECert CAs ...
✅ - Launching Fabric CAs ...
✅ - Enrolling bootstrap ECert CA users ...
...
🏁 - Network is ready.

View file

@ -6,39 +6,41 @@ blockchain.
## TL/DR :
```shell
```
$ export TEST_NETWORK_CHANNEL_NAME="mychannel"
$ ./network channel create
Creating channel "mychannel":
✅ - Registering org Admin users ...
✅ - Enrolling org Admin users ...
✅ - Creating channel MSP ...
✅ - Aggregating channel MSP ...
✅ - Launching admin CLIs ...
✅ - Creating channel "mychannel" ...
✅ - Joining org1 peers to channel "mychannel" ...
✅ - Joining org2 peers to channel "mychannel" ...
✅ - Creating channel genesis block ...
✅ - Joining orderers to channel mychannel ...
✅ - Joining org1 peers to channel mychannel ...
✅ - Joining org2 peers to channel mychannel ...
🏁 - Channel is ready.
```
## Process Overview
In order to construct a communication channel, the following steps must be performed:
In order to construct a Fabric channel, the following steps must be performed:
1. TLS and MSP public certificates must be aggregated and distributed to all participants in the network.
1. Admin users must be registered and enrolled with the CAs
2. TLS and enrollment certificates must be aggregated and distributed to all participants in the network.
2. Each organization will launch a command-line pod with the MSP environment such that all fabric binaries are
executed as the Admin user.
3. The channel genesis block is constructed from `configtx.yaml`, specifying the location of channel MSP.
3. The channel genesis block is constructed from `configtx.yaml`, and `osnadmin` is used to distribute the new
channel configuration block to all orderers in the network.
4. The network peers fetch the genesis block from the orderers, and use the configuration to join the channel.
4. Network orderers are joined to the channel using the channel participation API.
5. Network peers are joioned to the channel.
## Distributing Channel MSP
## Aggregating the Channel MSP
```shell
✅ - Registering org Admin users ...
✅ - Enrolling org Admin users ...
✅ - Creating channel MSP ...
✅ - Aggregating channel MSP ...
```
One of the responsibilities of a Hyperledger Fabric _Consortium Organizer_ is to distribute the public MSP and
@ -49,74 +51,16 @@ assets on a distinct persistent volume, invisible to other the other participant
To distribute the TLS and MSP _public_ certificates, the test network emulates the responsibilities of the
consortium organizer by constructing a [Channel MSP](https://hyperledger-fabric.readthedocs.io/en/latest/membership/membership.html#channel-msps)
structure, extracting the relevant certificate files into a single `msp-{org}.tar.gz` archive. This MSP
archive is then relayed to network participants, where it can be extracted and used to set the MSP context
in which the peer executes administrative commands.
structure, extracting the relevant certificate files into a local folder before constructing the channel
genesis block. The `configtx.yaml` specifies the channel root folder as the consortium org's (org0) `MSPDir`
attribute.
The kube-specific techniques employed in MSP [construction](link) and [distribution](link) are:
- Org admin users are registered with the `fabric-ca-client`, storing the enrollment MSP structures in the local
`${PWD}/build/enrollments/${org}` folder.
- Channel MSP is generated by piping shell commands into each org's ECert CA pod.
- Channel MSP is extracted to the local system by piping the output of `tar` through `kubectl` (equivalent
to the [kubectl cp command](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#cp)).
MSP archives are saved locally in `/build/msp-{org}.tar.gz` archives.
- Channel MSP is distributed across network participants by transferring the MSP archive files from
`build/msp/msp-{org}.tgz` into the cluster as a config map.
- An `initContainer` is launched in each organization's Admin CLI pod, unfurling the MSP context from the
`msp-config` config map into the local volume.
- Channel MSP certificates are extracted from the ECert CA and cert-manager TLS signing authorities, storing the files
locally in the `${PWD}/channel-msp` folder.
Despite this additional complexity, this technique allows us to carefully target the MSP context in which
remote peer commands execute. The construct of an _MSP Archive_ may be extended to other circumstances
in which a consortium organizer transfers the _public_ certificates in an out-of-band fashion to
participants of a blockchain network.
Aggregating the certificates as a local MSP archive is accomplished by piping a `tar` archive from the output
of a remote `kubectl` into a local archive files. These files are then mounted into the Kube namespace by
constructing the `msp-config` config map:
```shell
kubectl -n $NS exec deploy/org0-ca -- tar zcvf - -C /var/hyperledger/fabric organizations/ordererOrganizations/org0.example.com/msp > msp/msp-org0.example.com.tgz
kubectl -n $NS exec deploy/org1-ca -- tar zcvf - -C /var/hyperledger/fabric organizations/peerOrganizations/org1.example.com/msp > msp/msp-org1.example.com.tgz
kubectl -n $NS exec deploy/org2-ca -- tar zcvf - -C /var/hyperledger/fabric organizations/peerOrganizations/org2.example.com/msp > msp/msp-org2.example.com.tgz
kubectl -n $NS delete configmap msp-config || true
kubectl -n $NS create configmap msp-config --from-file=msp/```
```
## `Admin` Commands
```shell
✅ - Launching admin CLIs ...
```
After the channel MSP archives have been constructed and loaded into the `msp-config` ConfigMap, a series
of kubernetes pods are launched in the namespace with an environment suitable for running the Fabric
`peer` commands as the organization Administrator. Before starting the CLI admin container, an `initContainer`
reads the MSP archives and unfurls them into a location on the org's persistent volume:
```yaml
# This init container will unfurl all of the MSP archives listed in the msp-config config map.
initContainers:
- name: msp-unfurl
image: busybox
command:
- sh
- -c
- "for msp in $(ls /msp/msp-*.tgz); do echo $msp && tar zxvf $msp -C /var/hyperledger/fabric; done"
volumeMounts:
- name: msp-config
mountPath: /msp
- name: fabric-volume
mountPath: /var/hyperledger
```
Once the MSP archives are extracted, the CLI is launched and the environment set such that `peer` commands
will be executed with the organization's Administrative role.
```shell
cat kube/org0/org0-admin-cli.yaml | sed 's,{{FABRIC_VERSION}},'${FABRIC_VERSION}',g' | kubectl -n $NS apply -f -
cat kube/org1/org1-admin-cli.yaml | sed 's,{{FABRIC_VERSION}},'${FABRIC_VERSION}',g' | kubectl -n $NS apply -f -
cat kube/org2/org2-admin-cli.yaml | sed 's,{{FABRIC_VERSION}},'${FABRIC_VERSION}',g' | kubectl -n $NS apply -f -
```
## Create the Channel
```shell
@ -129,9 +73,9 @@ services to register the channel genesis block configuration on the ordering nod
```shell
configtxgen -profile TwoOrgsApplicationGenesis -channelID '${CHANNEL_NAME}' -outputBlock genesis_block.pb
osnadmin channel join --orderer-address org0-orderer1:9443 --channelID '${CHANNEL_NAME}' --config-block genesis_block.pb
osnadmin channel join --orderer-address org0-orderer2:9443 --channelID '${CHANNEL_NAME}' --config-block genesis_block.pb
osnadmin channel join --orderer-address org0-orderer3:9443 --channelID '${CHANNEL_NAME}' --config-block genesis_block.pb
osnadmin channel join --orderer-address org0-orderer1-admin.vcap.me --channelID '${CHANNEL_NAME}' --config-block genesis_block.pb
osnadmin channel join --orderer-address org0-orderer2-admin.vcap.me --channelID '${CHANNEL_NAME}' --config-block genesis_block.pb
osnadmin channel join --orderer-address org0-orderer3-admin.vcap.me --channelID '${CHANNEL_NAME}' --config-block genesis_block.pb
```
@ -151,7 +95,7 @@ by retrieving the genesis block from the orderers and then joining the channel:
fetch oldest \
genesis_block.pb \
-c '${CHANNEL_NAME}' \
-o org0-orderer1:6050 \
-o org0-orderer1.vcap.me \
--tls --cafile /var/hyperledger/fabric/organizations/ordererOrganizations/org0.example.com/msp/tlscacerts/org0-tls-ca.pem
# Join peer1 to the channel.
@ -159,7 +103,7 @@ by retrieving the genesis block from the orderers and then joining the channel:
peer channel \
join \
-b genesis_block.pb \
-o org0-orderer1:6050 \
-o org0-orderer1.vcap.me \
--tls --cafile /var/hyperledger/fabric/organizations/ordererOrganizations/org0.example.com/msp/tlscacerts/org0-tls-ca.pem
# Join peer2 to the channel.
@ -167,7 +111,7 @@ by retrieving the genesis block from the orderers and then joining the channel:
peer channel \
join \
-b genesis_block.pb \
-o org0-orderer1:6050 \
-o org0-orderer1.vcap.me \
--tls --cafile /var/hyperledger/fabric/organizations/ordererOrganizations/org0.example.com/msp/tlscacerts/org0-tls-ca.pem
```

View file

@ -4,18 +4,20 @@ To get started with the Kube test network, you will need access to a Kubernetes
## TL/DR :
```shell
```
$ ./network kind
Initializing KIND cluster "kind":
✅ - Pulling docker images for Fabric 2.3.2 ...
✅ - Creating cluster "kind" ...
✅ - Launching Nginx ingress controller ...
✅ - Launching ingress controller ...
✅ - Launching cert-manager ...
✅ - Launching container registry "kind-registry" at localhost:5000 ...
✅ - Waiting for cert-manager ...
✅ - Waiting for ingress controller ...
🏁 - Cluster is ready.
```
and :
```shell
```
$ ./network unkind
Deleting cluster "kind":
☠️ - Deleting KIND cluster kind ...
@ -26,7 +28,7 @@ Deleting cluster "kind":
## Kube Context:
For illustration purposes, this project attempts in all cases to _keep it simple_ as the
general rule. By default, we will rely on `kind` ([Kubernetes IN Docker](https://kind.sigs.k8s.io))
general rule. By default, we will rely on KIND ([Kubernetes IN Docker](https://kind.sigs.k8s.io))
as a mechanism to quickly spin up ephemeral, short-lived clusters for development and
illustration.
@ -46,10 +48,6 @@ for development, testing, or CI, you can create a new cluster with:
```shell
$ ./network kind
```
or:
```shell
$ kind create cluster
```
By default, `kind` will set the current Kube context to reference the new cluster. Any
interaction with `kubectl` (or kube-context aware SDKs) will inherit the current context.
@ -74,9 +72,9 @@ $ kind delete cluster
## Test Network Structure
To emulate a more realistic example of multi-party collaboration, the test network
forms a blockchain consensus group spanning three virtual organizations. Access to the
blockchain is entirely constrained to Kubernetes private networks, and consuming applications
make use of a Kube ingress controller for external visibility.
forms a blockchain consensus group spanning three virtual organizations. Network I/O between the
blockchain nodes is entirely constrained to Kubernetes private networks, and consuming applications
make use of a Kubernetes / Nginx ingress controller for external visibility.
In k8s terms:
@ -171,6 +169,46 @@ export TEST_NETWORK_FABRIC_VERSION=2.4.0
./network up
```
## Nginx Ingress Controller
When Fabric nodes communicate within the k8s cluster, TCP sockets are established via Kube DNS service
aliases (e.g. grpcs://org1-peer1.test-network.svc.cluster.local:443) and traverse private K8s network routes.
For access from _external clients_, all traffic into the network nodes are routed into the correct pod by
virtue of an Nginx ingress controller bound to the host OS ports :80 and :443. To differentiate between
services, the Nginx provides a "layer 6" traffic router based on the http(s) host alias. In addition to
constructing Deployments, Pods, and Services, each Fabric node exposes a set of `Ingress` routes binding
the virtual host name to the corresponding endpoint.
TLS traffic tunneled through the ingress controller has been configured in "ssl-passthrough" mode. For
secure access to services, client applications must present the TLS root certificate of the appropriate
organization when connecting to peers, orderers, and CAs.
## What is `*.vcap.me` ?
In order to expose a dynamic set of DNS host aliases matching the Nginx ingress controller, the test network
employs the public DNS wildcard domain `*.vcap.me` to resolve host and subdomains to the local loopback
address 127.0.0.1.
The vcap.me domain is managed by VMWare and is associated with the
[VMWare Cloud Application Platform](https://github.com/cloudfoundry-attic/vcap) (VCAP).
Using this DNS wildcard alias means that all ingress points bound to the *.vcap.me domain will resolve to your
local host, conveniently routing traffic into the KIND cluster on ports :80 and :443.
To override the *.vcap.me network ingress domain (for example in cloud-based environments supporting a DNS
wildcard resolver) set the `TEST_NETWORK_DOMAIN` environment variable before invoking `./network`
targets. E.g.:
```shell
export TEST_NETWORK_DOMAIN=lvh.me
./network up
curl -s --insecure https://org0-ca.lvh.me/cainfo | jq
```
## Cloud Vendors
While the test network primarily targets KIND clusters, the singular reliance on the Kube API plane
@ -187,11 +225,5 @@ In general, at a high-level the steps required to port the test network to ANY k
- Upload your chaincode, gateway clients, and application logic to an external Container Registry.
- Run with a `ServiceAccount` and role bindings suitable for creating `Pods`, `Deployments`, and `Services`.
Example configurations for common cloud vendors:
### IKS
### OCP
### AWS
### Azure
## Next : [Fabric Certificate Authorities](CA.md)

View file

@ -30,7 +30,6 @@ _Chaincode-as-a-Service_ running in a shared Kubernetes namespace.
## Detailed Guides
- [`./network`](NETWORK.md)
- [Working with Kubernetes](KUBERNETES.md)
- [Certificate Authorities](CA.md)
- [Planning for a CA](CA.md#planning-for-a-ca)

View file

@ -8,7 +8,7 @@ all of the nodes, using the local MSPs to launch our network peers and orderers.
### TL/DR :
```shell
```
./network up
...
✅ - Creating local node MSP ...
@ -17,7 +17,7 @@ all of the nodes, using the local MSPs to launch our network peers and orderers.
🏁 - Network is ready.
```
## Fabric MSP Context
## Fabric Node MSP Context
Before we launch the network peers and orderers, each node in the network needs to have available:
@ -47,12 +47,10 @@ For example, the ordering organization sets up the node local MSP with:
fabric-ca-client register --id.name org0-orderer1 --id.secret ordererpw --id.type orderer --url https://org0-ca --mspdir $FABRIC_CA_CLIENT_HOME/org0-ca/rcaadmin/msp
fabric-ca-client register --id.name org0-orderer2 --id.secret ordererpw --id.type orderer --url https://org0-ca --mspdir $FABRIC_CA_CLIENT_HOME/org0-ca/rcaadmin/msp
fabric-ca-client register --id.name org0-orderer3 --id.secret ordererpw --id.type orderer --url https://org0-ca --mspdir $FABRIC_CA_CLIENT_HOME/org0-ca/rcaadmin/msp
fabric-ca-client register --id.name org0-admin --id.secret org0adminpw --id.type admin --url https://org0-ca --mspdir $FABRIC_CA_CLIENT_HOME/org0-ca/rcaadmin/msp --id.attrs "hf.Registrar.Roles=client,hf.Registrar.Attributes=*,hf.Revoker=true,hf.GenCRL=true,admin=true:ecert,abac.init=true:ecert"
fabric-ca-client enroll --url https://org0-orderer1:ordererpw@org0-ca --csr.hosts org0-orderer1 --mspdir /var/hyperledger/fabric/organizations/ordererOrganizations/org0.example.com/orderers/org0-orderer1.org0.example.com/msp
fabric-ca-client enroll --url https://org0-orderer2:ordererpw@org0-ca --csr.hosts org0-orderer2 --mspdir /var/hyperledger/fabric/organizations/ordererOrganizations/org0.example.com/orderers/org0-orderer2.org0.example.com/msp
fabric-ca-client enroll --url https://org0-orderer3:ordererpw@org0-ca --csr.hosts org0-orderer3 --mspdir /var/hyperledger/fabric/organizations/ordererOrganizations/org0.example.com/orderers/org0-orderer3.org0.example.com/msp
fabric-ca-client enroll --url https://org0-admin:org0adminpw@org0-ca --mspdir /var/hyperledger/fabric/organizations/ordererOrganizations/org0.example.com/users/Admin@org0.example.com/msp
# Create an MSP config.yaml (why is this not generated by the enrollment by fabric-ca-client?)
echo "NodeOUs:
@ -140,23 +138,22 @@ in the core.yaml file. Note that each node's [environment](../kube/org1/org1-pe
node local MSP folders, certificates, and TLS signing keys that we generated above.
Note that the deployment yaml files include some basic template substitution and parameters. For simplicity and
clarity, we elected to use basic string substitution with sed/awk/bash/etc., rather than introduce a Kube template
binding system (e.g. Helm, Kustomize, Kapitan, Ansible, etc.) for manipulating yaml templates:
clarity, we elected to use basic string substitution with `envsubst`, rather than introduce a Kube template
binding system (e.g. Helm, Kustomize, Kapitan, Ansible, etc.) for manipulating yaml templates. Any environment
variables declared in the `./network` script will be applied to these k8s templates. E.g.:
```shell
cat kube/org0/org0-orderer1.yaml | sed 's,{{FABRIC_VERSION}},'${FABRIC_VERSION}',g' | kubectl -n $NS -f -
cat kube/org0/org0-orderer2.yaml | sed 's,{{FABRIC_VERSION}},'${FABRIC_VERSION}',g' | kubectl -n $NS -f -
cat kube/org0/org0-orderer3.yaml | sed 's,{{FABRIC_VERSION}},'${FABRIC_VERSION}',g' | kubectl -n $NS -f -
export FABRIC_VERSION=2.4.2
...
cat kube/org0/org0-orderer1.yaml | envsubst | kubectl -n $NS -f -
cat kube/org0/org0-orderer2.yaml | envsubst | kubectl -n $NS -f -
cat kube/org0/org0-orderer3.yaml | envsubts | kubectl -n $NS -f -
# Wait for the orderers to completely start before launching the network peer nodes.
kubectl -n $NS rollout status deploy/org0-orderer1
kubectl -n $NS rollout status deploy/org0-orderer2
kubectl -n $NS rollout status deploy/org0-orderer3
cat kube/org1/org1-peer1.yaml | sed 's,{{FABRIC_VERSION}},'${FABRIC_VERSION}',g' | kubectl -n $NS -f -
cat kube/org1/org1-peer2.yaml | sed 's,{{FABRIC_VERSION}},'${FABRIC_VERSION}',g' | kubectl -n $NS -f -
cat kube/org2/org2-peer1.yaml | sed 's,{{FABRIC_VERSION}},'${FABRIC_VERSION}',g' | kubectl -n $NS -f -
cat kube/org2/org2-peer2.yaml | sed 's,{{FABRIC_VERSION}},'${FABRIC_VERSION}',g' | kubectl -n $NS -f -
```
## Next Steps :

View file

@ -5,16 +5,6 @@
# SPDX-License-Identifier: Apache-2.0
#
function app_extract_MSP_archives() {
mkdir -p build/msp
set -ex
kubectl -n $NS exec deploy/org1-ca -- tar zcf - -C /var/hyperledger/fabric organizations/peerOrganizations/org1.example.com/msp | tar zxf - -C build/msp
kubectl -n $NS exec deploy/org2-ca -- tar zcf - -C /var/hyperledger/fabric organizations/peerOrganizations/org2.example.com/msp | tar zxf - -C build/msp
kubectl -n $NS exec deploy/org1-ca -- tar zcf - -C /var/hyperledger/fabric organizations/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp | tar zxf - -C build/msp
kubectl -n $NS exec deploy/org2-ca -- tar zcf - -C /var/hyperledger/fabric organizations/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp | tar zxf - -C build/msp
}
function app_one_line_pem {
echo "`awk 'NF {sub(/\\n/, ""); printf "%s\\\\\\\n",$0;}' $1`"
}
@ -43,18 +33,19 @@ function app_id {
function construct_application_configmap() {
push_fn "Constructing application connection profiles"
app_extract_MSP_archives
ENROLLMENT_DIR=${TEMP_DIR}/enrollments
CHANNEL_MSP_DIR=${TEMP_DIR}/channel-msp
mkdir -p build/application/wallet
mkdir -p build/application/gateways
local peer_pem=build/msp/organizations/peerOrganizations/org1.example.com/msp/tlscacerts/org1-tls-ca.pem
local ca_pem=build/msp/organizations/peerOrganizations/org1.example.com/msp/cacerts/org1-ca.pem
local peer_pem=$CHANNEL_MSP_DIR/peerOrganizations/org1/msp/tlscacerts/tlsca-signcert.pem
local ca_pem=$CHANNEL_MSP_DIR/peerOrganizations/org1/msp/cacerts/ca-signcert.pem
echo "$(json_ccp 1 $peer_pem $ca_pem)" > build/application/gateways/org1_ccp.json
peer_pem=build/msp/organizations/peerOrganizations/org2.example.com/msp/tlscacerts/org2-tls-ca.pem
ca_pem=build/msp/organizations/peerOrganizations/org2.example.com/msp/cacerts/org2-ca.pem
peer_pem=$CHANNEL_MSP_DIR/peerOrganizations/org2/msp/tlscacerts/tlsca-signcert.pem
ca_pem=$CHANNEL_MSP_DIR/peerOrganizations/org2/msp/cacerts/ca-signcert.pem
echo "$(json_ccp 2 $peer_pem $ca_pem)" > build/application/gateways/org2_ccp.json
@ -62,13 +53,13 @@ function construct_application_configmap() {
push_fn "Getting Application Identities"
local cert=build/msp/organizations/peerOrganizations/org1.example.com/users/Admin\@org1.example.com/msp/signcerts/cert.pem
local pk=build/msp/organizations/peerOrganizations/org1.example.com/users/Admin\@org1.example.com/msp/keystore/server.key
local cert=$ENROLLMENT_DIR/org1/users/org1admin/msp/signcerts/cert.pem
local pk=$ENROLLMENT_DIR/org1/users/org1admin/msp/keystore/key.pem
echo "$(app_id Org1MSP $cert $pk)" > build/application/wallet/appuser_org1.id
local cert=build/msp/organizations/peerOrganizations/org2.example.com/users/Admin\@org2.example.com/msp/signcerts/cert.pem
local pk=build/msp/organizations/peerOrganizations/org2.example.com/users/Admin\@org2.example.com/msp/keystore/server.key
local cert=$ENROLLMENT_DIR/org2/users/org2admin/msp/signcerts/cert.pem
local pk=$ENROLLMENT_DIR/org2/users/org2admin/msp/keystore/key.pem
echo "$(app_id Org2MSP $cert $pk)" > build/application/wallet/appuser_org2.id
@ -76,7 +67,7 @@ function construct_application_configmap() {
push_fn "Creating ConfigMap \"app-fabric-tls-v1-map\" with TLS certificates for the application"
kubectl -n $NS delete configmap app-fabric-tls-v1-map || true
kubectl -n $NS create configmap app-fabric-tls-v1-map --from-file=./build/msp/organizations/peerOrganizations/org1.example.com/msp/tlscacerts
kubectl -n $NS create configmap app-fabric-tls-v1-map --from-file=$CHANNEL_MSP_DIR/peerOrganizations/org1/msp/tlscacerts
pop_fn
push_fn "Creating ConfigMap \"app-fabric-ids-v1-map\" with identities for the application"
@ -103,7 +94,7 @@ data:
fabric_gateway_hostport: org1-peer-gateway-svc:7051
fabric_gateway_sslHostOverride: org1-peer-gateway-svc
fabric_user: appuser_org1
fabric_gateway_tlsCertPath: /fabric/tlscacerts/org1-tls-ca.pem
fabric_gateway_tlsCertPath: /fabric/tlscacerts/tlsca-signcert.pem
EOF
kubectl -n $NS apply -f build/app-fabric-org1-v1-map.yaml