fabric-samples/test-network-k8s/docs/CALIPER.md

Benchmarking the performance using Hyperledger Caliper

This document introduces how to use Hyperledger Caliper to benchmark the performance of the Hyperledger Fabric environment created with test-network-k8s.

Fabric adapter manual of Hyperledger Caliper v0.6.0 only describes how to connect to test-network. So this document will explain how to benchmark the performance of the Kubernetes test network using Hyperledger Caliper and Asset Transfer Basic chaincode.

The following documentation assumes that test-network-k8s and Hyperledger Caliper v0.6.0 are located on the same host.

Setting of test-network-k8s side

As described in the README of test-network-k8s, launch the network, create a channel, and deploy and invoke the basic-asset-transfer smart contract:

./network kind

./network cluster init

./network up

./network channel create

./network chaincode deploy asset-transfer-basic ../asset-transfer-basic/chaincode-java

./network chaincode invoke asset-transfer-basic '{"Args":["InitLedger"]}'

REST API will not be used in the procedure described below, but the connection profile will be generated by launching it:

./network rest-easy

Setting of Hyperledger Caliper side

Following Install manual of Hyperledger Caliper v0.6.0, install Hyperledger Caliper from npm:

git clone https://github.com/hyperledger/caliper-benchmarks.git
cd caliper-benchmarks
npm install --only=prod @hyperledger/caliper-cli@0.6.0
npx caliper bind --caliper-bind-sut fabric:fabric-gateway

Copy the connection profile created in test-network-k8s environment to Caliper environment.

cp <fabric-samples install dir>/test-network-k8s/build/fabric-rest-sample-config/HLF_CONNECTION_PROFILE_ORG1 networks/fabric/connection-profile.json

Edit "url" and "grpcOptions" in "peers" section of connection-profile.json as below:

    "peers": {
        "org1-peers": {
            "url": "grpcs://org1-peer1.localho.st:443",
            "tlsCACerts": {
                "pem": <contents of pem file>"
            },
            "grpcOptions": {
                "ssl-target-name-override": "org1-peer1.localho.st",
                "hostnameOverride": "org1-peer1.localho.st"
            }
        }
    },

Open networks/fabric/test-network.yaml and edit it as below:

name: Caliper Benchmarks
version: "2.0.0"

caliper:
  blockchain: fabric

channels:
  # channelName of mychannel matches the name of the channel created by test network
  - channelName: mychannel
    # the chaincodeIDs of all the fabric chaincodes in caliper-benchmarks
    contracts:
    - id: fabcar
    - id: fixed-asset
    - id: marbles
    - id: simple
    - id: smallbank
    - id: asset-transfer-basic

organizations:
  - mspid: Org1MSP
    # Identities come from cryptogen created material for test-network
    identities:
      certificates:
      - name: 'User1'
        clientPrivateKey:
          path: '<fabric-samples install absolute dir>/test-network-k8s/build/enrollments/org1/users/org1admin/msp/keystore/key.pem'

        clientSignedCert:
          path: '<fabric-samples install absolute dir>/test-network-k8s/build/enrollments/org1/users/org1admin/msp/signcerts/cert.pem'

    connectionProfile:
      path: 'networks/fabric/connection-profile.json'
      discover: true

Currently, sample code for running a performance benchmark targeting asset-transfer-basic is not published on the Caliper repository. Therefore, use the sample code published in Caliper's user manual to build a test workload.

As shown in "Step 3" of the above document, create workload/readAsset.js file and edit it as below:

'use strict';

const { WorkloadModuleBase } = require('@hyperledger/caliper-core');

class MyWorkload extends WorkloadModuleBase {
    constructor() {
        super();
    }

    async initializeWorkloadModule(workerIndex, totalWorkers, roundIndex, roundArguments, sutAdapter, sutContext) {
        await super.initializeWorkloadModule(workerIndex, totalWorkers, roundIndex, roundArguments, sutAdapter, sutContext);

        for (let i=0; i<this.roundArguments.assets; i++) {
            const assetID = `${this.workerIndex}_${i}`;
            console.log(`Worker ${this.workerIndex}: Creating asset ${assetID}`);
            const request = {
                contractId: this.roundArguments.contractId,
                contractFunction: 'CreateAsset',
                invokerIdentity: 'User1',
                contractArguments: [assetID,'blue','20','penguin','500'],
                readOnly: false
            };

            await this.sutAdapter.sendRequests(request);
        }
    }

    async submitTransaction() {
        const randomId = Math.floor(Math.random()*this.roundArguments.assets);
        const myArgs = {
            contractId: this.roundArguments.contractId,
            contractFunction: 'ReadAsset',
            invokerIdentity: 'User1',
            contractArguments: [`${this.workerIndex}_${randomId}`],
            readOnly: true
        };

        await this.sutAdapter.sendRequests(myArgs);
    }

    async cleanupWorkloadModule() {
        for (let i=0; i<this.roundArguments.assets; i++) {
            const assetID = `${this.workerIndex}_${i}`;
            console.log(`Worker ${this.workerIndex}: Deleting asset ${assetID}`);
            const request = {
                contractId: this.roundArguments.contractId,
                contractFunction: 'DeleteAsset',
                invokerIdentity: 'User1',
                contractArguments: [assetID],
                readOnly: false
            };

            await this.sutAdapter.sendRequests(request);
        }
    }
}

function createWorkloadModule() {
    return new MyWorkload();
}

module.exports.createWorkloadModule = createWorkloadModule;

As shown in "Step 4" of the above document, create benchmarks/myAssetBenchmark.yaml file and edit it as below:

test:
    name: basic-contract-benchmark
    description: test benchmark
    workers:
      number: 2
    rounds:
      - label: readAsset
        description: Read asset benchmark
        txDuration: 30
        rateControl:
          type: fixed-load
          opts:
            transactionLoad: 2
        workload:
          module: workload/readAsset.js
          arguments:
            assets: 10
            contractId: asset-transfer-basic

After creating the above files, you can run a performance benchmark with the following command, which will create some assets and measure the time it takes to reference the assets.

npx caliper launch manager --caliper-workspace . --caliper-benchconfig benchmarks/myAssetBenchmark.yaml --caliper-networkconfig networks/fabric/test-network.yaml