OpenTracing Support in TIBCO Cloud™ Integration for TIBCO Flogo® Apps which use TIBCO Cloud Mesh for Service Discovery

In today’s Cloud and Microservices world, it is very important to profile and monitor applications deployed on Cloud. It is important for end user especially for IT and DevOps teams to monitor the applications, pinpoint where failures occur, what causes poor performance for their applications.

Now when it comes to Monitoring most of you might have heard the term OpenTracing at some point. But what’s exactly is OpenTracing and why it is used for. Let us see in details.

OpenTracing is comprised of an API specification, frameworks and libraries that have implemented the specification, and documentation for the project. OpenTracing allows developers to add instrumentation to their application code using APIs that do not lock them into any one particular product or vendor.

Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing system released as open source by Uber Technologies. It is used for monitoring and troubleshooting microservices-based distributed systems, including:

• Distributed context propagation
• Distributed transaction monitoring
• Root cause analysis
• Service dependency analysis
• Performance / latency optimization

TIBCO Cloud™ Integration supports monitoring of TIBCO Flogo® Applications using OpenTracing. Let us see how.

TIBCO Cloud™ Integration provides an implementation of the OpenTracing framework using the Jaeger backend. In TIBCO Cloud™ Integration, the TIBCO Flogo® Application is built with Jaeger implementation and can be enabled by setting the FLOGO_APP_MONITORING_OT_JAEGER engine variable to true. Also you have to set JAEGER_ENDPOINT engine variable. You can track how the flow went through, execution time for each activity, or in case of failure, the cause of the failure.

Each app is displayed as a service in the Jaeger UI. In TIBCO Flogo® App, each flow is one operation (trace) and each activity in the flow is a span of the trace. A trace is the complete lifecycle of a group of spans. The flow is the root span and its activities are its child spans.

Steps to start the Jaeger Service :

1. Start Jaeger Service on-premise or on any PaaS platform like Kubernetes.

Option 1: Start Jaeger Service on-premise:

The simplest way to start the all-in-one is to use the pre-built image published to DockerHub (a single command line).

$ docker run -d --name jaeger \
  -e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \
  -p 5775:5775/udp \
  -p 6831:6831/udp \
  -p 6832:6832/udp \
  -p 5778:5778 \
  -p 16686:16686 \
  -p 14268:14268 \
  -p 9411:9411 \
  jaegertracing/all-in-one:1.8

You can then navigate to http://<HOST IP Address>:16686 to access the Jaeger UI.

To access on-premise service from TIBCO Cloud™ Integration, TIBCO Cloud™ — Proxy Agent/tibtunnel can be used. Please refer these links for more information.

Configure the tibtunnel profile using the below command -

./tibtunnel configure — profile <ProfileName> — accessKey <accessKey>

Configure the tibtunnel using the below command -

./tibcli app tibtunnel — launch — accessKey <accessKey> — profile <ProfileName> — spec 9268:<on-prem IP>:14268 — spec 9686:<on-prem IP>:16686

By default, TRACE_COLLECTOR_PORT set to 14268 which is outside of port range 9999 on TIBCO Cloud™ Integration. So, make sure you map any port below 9999 when running Jaeger locally. For eg, Host Port 9268 is mapped to On-prem Service Port 14268 in the above command.

Option 2: Start Jaeger Service on any PaaS platform like Kubernetes:

Use the below command to create jaeger-agent, jaeger-collector, jaeger-query, zipkin services on Kubernetes -

kubectl create -f https://raw.githubusercontent.com/jaegertracing/jaeger-kubernetes/master/all-in-one/jaeger-all-in-one-template.yml

It is recommended to create all of these in a single namespace. Also make sure jaeger-collector-http port is set to value below 9999 in the yaml file, since TIBCO Cloud™ Integration port range is below 9999.

You can refer this link for more info.

2. Set Engine Variables in Environment Controls Tab for TIBCO Flogo® App in TIBCO Cloud™ Integration

Set the following engine variables -

FLOGO_APP_MONITORING_OT_JAEGER = true
JAEGER_ENDPOINT = http://<JAEGER_HOST>:<TRACE_COLLECTOR_PORT>/api/traces

where JAEGER_HOST is the HOST IP where the jaeger service is running either an on-prem HOST IP or jaeger-collector service External IP if jaeger is running on Kubernetes.

TRACE_COLLECTOR_PORT is the HOST PORT exposed by on-prem service or jaeger-collector service deployed on Kubernetes

Engine Variables

If you are using TIBCO Cloud™ — Proxy Agent/tibtunnel to connect to your on-prem jaeger service, then you will need to attach the Access Key to your TIBCO Flogo® App from Hybrid connectivity Tab.

After making these changes, push the TIBCO Flogo® App and access the jaeger-query service UI using the host and port of the service.

3. Access the jaeger-query service UI using the HOST and PORT exposed by on-prem service or Kubernetes

You should see your TIBCO Flogo® App registered and in the Service dropdown as shown below.

If you see above, we see spans across subflow, and another service app which is called from OpenTracingDemo App. You will see that InvokeREST Service in OT_Service App is calling external REST endpoint “https://my-json-server.typicode.com/tibcosoftware/tci-flogo/Book" which is taking 218.24 ms

Understanding the workflow.
I have two TIBCO Flogo® Apps OpenTracingDemo and OT_Service.

1. OpenTracingDemo App :

This app calls a subflow in main flow and sublow is using TIBCO Cloud Mesh to invoke OT_Service deployed as private endpoint in TIBCO Cloud™ Integration

2. OT_Service App

This app calls the external books service.