Mike  Kozey

Mike Kozey

1620052800

Tracing in Grafana With Tempo and Jaeger

In this post, we will see how to introduce tracing in logs and visualize it easily. We will use Prometheus, Grafana Loki, Jaeger, and Grafana Tempo as data sources for monitoring metrics, logs, and traces respectively in Grafana.

Why Do I Need Tracing if I Have a Good Logging and Monitoring Framework?

Application logs are beneficial for displaying important events if something is not working as expected (failure, error, incorrect config, etc.). Although it is a very crucial element in application design, one should log thriftily. This is because log collection, transformation, and storage are costly.

Unlike logging, which is event triggered and discrete, tracing provides a broader and continuous application view. Tracing helps us understand the path of a process/transaction/entity while traversing the application stack and identifying the bottlenecks at various stages. This helps to optimize the application and increase performance.

In this post, we will see how to introduce tracing in logs and visualize it easily. In this example, we will use Prometheus, Grafana Loki, Jaeger, and Grafana Tempo as data sources for monitoring metrics, logs, and traces respectively in Grafana.

What Is Distributed Tracing?

In a microservices architecture, understanding an application behavior can be an intriguing task. This is because the incoming requests may span over multiple services, and each intermittent service may have one or more operations on that request. It thus increases complexity and takes more time while troubleshooting problems.

Distributed tracing helps to get insight into the individual operation and pinpoint the areas of failure caused by poor performance.

What Is OpenTracing?

OpenTracing comprises an API specification, frameworks, and libraries to enable distributed tracing in any application. OpenTracing APIs are very generic and prevents vendor/product lock-in. Recently, OpenTracing and OpenCensus merged to form  OpenTelemetry (acronym OTel). It targets the creation and management of telemetry data such as traces, metrics, and logs through a set of APIs, SDKs, tooling, and integrations. Note:  OpenCensus consists of a set of libs for various languages to collect metrics and traces from Applications, visualize them locally and send them remotely for storage and analysis.

#tutorial #microservices #monitoring #logging #grafana

What is GEEK

Buddha Community

Tracing in Grafana With Tempo and Jaeger
Carmen  Grimes

Carmen Grimes

1595498460

Contact Tracing App: The Technology, Approach to fight COVID-19/Corona

As COVID-19 staggeringly lands blows to nations across the world, governments are considering ways to see their citizens through this pandemic. At the moment, a WHO situation report clocks the number of confirmed cases above two million along with more than one hundred thousand deaths. With vaccines dubbed as the best possible chance to tackle COVID-19 having no precise time frame of being ready, the talk is quickly shifting away to Contact Tracing Applications.

Contact tracing apps are digital solutions that use mobile technology to power the process of manual contact tracing. The apps follow a user’s movement, either by the use of Bluetooth technology, QR codes, or geo-location data while also tracking and keeping data from other user phones nearby. If one user gets diagnosed, the apps alert other users that they may have been exposed to the virus. As such, Contact Tracing Applications are being welcomed and perceived as an important approach to stem the spread of COVID-19 by providing a more accurate platform with data and information about affected individuals.

How Contact Tracing App Works

contact-tracing-app-devathon-

As mentioned above, contact tracing apps leverage mobile technology to trace cases of possible infection more accurately. But how exactly? Once installed and operative, the phone runs the app simultaneously with Bluetooth or location data to transmit signals with unique keys or IDs to phones in the designated range of connection. Similarly, the other phones with the app installed to detect and send back the signals.

For instance, if ‘Individual A’ has the app installed and goes outdoors to run some errands, they will interact with other individuals. In such a case, supposing all the other individuals had functional Contact Tracing Apps, each phone would exchange and store the contact data anonymously. It is important to note that the data collected only covers the app range distance to disregard irrelevant contacts and that their keys repeatedly change as individuals move. In any event that ‘Individual A’ tests positive for COVID-19 through confirmed tests, users who were previously within the proximity of ‘Individual A’ are alerted. Consequently, they are notified to check for symptoms, self-isolate, or get tested. Each time a person tests positive, the app notifies and advises the affected individuals.

In a nutshell, Contact Tracing Apps automate and supplement the traditional concept of tracing contacts to achieve extensive and realistic results in the least time possible.

What are the Benefits of Contact Tracing Apps?

Contract Tracing Apps are assets that offer indispensable solutions to health institutions and the public against COVID-19. There are several reasons why many governments are urging their citizens to use digital contact tracing apps to combat the spread of COVID-19. They include:

  1. The apps are more effective than manual tracing. While not perfect, their predictive algorithms frequently observe individuals detect new cases and analyze the probability one was infected. If one has contact with an asymptomatic individual, they are immediately notified and advised accordingly. Therefore, this saves time, energy, and resources that would have otherwise been overused.
  2. Contact tracing apps facilitate the relaxation of imposed restrictions or lockdowns. With a large number of infected people identified by the apps and put under surveillance, healthy citizens can be allowed to go about their duties. This may be a significant turning point to try and revive economies.
  3. Users’ private data is encrypted and secured. Even if you test positive, other users will only get notifications of possible infections. Your information is protected from both other users and developers of the app.
  4. They will increase the capacity to test and detect COVID-19 cases. With infected users alert, users who come in contact with affected persons come forward to be tested and treated with a higher recovery chance.

Future of Contact Tracing Apps?

Currently, the role of contact tracing apps is limited to accurately identifying infected individuals and their contacts as well as facilitating a quicker response to the Covid-19 threat.

Beyond that, the use of contact tracing apps is projected to take a different turn. One key area bound to change is how people’s privacy is handled. Tech institutions are under growing pressure to devise ways to develop privacy-preserving Contact Tracing Apps.

This will earn the users-trust, which is a pillar for these apps to help contain the disease. Technically, the technology will also have to improve drastically. The apps will have to seamlessly integrate with the user’s phone lifestyle causing minimal or no interference. With most applications having an open-source code, Artificial Intelligence, Beacon Technology, and Big Data solutions will be increasingly harnessed to power and improve them. The apps may also cut across various types of industries apart from health institutions.

How Can It Help to Trace COVID-19 and Reduce the Spread of the Virus?

contact-tracing-app-devathon-3

Contact Tracing Apps will effectively help stem lowering the cases of COVID-19. By using the apps, officials are able to monitor high-risk individuals easily. Also, should any new case arise, both users and health officials get notified they will swiftly act to trace, test, or isolate infected individuals.

Unlike traditional contact tracing, which may not get all contacts, these apps ensure that once Covid-19 cases are detected, they are all treated early, and those other individuals are not exposed to the infection. They also ward off users from high-risk areas. In the long run, they help break the COVID-19 chain by preventing further spread. Illustratively, an online publication by  CNBC states that more than 500,000 using a Singapore-registered mobile number downloaded the TraceTogether app within the first 24 hours of its launch. Subsequently, together with other government efforts, Singapore has since lowered the infection rate and eased restrictions.

If Contact Tracing Apps are implemented and used alongside other policies, we may as well be a few steps way to curbing this virus.

#android app #ios app #mobile app development #news #technology #contact tracing #contact tracing app #contact tracing app approach #contact tracing app technology #contact tracing coronavirus #contact tracing process #corona virus detecting app #corona virus tracing app #corona virus tracker #corona virus tracker live

Comparison of Open Source API Analytics and Monitoring Tools

For any API-first company, implementing the right API analytics platform is critical for tracking the utilization of your APIs and to discover any performance or functional issues that may be impacting customers. There are a variety of open-source projects you can leverage to build a complete API analytics platform.

Before jumping into building an API analytics solution yourself, you should first list out your requirements and use cases. Not all tools support all use cases directly, and may require heavy investment in development and integration.

API analytics build requirements

Need to answer engineering or business questions

Engineers commonly turn to API logs and metrics to understand what’s happening over their APIs, inspect payloads, and root cause issues with their services that come up. Real-time API logging capabilities is a must for engineers that want to depend on their API analytics solution to put out the (hopefully far and few between) fires due to an API outage or reliability issue. Because real-time logging can raise the compute and storage cost, not all analytics infrastructure maintains a real-time pipeline. On the flip side, only the most recent data is needed to answer hair on fire engineering problems, so data can be retired after a short time period such as after 24 hours.

Engineering and product leaders make strategic decisions based on lines, not dots. Business questions are answered from historical trends in data, which may be over months or even years. This means your API analytics build should be capable of storing data for long retention periods such as for multiple years. This also requires infrastructure that can roll up and compress your data as storing and running aggregations on raw event logs will cripple your analytics infrastructure

Real-time alerting and monitoring?

While building dashboards and throwing them up on a monitor can be a great way to monitor your metrics, it still requires manually checking them periodically. To become more proactive, many API analytics builds also have some sort of monitoring and alerting features. These can vary from simple threshold-based alerts that sends an email when a metric reaches a certain value, to complex monitoring rules and workflows that can perform complex aggregations in real-time and route alerts to incident response platforms like PagerDuty and BigPanda.

Some customer-facing teams like developer relations and customer success may want to build automated workflows to notify customers or internal specific internal stakeholders based on complex user behavioral flows.

Access data anywhere vs controlled access

The post-COVID world has accelerated the shift towards work from home. This means internal users of your API analytics build may need to access metrics on home networks or on the go without a VPN. Placing unnecessary security restrictions to access the API data may limit the value your API analytics provides to your company or can even backfire creating bad habits such as password sharing or exporting large amounts of data to a personal device rather than using the API analytics build the way it was designed.

On the flip side, providing self-service API analytics from anywhere also means having strong security and access control. If you think typical enterprise access control like single sign on and role-based access control may be needed in the future, you should plan your API analytics build accordingly even if not needed immediately. Ripping out and changing authentication and authorization design is not an easy task and could force a complete rewrite. You don’t want to be caught off guard and be the engineer who failed at foreshadowing typical future enhancements.

Flexibility of visualizations

While most analytics platforms can display event data or plot basic single-value metrics over time, you platform may also be used for more advanced and specialized analysis like funnel analysis or cohort retention analysis. These are common queries used by business functions like marketing and growth teams but rarely used by developers themselves. Yet, building a funnel can be quote challenging if the right data model was not used.

Many times, you don’t know the types of queries that need to be displayed but should choose a project that enables flexibility in both the data model and the visualization layer.

Comparison

There are a variety of open-source analytics and monitoring projects out there. Some are focused on monitoring infrastructure metrics like Kibana and Grafana. While others are more focused on web analytics like Matomo (also known as Piwik). While none of these are designed for API products, you could develop custom code to piece a few components together to build an open-source API analytics platform

Kibana

Kibana is one of the de facto open-source log visualization tools out there for engineers. It’s part of the official ELK stack (Elasticsearch Logstash Kibana) which makes it one of the quickest visualization tools to get up and running due to it’s tight integration with Elasticsearch relative to tools like Grafana which are far more complex to set up. Elasticsearch itself is well suited for high-cardinality, high-dimension log data which is a must for API logs. The downside is that Kibana is only compatible with Elasticsearch. If you want to visualize data stored in a SQL database or other data store, you’ll need to look elsewhere.

While quick to set up, Kibana is also quite limited in the types of visualizations and flexibility supported. Kibana’s primary use case is to provide log search and light analysis on raw event data rather than offering a true API monitoring tool. For debugging use cases, this may be sufficient, but popular business metrics like funnels and retention analysis cannot be performed by Kibana limiting it’s application outside of engineering teams.

Kibana and Elasticsearch is designed to be paired with a Logstash instance which enables you to design a logging pipeline to process and enrich API logs such as normalized any HTTP headers or add geo IP information to each API call. Keep in mind Logstash cannot perform aggregations across multiple events at a time. Such processing requires a separate cluster-computing framework for map-reduce operations like Spark or Hadoop.

Kibana is designed for use cases where you want to explore your data in an ad hoc fashion rather than create a daily dashboard. You’re able to leverage Elasticsearch query DSL or Lucene query syntax providing great flexibility, but these do have a steep learning curve.

By default, Kibana is purely a visualization tool which means things like alerting, anomaly detection, and authentication are separate. This means anyone with access to your Kibana endpoint can access your data so you shouldn’t have it public.

However, you can purchase and install the Elasticsearch X-Pack to gain some monitoring functionality along with access control.

Grafana

Unlike Kibana which focuses on log search, Grafana focuses on time-series based metrics. You can visualize data in a variety of databases including Elasticsearch, InfluxDb, OpenTSDB, Graphite, and Prometheus. Grafana does one thing, and one thing really well, which is visualizing time series metrics stored in a database with beautiful dashboards. This does leave everything else up to you including configuring your data source and processing your data into a time series metric that can be displayed by Grafana.

Compared to Kibana, Grafana only works on time-series data already stored in a database and does not have any real-time log search nor a way to browse or explore your raw data in an ad hoc way. The primary use case for Grafana is to design a dashboard to monitor time-series metrics regularly such as on a TV in the office. For example, you may want to display disk utilization, system CPI, and requests per minute for your servers. Grafana has a lot of options to display your metrics in the way you want, like showing storage capacity using base 2 units and percentile based metrics.

Due to it’s time-series based architecture, Grafana’s application for high-cardinality, high-dimension analytics on API calls is limited. Instead, you need to spend time deciding exactly what specific time-series metrics you want to track ahead of time and model your data accordingly. This also limits Grafna’s use case for the self-service data exploration that business users may be looking for where you’re looking to segment with multiple group by’s or correlate multiple dimensions in your data.

Compared to Kibana, Grafana is known for built-in support for authentication and access control. You can also hook up your Grafana instance to an external Lightweight Directory Access Protocol (LDAP) server or SQL server to better control access in an enterprise setting.

You can also connect Grafana to an incident response platform like PagerDuty to create and trigger alerts from your Grafana instance. Keep in mind these alerts are limited to the same time series metrics you are already monitoring in Grafana.

Grafana does have a separate product called Loiki, which provides some of the log exploration features that Kibana has.

#kibana #logstash #grafana #api best practices #api monitoring #elasticseach #jaeger #jaeger tracing #api logs #kibana tutorial

Mike  Kozey

Mike Kozey

1620052800

Tracing in Grafana With Tempo and Jaeger

In this post, we will see how to introduce tracing in logs and visualize it easily. We will use Prometheus, Grafana Loki, Jaeger, and Grafana Tempo as data sources for monitoring metrics, logs, and traces respectively in Grafana.

Why Do I Need Tracing if I Have a Good Logging and Monitoring Framework?

Application logs are beneficial for displaying important events if something is not working as expected (failure, error, incorrect config, etc.). Although it is a very crucial element in application design, one should log thriftily. This is because log collection, transformation, and storage are costly.

Unlike logging, which is event triggered and discrete, tracing provides a broader and continuous application view. Tracing helps us understand the path of a process/transaction/entity while traversing the application stack and identifying the bottlenecks at various stages. This helps to optimize the application and increase performance.

In this post, we will see how to introduce tracing in logs and visualize it easily. In this example, we will use Prometheus, Grafana Loki, Jaeger, and Grafana Tempo as data sources for monitoring metrics, logs, and traces respectively in Grafana.

What Is Distributed Tracing?

In a microservices architecture, understanding an application behavior can be an intriguing task. This is because the incoming requests may span over multiple services, and each intermittent service may have one or more operations on that request. It thus increases complexity and takes more time while troubleshooting problems.

Distributed tracing helps to get insight into the individual operation and pinpoint the areas of failure caused by poor performance.

What Is OpenTracing?

OpenTracing comprises an API specification, frameworks, and libraries to enable distributed tracing in any application. OpenTracing APIs are very generic and prevents vendor/product lock-in. Recently, OpenTracing and OpenCensus merged to form  OpenTelemetry (acronym OTel). It targets the creation and management of telemetry data such as traces, metrics, and logs through a set of APIs, SDKs, tooling, and integrations. Note:  OpenCensus consists of a set of libs for various languages to collect metrics and traces from Applications, visualize them locally and send them remotely for storage and analysis.

#tutorial #microservices #monitoring #logging #grafana

Norbert  Ernser

Norbert Ernser

1595389260

MicroProfile OpenTracing With Jaeger

In this short video I demonstrate how to use MicroProfile OpenTracing with Jaeger in combination with Payara Micro.

The MicroProfile OpenTracing specification defines behaviors and an API for accessing an OpenTracing compliant Tracer object within your JAX-RS application. The behaviors specify how incoming and outgoing requests will have OpenTracing Spans automatically created. The API defines how to explicitly disable or enable tracing for given endpoints.

Jaeger is a distributed tracing system released as open source by Uber Technologies (inspired by Dapper and OpenZipkin). Utilized for monitoring and troubleshooting microservices-based distributed systems, including: Distributed context propagation, Distributed transaction monitoring, Root cause analysis, Service dependency analysis, Performance/latency optimization.

#java #microservice #microprofile #opentracing #jaeger #jaeger tracing

Norbert  Ernser

Norbert Ernser

1595353080

MicroProfile OpenTracing With Jaeger

In this short video I demonstrate how to use MicroProfile OpenTracing with Jaeger in combination with Payara Micro.

The MicroProfile OpenTracing specification defines behaviors and an API for accessing an OpenTracing compliant Tracer object within your JAX-RS application. The behaviors specify how incoming and outgoing requests will have OpenTracing Spans automatically created. The API defines how to explicitly disable or enable tracing for given endpoints.

Jaeger is a distributed tracing system released as open source by Uber Technologies (inspired by Dapper and OpenZipkin). Utilized for monitoring and troubleshooting microservices-based distributed systems, including: Distributed context propagation, Distributed transaction monitoring, Root cause analysis, Service dependency analysis, Performance/latency optimization.

#java #microservice #opentracing #jaeger #jaeger tracing