Arno Bradtke

Arno Bradtke

1598110560

Jaeger’s multitenancy with Elasticsearch

For the past year, due to the need of distributed transaction monitoring and root cause analysis in a complex distributed micro-service environment, we introduced Jaeger framework to help us tackle the problem. Since our platform is being used by multiple tenants, we had to take a decision on how we would implement the multi-tenancy Jaeger with Elasticsearch as backend.

This is a practical exercise on how to setup Jaeger with Elasticsearch to support multiple tenants. But first, you should read the following article Jaeger and multitenancy which talks about various multi-tenancy options with Jaeger.

The context

We are building and running a platform based on Kubernetes, which allows our customers to build and deploy their own applications using our platform, thus the specific requirements when it comes to tracing data:

  • One Elasticsearch instance supporting all the tenants,
  • Each tenant’s trace data has to be persisted separately — with various retention timeframes,
  • Ability for every tenant to view and query its own tracing data,
  • As minimal development activities as possible, thus reusing the existing Jaeger functionalities.

The solution

After going through enough material from different sources to have a clear picture, I decided on the following solution which consists of:

  • An Elasticsearch instance installed in an independent namespace of the tenants ones — we want to manage our own ES cluster,
  • Install a Jaeger collector for each tenant, configured to use the ES cluster with the specific tenant name/id,
  • Install the Jaeger agents as sidecars to the services that are being traced,
  • For each tenant configure it’s own Jaeger’s elastic search index cleaner settings,
  • Everything set up with 0 development effort.

#multitenancy #kubernetes #helm #jaeger #elasticsearch

What is GEEK

Buddha Community

Jaeger’s multitenancy with Elasticsearch
Mike Kozey

Mike Kozey

1656151740

Test_cov_console: Flutter Console Coverage Test

Flutter Console Coverage Test

This small dart tools is used to generate Flutter Coverage Test report to console

How to install

Add a line like this to your package's pubspec.yaml (and run an implicit flutter pub get):

dev_dependencies:
  test_cov_console: ^0.2.2

How to run

run the following command to make sure all flutter library is up-to-date

flutter pub get
Running "flutter pub get" in coverage...                            0.5s

run the following command to generate lcov.info on coverage directory

flutter test --coverage
00:02 +1: All tests passed!

run the tool to generate report from lcov.info

flutter pub run test_cov_console
---------------------------------------------|---------|---------|---------|-------------------|
File                                         |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
 print_cov_constants.dart                    |    0.00 |    0.00 |    0.00 |    no unit testing|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|

Optional parameter

If not given a FILE, "coverage/lcov.info" will be used.
-f, --file=<FILE>                      The target lcov.info file to be reported
-e, --exclude=<STRING1,STRING2,...>    A list of contains string for files without unit testing
                                       to be excluded from report
-l, --line                             It will print Lines & Uncovered Lines only
                                       Branch & Functions coverage percentage will not be printed
-i, --ignore                           It will not print any file without unit testing
-m, --multi                            Report from multiple lcov.info files
-c, --csv                              Output to CSV file
-o, --output=<CSV-FILE>                Full path of output CSV file
                                       If not given, "coverage/test_cov_console.csv" will be used
-t, --total                            Print only the total coverage
                                       Note: it will ignore all other option (if any), except -m
-p, --pass=<MINIMUM>                   Print only the whether total coverage is passed MINIMUM value or not
                                       If the value >= MINIMUM, it will print PASSED, otherwise FAILED
                                       Note: it will ignore all other option (if any), except -m
-h, --help                             Show this help

example run the tool with parameters

flutter pub run test_cov_console --file=coverage/lcov.info --exclude=_constants,_mock
---------------------------------------------|---------|---------|---------|-------------------|
File                                         |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|

report for multiple lcov.info files (-m, --multi)

It support to run for multiple lcov.info files with the followings directory structures:
1. No root module
<root>/<module_a>
<root>/<module_a>/coverage/lcov.info
<root>/<module_a>/lib/src
<root>/<module_b>
<root>/<module_b>/coverage/lcov.info
<root>/<module_b>/lib/src
...
2. With root module
<root>/coverage/lcov.info
<root>/lib/src
<root>/<module_a>
<root>/<module_a>/coverage/lcov.info
<root>/<module_a>/lib/src
<root>/<module_b>
<root>/<module_b>/coverage/lcov.info
<root>/<module_b>/lib/src
...
You must run test_cov_console on <root> dir, and the report would be grouped by module, here is
the sample output for directory structure 'with root module':
flutter pub run test_cov_console --file=coverage/lcov.info --exclude=_constants,_mock --multi
---------------------------------------------|---------|---------|---------|-------------------|
File                                         |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|
---------------------------------------------|---------|---------|---------|-------------------|
File - module_a -                            |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|
---------------------------------------------|---------|---------|---------|-------------------|
File - module_b -                            |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|

Output to CSV file (-c, --csv, -o, --output)

flutter pub run test_cov_console -c --output=coverage/test_coverage.csv

#### sample CSV output file:
File,% Branch,% Funcs,% Lines,Uncovered Line #s
lib/,,,,
test_cov_console.dart,0.00,0.00,0.00,no unit testing
lib/src/,,,,
parser.dart,100.00,100.00,97.22,"97"
parser_constants.dart,100.00,100.00,100.00,""
print_cov.dart,100.00,100.00,82.91,"29,49,51,52,171,174,177,180,183,184,185,186,187,188,279,324,325,387,388,389,390,391,392,393,394,395,398"
print_cov_constants.dart,0.00,0.00,0.00,no unit testing
All files with unit testing,100.00,100.00,86.07,""

Installing

Use this package as an executable

Install it

You can install the package from the command line:

dart pub global activate test_cov_console

Use it

The package has the following executables:

$ test_cov_console

Use this package as a library

Depend on it

Run this command:

With Dart:

 $ dart pub add test_cov_console

With Flutter:

 $ flutter pub add test_cov_console

This will add a line like this to your package's pubspec.yaml (and run an implicit dart pub get):

dependencies:
  test_cov_console: ^0.2.2

Alternatively, your editor might support dart pub get or flutter pub get. Check the docs for your editor to learn more.

Import it

Now in your Dart code, you can use:

import 'package:test_cov_console/test_cov_console.dart';

example/lib/main.dart

import 'package:flutter/material.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        // This is the theme of your application.
        //
        // Try running your application with "flutter run". You'll see the
        // application has a blue toolbar. Then, without quitting the app, try
        // changing the primarySwatch below to Colors.green and then invoke
        // "hot reload" (press "r" in the console where you ran "flutter run",
        // or simply save your changes to "hot reload" in a Flutter IDE).
        // Notice that the counter didn't reset back to zero; the application
        // is not restarted.
        primarySwatch: Colors.blue,
        // This makes the visual density adapt to the platform that you run
        // the app on. For desktop platforms, the controls will be smaller and
        // closer together (more dense) than on mobile platforms.
        visualDensity: VisualDensity.adaptivePlatformDensity,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key? key, required this.title}) : super(key: key);

  // This widget is the home page of your application. It is stateful, meaning
  // that it has a State object (defined below) that contains fields that affect
  // how it looks.

  // This class is the configuration for the state. It holds the values (in this
  // case the title) provided by the parent (in this case the App widget) and
  // used by the build method of the State. Fields in a Widget subclass are
  // always marked "final".

  final String title;

  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;

  void _incrementCounter() {
    setState(() {
      // This call to setState tells the Flutter framework that something has
      // changed in this State, which causes it to rerun the build method below
      // so that the display can reflect the updated values. If we changed
      // _counter without calling setState(), then the build method would not be
      // called again, and so nothing would appear to happen.
      _counter++;
    });
  }

  @override
  Widget build(BuildContext context) {
    // This method is rerun every time setState is called, for instance as done
    // by the _incrementCounter method above.
    //
    // The Flutter framework has been optimized to make rerunning build methods
    // fast, so that you can just rebuild anything that needs updating rather
    // than having to individually change instances of widgets.
    return Scaffold(
      appBar: AppBar(
        // Here we take the value from the MyHomePage object that was created by
        // the App.build method, and use it to set our appbar title.
        title: Text(widget.title),
      ),
      body: Center(
        // Center is a layout widget. It takes a single child and positions it
        // in the middle of the parent.
        child: Column(
          // Column is also a layout widget. It takes a list of children and
          // arranges them vertically. By default, it sizes itself to fit its
          // children horizontally, and tries to be as tall as its parent.
          //
          // Invoke "debug painting" (press "p" in the console, choose the
          // "Toggle Debug Paint" action from the Flutter Inspector in Android
          // Studio, or the "Toggle Debug Paint" command in Visual Studio Code)
          // to see the wireframe for each widget.
          //
          // Column has various properties to control how it sizes itself and
          // how it positions its children. Here we use mainAxisAlignment to
          // center the children vertically; the main axis here is the vertical
          // axis because Columns are vertical (the cross axis would be
          // horizontal).
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Text(
              'You have pushed the button this many times:',
            ),
            Text(
              '$_counter',
              style: Theme.of(context).textTheme.headline4,
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: _incrementCounter,
        tooltip: 'Increment',
        child: Icon(Icons.add),
      ), // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}

Author: DigitalKatalis
Source Code: https://github.com/DigitalKatalis/test_cov_console 
License: BSD-3-Clause license

#flutter #dart #test 

Arno Bradtke

Arno Bradtke

1598110560

Jaeger’s multitenancy with Elasticsearch

For the past year, due to the need of distributed transaction monitoring and root cause analysis in a complex distributed micro-service environment, we introduced Jaeger framework to help us tackle the problem. Since our platform is being used by multiple tenants, we had to take a decision on how we would implement the multi-tenancy Jaeger with Elasticsearch as backend.

This is a practical exercise on how to setup Jaeger with Elasticsearch to support multiple tenants. But first, you should read the following article Jaeger and multitenancy which talks about various multi-tenancy options with Jaeger.

The context

We are building and running a platform based on Kubernetes, which allows our customers to build and deploy their own applications using our platform, thus the specific requirements when it comes to tracing data:

  • One Elasticsearch instance supporting all the tenants,
  • Each tenant’s trace data has to be persisted separately — with various retention timeframes,
  • Ability for every tenant to view and query its own tracing data,
  • As minimal development activities as possible, thus reusing the existing Jaeger functionalities.

The solution

After going through enough material from different sources to have a clear picture, I decided on the following solution which consists of:

  • An Elasticsearch instance installed in an independent namespace of the tenants ones — we want to manage our own ES cluster,
  • Install a Jaeger collector for each tenant, configured to use the ES cluster with the specific tenant name/id,
  • Install the Jaeger agents as sidecars to the services that are being traced,
  • For each tenant configure it’s own Jaeger’s elastic search index cleaner settings,
  • Everything set up with 0 development effort.

#multitenancy #kubernetes #helm #jaeger #elasticsearch

Rusty Shanahan

Rusty Shanahan

1598155740

Elasticsearch 7.x Backup — “Snapshot & Restore” on AWS S3

In 2016 I wrote an Article about Elasticsearch Backup, it had and still has quite good interests from people. I decided to start a new series of articles with the Backup topic as the main argument.

The old article covered Snapshot & Restore functionalities based on Elasticsearch 2.4.x and the upcoming version, the 5.0. As it was 4 years ago I choose to refresh this tutorial and making it the first of a series of more.

I will prepare a small article on how to use the snapshot & restore functionality with different cloud-provider. This article is based on Elasticsearch 7.x, it doesn’t mean it couldn’t work on older versions but I focused on the latest one.

Elasticsearch Snapshot & Restore

Elasticsearch has a smart solution to backup single indices or entire clusters to remote shared filesystem or S3 or HDFS. The snapshot ES creates does not so resource consuming and is relatively small.

The idea behind these snapshots is that they are not “archive” in a strict sense, these snapshots can only be read by a version of Elasticsearch that is capable to read the index version stored inside the snapshot.

So you can follow this quick scheme if you want to restore ES snapshots :

  • A snapshot of an index created in 6.x can be restored to 7.x.
  • A snapshot of an index created in 5.x can be restored to 6.x.
  • A snapshot of an index created in 2.x can be restored to 5.x.
  • A snapshot of an index created in 1.x can be restored to 2.x.

Snapshots of indices created with ES 1.x cannot be restored to 5.x or 6.x, snapshots of indices created in 2.x cannot be restored to 6.x or 7.x, and snapshots of indices created in 5.x cannot be restored to 7.x or 8.x.

#elasticsearch-snapshot #elasticsearch-plugins #elasticsearch #backup #elasticsearch-backup #aws

田辺 亮介

田辺 亮介

1661842560

輕鬆部署 Jaeger 和 Elasticsearch

什麼是 Docker Compose？

Compose是一個用於定義和運行多容器 Docker 應用程序的工具。使用 Compose，您可以使用 YAML 文件來配置應用程序的服務。然後，使用一個命令，您可以從您的配置中創建並啟動所有服務。

為什麼使用 Elasticsearch 作為存儲後端？

在生產策略中，建議使用持久存儲後端。請記住，使用內存存儲組件（即 All-in-One Jaeger 映像）是為本地測試而設計的。

讓我們更新我們的docker-compose文件，看看我們如何集成Jaeger 以使用 Elasticsearch。

如前所述，您的應用程序必須先進行檢測，然後才能將跟踪數據發送到 Jaeger 後端。因此，我們將繼續使用演示應用程序（Hot ROD）。

如何使用 Docker Compose 部署 Jaeger & HotROD & Elasticsearch

創建一個名為的新文件docker-compose.yml

將其複制並粘貼到文件中：

version: "3"

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.9.3
    networks:
      - elastic-jaeger
    ports:
      - "127.0.0.1:9200:9200"
      - "127.0.0.1:9300:9300"
    restart: on-failure
    environment:
      - cluster.name=jaeger-cluster
      - discovery.type=single-node
      - http.host=0.0.0.0
      - transport.host=127.0.0.1
      - ES_JAVA_OPTS=-Xms512m -Xmx512m
      - xpack.security.enabled=false
    volumes:
      - esdata:/usr/share/elasticsearch/data

  jaeger-collector:
    image: jaegertracing/jaeger-collector
    ports:
      - "14269:14269"
      - "14268:14268"
      - "14267:14267"
      - "14250:14250"
      - "9411:9411"
    networks:
      - elastic-jaeger
    restart: on-failure
    environment:
      - SPAN_STORAGE_TYPE=elasticsearch
    command: [
      "--es.server-urls=http://elasticsearch:9200",
      "--es.num-shards=1",
      "--es.num-replicas=0",
      "--log-level=error"
    ]
    depends_on:
      - elasticsearch

  jaeger-agent:
    image: jaegertracing/jaeger-agent
    hostname: jaeger-agent
    command: ["--reporter.grpc.host-port=jaeger-collector:14250"]
    ports:
      - "5775:5775/udp"
      - "6831:6831/udp"
      - "6832:6832/udp"
      - "5778:5778"
    networks:
      - elastic-jaeger
    restart: on-failure
    environment:
      - SPAN_STORAGE_TYPE=elasticsearch
    depends_on:
      - jaeger-collector

  jaeger-query:
    image: jaegertracing/jaeger-query
    environment:
      - SPAN_STORAGE_TYPE=elasticsearch
      - no_proxy=localhost
    ports:
      - "16686:16686"
      - "16687:16687"
    networks:
      - elastic-jaeger
    restart: on-failure
    command: [
      "--es.server-urls=http://elasticsearch:9200",
      "--span-storage.type=elasticsearch",
      "--log-level=debug"
    ]
    depends_on:
      - jaeger-agent

  hotrod:
    image: jaegertracing/example-hotrod:latest
    ports: 
      - "8080:8080"
    command: ["all"]
    environment:
      - JAEGER_AGENT_HOST=jaeger-agent
      # Note: if your application is using Node.js Jaeger Client, you need port 6832,
      #       unless issue https://github.com/jaegertracing/jaeger/issues/1596 is resolved.
      - JAEGER_AGENT_PORT=6831
    networks:
      - elastic-jaeger
    depends_on:
      - jaeger

volumes:
  esdata:
    driver: local

networks:
  elastic-jaeger:
    driver: bridge

此docker-compose.yml文件部署：

  • 積家代理
  • 積家收藏家
  • 積家查詢
  • 熱棒
  • 彈性搜索

啟動所有服務：docker-compose up

注意：最初我在讓它工作時遇到了一些錯誤，但這個stackoverflow帖子解決了它。

所有容器都運行後，您可以通過鍵入以下內容訪問 Elasticsearch 集群：

curl -u elastic:changeme localhost:9200

{
  "name" : "37c3e1d2ed3b",
  "cluster_name" : "jaeger-cluster",
  "cluster_uuid" : "v0r0tQKFTz-W_65FgX5VfQ",
  "version" : {
    "number" : "7.9.3",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "c4138e51121ef06a6404866cddc601906fe5c868",
    "build_date" : "2020-10-16T10:36:16.141335Z",
    "build_snapshot" : false,
    "lucene_version" : "8.6.2",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

訪問HotROD 應用程序

通過單擊任何按鈕向演示應用程序 (HotROD) 應用程序發送一些用戶請求。

再次運行該命令幾次以查看跟踪是否進入了 Elasticsearch 集群。

集群中所有索引的列表

curl -u "elastic:changeme" -X GET 'http://localhost:9200/_cat/indices?v'

health status index                     uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   jaeger-span-2022-07-24    F_H_9owpTzy-fFoJyKL3jg   1   0        953            0       56kb           56kb
green  open   jaeger-service-2022-07-24 022T1BQlTlan6TnPMlKhow   1   0         13           38      9.9kb          9.9kb

查看健康狀況

curl -u "elastic:changeme" -H 'Content-Type: application/json' -XGET http://localhost:9200/_cluster/health\?pretty

{
  "cluster_name" : "jaeger-cluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 2,
  "active_shards" : 2,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

在這一部分中，我們添加了 Elasticsearch。Elasticsearch 是用於跟踪的推薦存儲後端之一。

鏈接：https ://faun.pub/how-to-deploy-jaeger-and-elasticsearch-bf326e774cc8

#jaeger #elasticsearch #go

August Larson

August Larson

1661964720

Deploying Jaeger and Elasticsearch Easily

What is Docker Compose?

Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration.

Why use Elasticsearch as a storage backend?

In a production strategy, its recommended to use a persistent storage backend. Remember, using an in-memory storage component (that the All-in-One Jaeger image) provides is designed for local testing.

Let’s update our docker-compose file to see how we can integrate Jaeger to use Elasticsearch.

As mentioned earlier, your applications must be instrumented before they can send tracing data to Jaeger backend. As such, we will continue to use the demo application (Hot ROD).

How to deploy Jaeger & HotROD & Elasticsearch with Docker Compose

Create a new file called docker-compose.yml

Copy and paste this to the file:

version: "3"

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.9.3
    networks:
      - elastic-jaeger
    ports:
      - "127.0.0.1:9200:9200"
      - "127.0.0.1:9300:9300"
    restart: on-failure
    environment:
      - cluster.name=jaeger-cluster
      - discovery.type=single-node
      - http.host=0.0.0.0
      - transport.host=127.0.0.1
      - ES_JAVA_OPTS=-Xms512m -Xmx512m
      - xpack.security.enabled=false
    volumes:
      - esdata:/usr/share/elasticsearch/data

  jaeger-collector:
    image: jaegertracing/jaeger-collector
    ports:
      - "14269:14269"
      - "14268:14268"
      - "14267:14267"
      - "14250:14250"
      - "9411:9411"
    networks:
      - elastic-jaeger
    restart: on-failure
    environment:
      - SPAN_STORAGE_TYPE=elasticsearch
    command: [
      "--es.server-urls=http://elasticsearch:9200",
      "--es.num-shards=1",
      "--es.num-replicas=0",
      "--log-level=error"
    ]
    depends_on:
      - elasticsearch

  jaeger-agent:
    image: jaegertracing/jaeger-agent
    hostname: jaeger-agent
    command: ["--reporter.grpc.host-port=jaeger-collector:14250"]
    ports:
      - "5775:5775/udp"
      - "6831:6831/udp"
      - "6832:6832/udp"
      - "5778:5778"
    networks:
      - elastic-jaeger
    restart: on-failure
    environment:
      - SPAN_STORAGE_TYPE=elasticsearch
    depends_on:
      - jaeger-collector

  jaeger-query:
    image: jaegertracing/jaeger-query
    environment:
      - SPAN_STORAGE_TYPE=elasticsearch
      - no_proxy=localhost
    ports:
      - "16686:16686"
      - "16687:16687"
    networks:
      - elastic-jaeger
    restart: on-failure
    command: [
      "--es.server-urls=http://elasticsearch:9200",
      "--span-storage.type=elasticsearch",
      "--log-level=debug"
    ]
    depends_on:
      - jaeger-agent

  hotrod:
    image: jaegertracing/example-hotrod:latest
    ports: 
      - "8080:8080"
    command: ["all"]
    environment:
      - JAEGER_AGENT_HOST=jaeger-agent
      # Note: if your application is using Node.js Jaeger Client, you need port 6832,
      #       unless issue https://github.com/jaegertracing/jaeger/issues/1596 is resolved.
      - JAEGER_AGENT_PORT=6831
    networks:
      - elastic-jaeger
    depends_on:
      - jaeger

volumes:
  esdata:
    driver: local

networks:
  elastic-jaeger:
    driver: bridge

This docker-compose.yml file deploys:

  • Jaeger Agent
  • Jaeger Collector
  • Jaeger Query
  • Hot ROD
  • Elasticsearch

Start all services with : docker-compose up

Note: Initially I had some errors getting it to work, but this stackoverflow post solved it.

Once all the containers are running, you can access the Elasticsearch cluster by typing:

curl -u elastic:changeme localhost:9200

{
  "name" : "37c3e1d2ed3b",
  "cluster_name" : "jaeger-cluster",
  "cluster_uuid" : "v0r0tQKFTz-W_65FgX5VfQ",
  "version" : {
    "number" : "7.9.3",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "c4138e51121ef06a6404866cddc601906fe5c868",
    "build_date" : "2020-10-16T10:36:16.141335Z",
    "build_snapshot" : false,
    "lucene_version" : "8.6.2",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Access HotROD app

Send a few user requests to the demo app (HotROD) application by clicking on any of the buttons.

Run the command again a few times to see that the traces are coming into the Elasticsearch cluster.

List of all indices in your cluster

curl -u "elastic:changeme" -X GET 'http://localhost:9200/_cat/indices?v'

health status index                     uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   jaeger-span-2022-07-24    F_H_9owpTzy-fFoJyKL3jg   1   0        953            0       56kb           56kb
green  open   jaeger-service-2022-07-24 022T1BQlTlan6TnPMlKhow   1   0         13           38      9.9kb          9.9kb

View the health

curl -u "elastic:changeme" -H 'Content-Type: application/json' -XGET http://localhost:9200/_cluster/health\?pretty

{
  "cluster_name" : "jaeger-cluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 2,
  "active_shards" : 2,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

In this part, we added Elasticsearch to the mix. Elasticsearch is one of the recommend storage backend for tracing.

Link: https://faun.pub/how-to-deploy-jaeger-and-elasticsearch-bf326e774cc8

#jaeger #elasticsearch #go