Sending InfluxDB Line Protocol to QuestDB

At QuestDB we’ve had a UDP version of the InfluxDB Line Protocol (ILP) reader in QuestDB for quite some time, but we’ve had customers ask for a TCP version of it, so we delivered!

Using it, and configuring it, are relatively simple so don’t expect this to be a long post but I’ll walk you through the basics of how to set it up and use it.

For an added bonus I’ll show you how to migrate from using InfluxDB to using QuestDB with a less than a line of configuration.

Configuring TCP InfluxDB Line Protocol Listener

Here’s the best part, at least for a basic implementation that you don’t need to performance tune at all: It’s already set up.

That’s right, as soon as you start QuestDB both the UDP and TCP ILP listeners start automatically on port 9009. Yes, TCP and UDP both use the same port. No, that’s not a problem since one is UDP and one is TCP.

There are a bunch of configuration options you can tune in your conf/server.conf file if you’re interested. I won’t go through them here, but you can read all about them in our docs. I hope they are relatively self-explanatory.

InfluxDB Line Protocol (ILP) Refresher

If you have used ILP before, this should all be review. If you’re new to ILP, this will tell you how you should write your data to QuestDB.

Basic Structure

TOML

1

table_name,tagset valueset timestamp

Pretty basic. So let’s dive into what each element actually is, and how to structure a line of ILP for writing.

The first element is the table_name portion, which tells the ILP writer which database table to write values into.

Next comes the set of tags you want to use. These are standard key=value pairs, and you can add as many of them as you want or need. Just separate them with commas.

There should only ever be 2 spaces in your line protocol. No more. The first space separates your tags from the values you want to associate with those tagss. The second space separates the values from the timestamp for those tags and values.

The values are also key=value pairs, and again you can send as many as you want in a line.

Finally comes your timestamp value, typically in µSeconds.

Example ILP

Let’s use an example of writing some environmental data to QuestDB. I have a sensor that reads temperature, atmospheric pressure, humidity, and the altitude.

ReadingValueTemperature23.180000Humidity51.982422Pressure1002.112061Altitude93.146370

And I want to use the following tags:

Tag NameTag Valuedev_idTHP002dev_locApexdev_nameBME280

And my table_name is iot

Now I have all the basic elements I need to construct my ILP, which will look like this:

iot,dev_id=THP002,dev_loc=Apex,dev_name=BME280 temp_c=23.18,altitude=93.10,humidity=52.16,pressure=1002.12

And yes, I rounded those values. But you’ll notice that I did not add a timestamp value. In this case, it’s because I am sending the values from a small, embedded sensor device that really doesn’t have a great sense of time. By sending the ILP without a timestamp I’m telling the database itself to add one for me, using the arrival time as the timestamp.

Database Structure

One of the cool features of using the ILP reader (well, QuestDB in general really) is the ability to do ‘Schema on Write’.

What that means is that if an ILP message arrives, QuestDB will automatically create tables and columns to fit the incoming ILP. So if you need to add a tag later, you can add it to the new device’s tagset and start writing. The new tag will get added to the schema.

If you leave a tag value off, and it exists in the database, it will get filled with a null value.

When I start writing the above ILP to QuestDB, I’ll get a table that looks like this:

dev_iddev_nametemp_chumiditytimestampdev_localtitudepressureTHP002BME28026.5251.942020-07-21T14:54:59.156202ZApex76.271004.12THP002BME28026.5451.852020-07-21T14:54:59.157358ZApex75.971004.16THP002BME28026.5651.832020-07-21T14:54:59.157389ZApex75.841004.17THP002BME28026.5851.792020-07-21T14:54:59.287416ZApex75.931004.16

This is what that table looks like in the QuestDB Web Console:

#tutorial #devops #iot #influxdb #observability

What is GEEK

Buddha Community

Sending InfluxDB Line Protocol to QuestDB
Grace  Lesch

Grace Lesch

1622708846

Schemaless Ingestion in QuestDB Using InfluxDB Line Protocol

For anyone who is new to QuestDB, it is a high performance open source time-series database. It has a wide range of use cases related to IoT, logging & application monitoring, financial trading and more. It uses a relational model with column-oriented storage, supports SQL and works best with append-only workloads.

#database #questdb #schemaless ingestion #influxdb #protocol

Python  Library

Python Library

1645053360

InfluxDB Python: Python Client for InfluxDB

InfluxDB-Python

InfluxDB-Python is a client for interacting with InfluxDB.

Note: This library is for use with InfluxDB 1.x. For connecting to InfluxDB 2.x instances, please use the the influxdb-client-python client.

Development of this library is maintained by:

Github IDURL
@aviau(https://github.com/aviau)
@xginn8(https://github.com/xginn8)
@sebito91(https://github.com/sebito91)

InfluxDB is an open-source distributed time series database, find more about InfluxDB at https://docs.influxdata.com/influxdb/latest

InfluxDB pre v1.1.0 users

This module is tested with InfluxDB versions: v1.2.4, v1.3.9, v1.4.3, v1.5.4, v1.6.4, and 1.7.4.

Those users still on InfluxDB v0.8.x users may still use the legacy client by importing from influxdb.influxdb08 import InfluxDBClient.

Installation

Install, upgrade and uninstall influxdb-python with these commands:

$ pip install influxdb
$ pip install --upgrade influxdb
$ pip uninstall influxdb

On Debian/Ubuntu, you can install it with this command:

$ sudo apt-get install python-influxdb

Dependencies

The influxdb-python distribution is supported and tested on Python 2.7, 3.5, 3.6, 3.7, PyPy and PyPy3.

Note: Python <3.5 are currently untested. See .travis.yml.

Main dependency is:

Additional dependencies are:

Documentation

Documentation is available at https://influxdb-python.readthedocs.io/en/latest/.

You will need Sphinx installed to generate the documentation.

The documentation can be generated by running:

$ tox -e docs

Generated documentation can be found in the docs/build/html/ directory.

Examples

Here's a basic example (for more see the examples directory):

$ python

>>> from influxdb import InfluxDBClient

>>> json_body = [
    {
        "measurement": "cpu_load_short",
        "tags": {
            "host": "server01",
            "region": "us-west"
        },
        "time": "2009-11-10T23:00:00Z",
        "fields": {
            "value": 0.64
        }
    }
]

>>> client = InfluxDBClient('localhost', 8086, 'root', 'root', 'example')

>>> client.create_database('example')

>>> client.write_points(json_body)

>>> result = client.query('select value from cpu_load_short;')

>>> print("Result: {0}".format(result))

Testing

Make sure you have tox by running the following:

$ pip install tox

To test influxdb-python with multiple version of Python, you can use Tox:

$ tox

Support

For issues with, questions about, or feedback for InfluxDB, please look into our community page: http://influxdb.com/community/.

We are also lurking on the following:

  • #influxdb on irc.freenode.net
  • #influxdb on gophers.slack.com

Development

All development is done on Github. Use Issues to report problems or submit contributions.

Please note that we WILL get to your questions/issues/concerns as quickly as possible. We maintain many software repositories and sometimes things may get pushed to the backburner. Please don't take offense, we will do our best to reply as soon as possible!

Source code

The source code is currently available on Github: https://github.com/influxdata/influxdb-python

TODO

The TODO/Roadmap can be found in Github bug tracker: https://github.com/influxdata/influxdb-python/issues

Download Details:
Author: influxdata
Source Code: https://github.com/influxdata/influxdb-python
License: MIT License

#python #InfluxDB

HI Python

HI Python

1640050218

InfluxDB-Python Library: Python Client for InfluxDB

InfluxDB-Python

InfluxDB-Python is a client for interacting with InfluxDB.

Development of this library is maintained by:

Github IDURL
@aviau(https://github.com/aviau)
@xginn8(https://github.com/xginn8)
@sebito91(https://github.com/sebito91)

InfluxDB is an open-source distributed time series database, find more about 

InfluxDB at https://docs.influxdata.com/influxdb/latest

InfluxDB pre v1.1.0 users

This module is tested with InfluxDB versions: v1.2.4, v1.3.9, v1.4.3, v1.5.4, v1.6.4, and 1.7.4.

Those users still on InfluxDB v0.8.x users may still use the legacy client by importing from influxdb.influxdb08 import InfluxDBClient.

Installation

Install, upgrade and uninstall influxdb-python with these commands:

$ pip install influxdb
$ pip install --upgrade influxdb
$ pip uninstall influxdb

On Debian/Ubuntu, you can install it with this command:

$ sudo apt-get install python-influxdb

Dependencies

The influxdb-python distribution is supported and tested on Python 2.7, 3.5, 3.6, 3.7, PyPy and PyPy3.

Note: Python <3.5 are currently untested. See .travis.yml.

Main dependency is:

Additional dependencies are:

Documentation

Documentation is available at https://influxdb-python.readthedocs.io/en/latest/.

You will need Sphinx installed to generate the documentation.

The documentation can be generated by running:

$ tox -e docs

Generated documentation can be found in the docs/build/html/ directory.

Examples

Here’s a basic example (for more see the examples directory):

$ python

>>> from influxdb import InfluxDBClient

>>> json_body = [
    {
        "measurement": "cpu_load_short",
        "tags": {
            "host": "server01",
            "region": "us-west"
        },
        "time": "2009-11-10T23:00:00Z",
        "fields": {
            "value": 0.64
        }
    }
]

>>> client = InfluxDBClient('localhost', 8086, 'root', 'root', 'example')

>>> client.create_database('example')

>>> client.write_points(json_body)

>>> result = client.query('select value from cpu_load_short;')

>>> print("Result: {0}".format(result))

Testing

Make sure you have tox by running the following:

$ pip install tox

To test influxdb-python with multiple version of Python, you can use Tox:

$ tox

Support

For issues with, questions about, or feedback for InfluxDB, please look into our community page: http://influxdb.com/community/.

We are also lurking on the following:

  • #influxdb on irc.freenode.net
  • #influxdb on gophers.slack.com

Development

All development is done on Github. Use Issues to report problems or submit contributions.

Please note that we WILL get to your questions/issues/concerns as quickly as possible. We maintain many software repositories and sometimes things may get pushed to the backburner. Please don’t take offense, we will do our best to reply as soon as possible!

TODO

The TODO/Roadmap can be found in Github bug tracker: https://github.com/influxdata/influxdb-python/issues

Download Details:
Author: influxdata
Source Code: https://github.com/influxdata/influxdb-python
License: MIT License

#python #influxdb 

Sending InfluxDB Line Protocol to QuestDB

At QuestDB we’ve had a UDP version of the InfluxDB Line Protocol (ILP) reader in QuestDB for quite some time, but we’ve had customers ask for a TCP version of it, so we delivered!

Using it, and configuring it, are relatively simple so don’t expect this to be a long post but I’ll walk you through the basics of how to set it up and use it.

For an added bonus I’ll show you how to migrate from using InfluxDB to using QuestDB with a less than a line of configuration.

Configuring TCP InfluxDB Line Protocol Listener

Here’s the best part, at least for a basic implementation that you don’t need to performance tune at all: It’s already set up.

That’s right, as soon as you start QuestDB both the UDP and TCP ILP listeners start automatically on port 9009. Yes, TCP and UDP both use the same port. No, that’s not a problem since one is UDP and one is TCP.

There are a bunch of configuration options you can tune in your conf/server.conf file if you’re interested. I won’t go through them here, but you can read all about them in our docs. I hope they are relatively self-explanatory.

InfluxDB Line Protocol (ILP) Refresher

If you have used ILP before, this should all be review. If you’re new to ILP, this will tell you how you should write your data to QuestDB.

Basic Structure

TOML

1

table_name,tagset valueset timestamp

Pretty basic. So let’s dive into what each element actually is, and how to structure a line of ILP for writing.

The first element is the table_name portion, which tells the ILP writer which database table to write values into.

Next comes the set of tags you want to use. These are standard key=value pairs, and you can add as many of them as you want or need. Just separate them with commas.

There should only ever be 2 spaces in your line protocol. No more. The first space separates your tags from the values you want to associate with those tagss. The second space separates the values from the timestamp for those tags and values.

The values are also key=value pairs, and again you can send as many as you want in a line.

Finally comes your timestamp value, typically in µSeconds.

Example ILP

Let’s use an example of writing some environmental data to QuestDB. I have a sensor that reads temperature, atmospheric pressure, humidity, and the altitude.

ReadingValueTemperature23.180000Humidity51.982422Pressure1002.112061Altitude93.146370

And I want to use the following tags:

Tag NameTag Valuedev_idTHP002dev_locApexdev_nameBME280

And my table_name is iot

Now I have all the basic elements I need to construct my ILP, which will look like this:

iot,dev_id=THP002,dev_loc=Apex,dev_name=BME280 temp_c=23.18,altitude=93.10,humidity=52.16,pressure=1002.12

And yes, I rounded those values. But you’ll notice that I did not add a timestamp value. In this case, it’s because I am sending the values from a small, embedded sensor device that really doesn’t have a great sense of time. By sending the ILP without a timestamp I’m telling the database itself to add one for me, using the arrival time as the timestamp.

Database Structure

One of the cool features of using the ILP reader (well, QuestDB in general really) is the ability to do ‘Schema on Write’.

What that means is that if an ILP message arrives, QuestDB will automatically create tables and columns to fit the incoming ILP. So if you need to add a tag later, you can add it to the new device’s tagset and start writing. The new tag will get added to the schema.

If you leave a tag value off, and it exists in the database, it will get filled with a null value.

When I start writing the above ILP to QuestDB, I’ll get a table that looks like this:

dev_iddev_nametemp_chumiditytimestampdev_localtitudepressureTHP002BME28026.5251.942020-07-21T14:54:59.156202ZApex76.271004.12THP002BME28026.5451.852020-07-21T14:54:59.157358ZApex75.971004.16THP002BME28026.5651.832020-07-21T14:54:59.157389ZApex75.841004.17THP002BME28026.5851.792020-07-21T14:54:59.287416ZApex75.931004.16

This is what that table looks like in the QuestDB Web Console:

#tutorial #devops #iot #influxdb #observability

Desmond Ivana

1595572272

What are the features of the Line clone messaging app?

The Line is a Japan-based text messaging app. It has over 250 million users worldwide, with around two-thirds of them based on Japan, Taiwan, Indonesia, and Thailand. The main perk of investing in the instant messaging app is that it gets popular within the few days of its launch. Here are the essential features of Line clone messaging app:

Profile: Users can customize their profiles, add a name, edit display picture, etc. Apart from that, an app like Line includes several privacy settings such as regulating the viewers for last seen, profile pic, status, etc.

Chat options: There are two kinds of chats available - individual chats and group chats. The individual conversations refer to one-on-one chat. A group chat allows upto 200 members in a single group. People can share videos, files, pictures, GIFs, stickers in addition to messages.

Video/voice calls: Line currently allows upto 200 members in a group call. Users can directly click on the video chat icon on the group they wish to connect and start the video call. It would be very beneficial for people as it allows conference calls with so many participants.

Security: The messages are protected so that any third party cannot view them. If the users do not feel safe with communicating with a particular user, they can block them. Then, they can send them messages.

Appdupe offers a messaging app script with all the above-mentioned salient features. Get the clone app from us, see the business growth it offers.

#line clone app development #app like line #line clone script #messaging app #line clone #line clone app