iOS App Dev

iOS App Dev

1643381040

TDengine: Open-source Big Data Platform Designed and Optimized for IOT

What is TDengine?

TDengine is an open-sourced big data platform under GNU AGPL v3.0, designed and optimized for the Internet of Things (IoT), Connected Cars, Industrial IoT, and IT Infrastructure and Application Monitoring. Besides the 10x faster time-series database, it provides caching, stream computing, message queuing and other functionalities to reduce the complexity and cost of development and operation.

  • 10x Faster on Insert/Query Speeds: Through the innovative design on storage, on a single-core machine, over 20K requests can be processed, millions of data points can be ingested, and over 10 million data points can be retrieved in a second. It is 10 times faster than other databases.
  • 1/5 Hardware/Cloud Service Costs: Compared with typical big data solutions, less than 1/5 of computing resources are required. Via column-based storage and tuned compression algorithms for different data types, less than 1/10 of storage space is needed.
  • Full Stack for Time-Series Data: By integrating a database with message queuing, caching, and stream computing features together, it is no longer necessary to integrate Kafka/Redis/HBase/Spark or other software. It makes the system architecture much simpler and more robust.
  • Powerful Data Analysis: Whether it is 10 years or one minute ago, data can be queried just by specifying the time range. Data can be aggregated over time, multiple time streams or both. Ad Hoc queries or analyses can be executed via TDengine shell, Python, R or Matlab.
  • Seamless Integration with Other Tools: Telegraf, Grafana, Matlab, R, and other tools can be integrated with TDengine without a line of code. MQTT, OPC, Hadoop, Spark, and many others will be integrated soon.
  • Zero Management, No Learning Curve: It takes only seconds to download, install, and run it successfully; there are no other dependencies. Automatic partitioning on tables or DBs. Standard SQL is used, with C/C++, Python, JDBC, Go and RESTful connectors.

Documentation

For user manual, system design and architecture, engineering blogs, refer to TDengine Documentation(中文版请点击这里) for details. The documentation from our website can also be downloaded locally from documentation/tdenginedocs-en or documentation/tdenginedocs-cn.

Building

At the moment, TDengine only supports building and running on Linux systems. You can choose to install from packages or from the source code. This quick guide is for installation from the source only.

To build TDengine, use CMake 2.8.12.x or higher versions in the project directory.

Install tools

Ubuntu 16.04 and above & Debian:

sudo apt-get install -y gcc cmake build-essential git

Ubuntu 14.04:

sudo apt-get install -y gcc cmake3 build-essential git binutils-2.26
export PATH=/usr/lib/binutils-2.26/bin:$PATH

To compile and package the JDBC driver source code, you should have a Java jdk-8 or higher and Apache Maven 2.7 or higher installed. To install openjdk-8:

sudo apt-get install -y openjdk-8-jdk

To install Apache Maven:

sudo apt-get install -y  maven

Centos 7:

sudo yum install -y gcc gcc-c++ make cmake git

To install openjdk-8:

sudo yum install -y java-1.8.0-openjdk

To install Apache Maven:

sudo yum install -y maven

Centos 8 & Fedora:

sudo dnf install -y gcc gcc-c++ make cmake epel-release git

To install openjdk-8:

sudo dnf install -y java-1.8.0-openjdk

To install Apache Maven:

sudo dnf install -y maven

Get the source codes

First of all, you may clone the source codes from github:

git clone https://github.com/taosdata/TDengine.git
cd TDengine

The connectors for go & grafana have been moved to separated repositories, so you should run this command in the TDengine directory to install them:

git submodule update --init --recursive

Build TDengine

On Linux platform

mkdir debug && cd debug
cmake .. && cmake --build .

You can use Jemalloc as memory allocator instead of glibc:

apt install autoconf
cmake .. -DJEMALLOC_ENABLED=true

TDengine build script can detect the host machine's architecture on X86-64, X86, arm64, arm32 and mips64 platform. You can also specify CPUTYPE option like aarch64 or aarch32 too if the detection result is not correct:

aarch64:

cmake .. -DCPUTYPE=aarch64 && cmake --build .

aarch32:

cmake .. -DCPUTYPE=aarch32 && cmake --build .

mips64:

cmake .. -DCPUTYPE=mips64 && cmake --build .

On Windows platform

If you use the Visual Studio 2013, please open a command window by executing "cmd.exe". Please specify "amd64" for 64 bits Windows or specify "x86" is for 32 bits Windows when you execute vcvarsall.bat.

mkdir debug && cd debug
"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\vcvarsall.bat" < amd64 | x86 >
cmake .. -G "NMake Makefiles"
nmake

If you use the Visual Studio 2019 or 2017:

please open a command window by executing "cmd.exe". Please specify "x64" for 64 bits Windows or specify "x86" is for 32 bits Windows when you execute vcvarsall.bat.

mkdir debug && cd debug
"c:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" < x64 | x86 >
cmake .. -G "NMake Makefiles"
nmake

Or, you can simply open a command window by clicking Windows Start -> "Visual Studio < 2019 | 2017 >" folder -> "x64 Native Tools Command Prompt for VS < 2019 | 2017 >" or "x86 Native Tools Command Prompt for VS < 2019 | 2017 >" depends what architecture your Windows is, then execute commands as follows:

mkdir debug && cd debug
cmake .. -G "NMake Makefiles"
nmake

On Mac OS X platform

Please install XCode command line tools and cmake. Verified with XCode 11.4+ on Catalina and Big Sur.

mkdir debug && cd debug
cmake .. && cmake --build .

Installing

After building successfully, TDengine can be installed by: (On Windows platform, the following command should be nmake install)

sudo make install

Users can find more information about directories installed on the system in the directory and files section. Since version 2.0, installing from source code will also configure service management for TDengine. Users can also choose to install from packages for it.

To start the service after installation, in a terminal, use:

sudo systemctl start taosd

Then users can use the TDengine shell to connect the TDengine server. In a terminal, use:

taos

If TDengine shell connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown.

Quick Run

If you don't want to run TDengine as a service, you can run it in current shell. For example, to quickly start a TDengine server after building, run the command below in terminal: (We take Linux as an example, command on Windows will be taosd.exe)

./build/bin/taosd -c test/cfg

In another terminal, use the TDengine shell to connect the server:

./build/bin/taos -c test/cfg

option "-c test/cfg" specifies the system configuration file directory.

Try TDengine

It is easy to run SQL commands from TDengine shell which is the same as other SQL databases.

create database db;
use db;
create table t (ts timestamp, a int);
insert into t values ('2019-07-15 00:00:00', 1);
insert into t values ('2019-07-15 01:00:00', 2);
select * from t;
drop database db;

Developing with TDengine

Official Connectors

TDengine provides abundant developing tools for users to develop on TDengine. Follow the links below to find your desired connectors and relevant documentation.

Third Party Connectors

The TDengine community has also kindly built some of their own connectors! Follow the links below to find the source code for them.

How to run the test cases and how to add a new test case?

TDengine's test framework and all test cases are fully open source. Please refer to this document for how to run test and develop new test case.

TDengine Roadmap

  • Support event-driven stream computing
  • Support user defined functions
  • Support MQTT connection
  • Support OPC connection
  • Support Hadoop, Spark connections
  • Support Tableau and other BI tools

Contribute to TDengine

Please follow the contribution guidelines to contribute to the project.

Join TDengine WeChat Group

Add WeChat “tdengine” to join the group,you can communicate with other users.

User List

If you are using TDengine and feel it helps or you'd like to do some contributions, please add your company to user list and let us know your needs.

Download Details:
Author: taosdata
Source Code: https://github.com/taosdata/TDengine
License: AGPL-3.0 License

#big-data #iot #python #c #database 

What is GEEK

Buddha Community

TDengine: Open-source Big Data Platform Designed and Optimized for IOT
 iOS App Dev

iOS App Dev

1620466520

Your Data Architecture: Simple Best Practices for Your Data Strategy

If you accumulate data on which you base your decision-making as an organization, you should probably think about your data architecture and possible best practices.

If you accumulate data on which you base your decision-making as an organization, you most probably need to think about your data architecture and consider possible best practices. Gaining a competitive edge, remaining customer-centric to the greatest extent possible, and streamlining processes to get on-the-button outcomes can all be traced back to an organization’s capacity to build a future-ready data architecture.

In what follows, we offer a short overview of the overarching capabilities of data architecture. These include user-centricity, elasticity, robustness, and the capacity to ensure the seamless flow of data at all times. Added to these are automation enablement, plus security and data governance considerations. These points from our checklist for what we perceive to be an anticipatory analytics ecosystem.

#big data #data science #big data analytics #data analysis #data architecture #data transformation #data platform #data strategy #cloud data platform #data acquisition

Big Data Consulting Services | Big Data Development Experts USA

Big Data Consulting Services

Traditional data processing application has limitations of its own in terms of processing the large chunk of complex data and this is where the big data processing application comes into play. Big data processing app can easily process complex and large information with their advanced capabilities.

Want to develop a Big Data Processing Application?

WebClues Infotech with its years of experience and serving 350+ clients since our inception is the agency to trust for the Big Data Processing Application development services. With a team that is skilled in the latest technologies, there can be no one better for fulfilling your development requirements.

Want to know more about our Big Data Processing App development services?

Visit: https://www.webcluesinfotech.com/big-data-solutions/

Share your requirements https://www.webcluesinfotech.com/contact-us/

View Portfolio https://www.webcluesinfotech.com/portfolio/

#big data consulting services #big data development experts usa #big data analytics services #big data services #best big data analytics solution provider #big data services and consulting

Silly mistakes that can cost ‘Big’ in Big Data Analytics

Big Data has played a major role in defining the expansion of businesses of all kinds as it helps the companies to understand their audience and devise their business techniques in accordance with the requirement.

The importance of ‘Data’ has been spoken very highly in the modern-day business. Thus, while using big data analysis, the companies must keep away from these minor mistakes otherwise it could have a major impact on their performances. Big Data analysis can be the silver bullet that can answer your questions and help your business to scale newer heights.

Read More: Silly mistakes that can cost ‘Big’ in Big Data Analytics

#top big data analytics companies #best big data service providers #big data for business #big data technology #big data mistakes #big data analytics

Big Data can be The ‘Big’ boon for The Modern Age Businesses

The rapid growth of technology has led to many people opting for online services, and thus the collection and maintenance of data becomes a significant factor for any company. Big data analytics service providers can help the companies get a massive edge over their competitors as they would manage the data well and allow the businesses to make better business decisions. It will provide you with a combination of increased customer experience, revenue, and reduced cost and thus will create a win-win situation for your business. Big data technologies will be your perfect ally in excelling in the cut-throat business environment and come out with flying colors.

Read More: Big Data can be The ‘Big’ boon for The Modern Age Businesses

#big data analytics service providers #top big data analytics companies #impact of big data on businesses #best big data consulting firms #big data #big data for businesses

Top Microsoft big data solutions Companies | Best Microsoft big data Developers

An extensively researched list of top Microsoft big data analytics and solution with ratings & reviews to help find the best Microsoft big data solutions development companies around the world.
An exclusive list of Microsoft Big Data consulting and solution providers, after examining various factors of expert big data analytics firms and found the equivalent matches that boast the ace qualities with proven fineness in data analytics. For business growth and enterprise acceleration getting inputs from the whole data of the organization have become necessary, thus we bring to you the most trustworthy Microsoft Big Data consultants and solutions providers for your assistance.
Let’s take a look at the List of Best Microsoft big data solutions Companies.

#microsoft big data solutions development companies #microsoft big data analytics and solution #microsoft big data consultants #microsoft big data developers #microsoft big data #microsoft big data solution providers