IoTDB Provides Data Management for Industrial Edge IT. IoTDB has recently graduated from the Apache Software Foundation incubator to become a top-level project.
As industrial Internet-of-Things (IIoT) applications produce a blinding amount of data 24/7, relational and key- value-based databases struggle to keep up.
It’s a problem the team at the Tsinghua University School of Software has been working on for going on a decade. The result is IoTDB, which recently graduated from the Apache Software Foundation incubator to become a top-level project.
It reached this status “at a time of confluence of database, Internet-of-Things (IoT) and AI technologies in conjunction with a wider adoption of Industry 4.0 and automation approaches to further enable remote work and increased efficiencies,” said C. Mohan, retired IBM Fellow, former chief scientist at IBM India, and a member of the US National Academy of Engineering.
As a Distinguished Visiting Professor working with the team, “I have seen this project reach maturity and build up a vibrant open source community around it,” he said.
As a PhD student at the Chinese university starting around 2012, Xiangdong Huang, now vice president of the Apache project, was assigned to manage the time-series data being generated by the minute by a large company’s 200,000 machines. Reading the data from Oracle quickly proved to be too slow and buying a more advanced license too expensive.
They decided to try NoSQL — Cassandra — but ran into performance problems with it as well.
“Apache Cassandra is good, and we used five nodes to manage all the data,” he said, explaining, “The user may create more than 5,000 tables in Cassandra, and they do not want to buy more servers to form a larger cluster (as the budget is limited). From then on, I spent about two years to read the source code of Cassandra, and did some modifications on Cassandra, and spent a lot of effort to use the limited server resources to provide better performance, which made us tired. … Even [though] we spent a lot of effort, we find it hard to use five nodes to reach 10 million data points written per second.”
They tried saving a packet of hundreds of data points to Cassandra as key-value pairs. But that meant maintaining everything themselves and still ran into performance issues and limitations with the data structure.
They then decided to create a time-series database from scratch. Professor Jianmin Wang came up with the idea of donating the project to ASF in 2018 as a way to get more people involved.
Data Science and Analytics market evolves to adapt to the constantly changing economic and business environments. Our latest survey report suggests that as the overall Data Science and Analytics market evolves to adapt to the constantly changing economic and business environments, data scientists and AI practitioners should be aware of the skills and tools that the broader community is working on. A good grip in these skills will further help data science enthusiasts to get the best jobs that various industries in their data science functions are offering.
The agenda of the talk included an introduction to 3D data, its applications and case studies, 3D data alignment and more.
Become a data analysis expert using the R programming language in this [data science](https://360digitmg.com/usa/data-science-using-python-and-r-programming-in-dallas "data science") certification training in Dallas, TX. You will master data...
Need a data set to practice with? Data Science Dojo has created an archive of 32 data sets for you to use to practice and improve your skills as a data scientist.
Top Web & Mobile Application Development Company in India & USA. We specialize in Golang, Ruby on Rails, Symfony, Laravel PHP, Python, Angular, Mobile Apps, Blockchain, & Chatbots