A Manifesto for Live & Sharable Data

A Manifesto for Live & Sharable Data

A Manifesto for Live & Sharable Data. In this blog, I’ll argue that — with recent advances in computer science — we can make improvements to this for many applications.

The ‘truth’ should be the data that is being used, not the data in distant storage.

Distribute the data automatically, with the guarantee that all of it will converge on the same ‘truth’.

Use a published open standard for encoding data with its meaning, and communicating changes to it.

Hi, I’m George. This year I left my day job as a software engineering leader and plunged into lockdown under a mountain of work, uncertainty, and risk. Last week, I pushed the button to launch the m-ld Developer Preview. The period between now and when lockdown began has been a mad journey filled with moments of creativity, anxiety, frustration, imposter syndrome, fight and flight urges, elation and time-dilation, and so! much! coffee!

The Why

As a data management app developer, I’ve used many ways to encode and store data. Frequently, they are combined in the same architecture with one of the locations becoming known as the ‘central truth’:

While the specific technologies vary, the overall pattern is very common. Motivations include properties of security, integrity, consistency, operational efficiency, and cost. However, there are some other peculiar properties that stand out:

  • The ‘truth’ is on the far right-hand side; but the data is being used throughout, with the particular value being realized on the left.
  • The software application is responsible for both distributing the data and for operating on it.
  • Every encoding syntax is specific to a technology and does not expose the data’s meaning enough to be independently understood.

The main consequence of these properties is application code complexity. We have to be incredibly careful to maintain an understanding of the code as to how current (how close to the truth) our copy of the data is. We must then operate on the data accordingly, and share the understanding with other components. This is hard and frequently goes awry; resulting in software bugs that are very hard to reproduce, let alone fix.

In this blog, I’ll argue that — with recent advances in computer science — we can make improvements to this for many applications. Applying our manifesto, we want our architecture to look more like this:

collaboration software data decentralization data-science

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

50 Data Science Jobs That Opened Just Last Week

Data Science and Analytics market evolves to adapt to the constantly changing economic and business environments. Our latest survey report suggests that as the overall Data Science and Analytics market evolves to adapt to the constantly changing economic and business environments, data scientists and AI practitioners should be aware of the skills and tools that the broader community is working on. A good grip in these skills will further help data science enthusiasts to get the best jobs that various industries in their data science functions are offering.

Applications Of Data Science On 3D Imagery Data

The agenda of the talk included an introduction to 3D data, its applications and case studies, 3D data alignment and more.

Data Science Course in Dallas

Become a data analysis expert using the R programming language in this [data science](https://360digitmg.com/usa/data-science-using-python-and-r-programming-in-dallas "data science") certification training in Dallas, TX. You will master data...

32 Data Sets to Uplift your Skills in Data Science | Data Sets

Need a data set to practice with? Data Science Dojo has created an archive of 32 data sets for you to use to practice and improve your skills as a data scientist.

Data Cleaning in R for Data Science

A data scientist/analyst in the making needs to format and clean data before being able to perform any kind of exploratory data analysis.