Dock  Koelpin

Dock Koelpin

1595808240

Graph Databases, GraphQL and Dgraph [Tutorial]

In the first quarter of 2020, I had the experience of being in a short term advisory role for a company that had built a type of Linkedin product for a specific vertical market. They had done a valiant job, but it was clear that scalability was a problem and they had created a problem for themselves by having used MongoDB as the datastore.

The client knew this and wanted to do a rebuild using the neo4j, but despite its popularity, I am not a big fan of it as an implementation of a graph database for a variety of reasons, one of which is that it is written in Java. This led me to hunt around for alternatives which ultimately led me to Dgraph, which I initially liked because it was written in golang but I was won over by just how well the system is built and thought out.

In the process of thinking about Dgraph, they released a private beta for their managed GraphQL backend service called Slash GraphQL. This really caught my attention as it allowed me to just dive in and try out some ideas without having to deal with learning how to install the system. So with that as some preview, I went ahead and signed up for the private beta, I was immediately accepted, I don’t know if there is any real threshold to get access other than they want your information, but a small price to pay.

Let’s cover a bit of the ecosystem before we dive in. While GraphQL has become something of a standard for working with Graph databases, that isn’t what it was developed to do, so Dgraph set out to make something that was more specifically suited by modifying GraphQL into GraphQL± and I assume the ± in the name is in reference to the fact that they added and removed features from it.

When working with Dgraph you can use either GraphQL or GraphQL±, but keep in mind that those languages are really targeted to backend developers and this new Slash GraphQL environment is targeted at full-stack developers.

So with all that said, let’s get into it when we first login we see the following screen:

The quickstart video is definitely worth a few minutes to get a sense of what is going on, in addition, you should read the in-process, quick start guide for Slash GraphQL.

The quick start guide has some specific examples and a working ToDo app at the end that you can check out, but I wanted to build something from scratch. I gave it a lot of thought as to various clever things I could do, but ultimately I decided to use the example in the quick start to provide consistency. If you are familiar with GraphQL, then the syntax is easy.

If you aren’t, but know how to code and know JSON, then it’s not too hard to figure out. This is probably a good time to note some distinguishing characteristics of Dgraph.

Dgraph doesn’t not run on top of another DBMS, it manages data in a way the is optimized for how GraphQL stores and retrieves data. That means you don’t have to deal with creating the schema for the underlying DBMS and then doing it in GraphQL and try to keep them in sync whenever you need to make a change. That means that Slash GraphQL empowers you to only deal with GraphQL schemes without knowing anything about the database.

With that said, let’s create our schema:

#graphql #lowcode #nocode #pros-of-graphql #tutorial-for-beginners #tutorial #learn-to-code #design

What is GEEK

Buddha Community

Graph Databases, GraphQL and Dgraph [Tutorial]

Benchmarking the Mainstream Open Source Distributed Graph Databases

The deep learning and knowledge graph technologies have been developing rapidly in recent years. Compared with the “black box” of deep learning, knowledge graphs are highly interpretable, thus are widely adopted in such scenarios as search recommendations, intelligent customer support, and financial risk management.

Meituan has been digging deep in the connections buried in the huge amount of business data over the past few years and has gradually developed the knowledge graphs in nearly ten areas, including cuisine graphs, tourism graphs, and commodity graphs. The ultimate goal is to enhance the smart local life.

Compared with the traditional RDBMS, graph databases can store and query knowledge graphs more efficiently. It gains obvious performance advantage in multi-hop queries to select graph databases as the storage engine. Currently, there are dozens of graph database solutions out there on the market.

It is imperative for the Meituan team to select a graph database solution that can meet the business requirements and to use the solution as the basis of Meituan’s graph storage and graph learning platform. The team has outlined the basic requirements as below per our business status quo:

  1. It should be an open-source project which is also business-friendly

By having control over the source code, the Meituan team can ensure data security and service availability.

  1. It should support clustering and should be able to scale horizontally in terms of both storage and computation capabilities

The knowledge graph data size in Meituan can reach hundreds of billions of vertices and edges in total and the throughput can reach tens of thousands of QPS. With that being said, the single-node deployment cannot meet Meituan’s storage requirements.

  1. It should work under OLTP scenarios with the capability of multi-hop queries at the millisecond level.

To ensure the best search experience for Meituan users, the team has strictly restricted the timeout value within all chains of paths. Therefore, it is unacceptable to respond to a query at the second level.

  1. It should be able to import data in batch

The knowledge graph data is usually stored in data warehouses like Hive. The graph database should be equipped with the capability to quickly import data from such warehouses to the graph storage to ensure service effectiveness.

The Meituan team has tried the top 30 graph databases on DB-Engines and found that most well-known graph databases only support single-node deployment with their open-source edition, for example, Neo4j, ArangoDB, Virtuoso, TigerGraph, RedisGraph. This means that the storage service cannot scale horizontally and the requirement to store large-scale knowledge graph data cannot be met.

After thorough research and comparison, the team has selected the following graph databases for the final round: Nebula Graph (developed by a startup team who originally came from Alibaba), Dgraph (developed by a startup team who originally came from Google), and HugeGraph (developed by Baidu).

A Summary of The Testing Process

Hardware Configuration

  1. Database instances: Docker containers running on different machines
  2. Single instance resources: 32 Cores, 64 GB Memory, 1 TB SSD (Intel® Xeon® Gold 5218 CPU @ 2.30 GHz)
  3. Number of instances: Three

#database #tutorial #graph database #database performance #nebula graph #dgraph #graph database adoption

Brain  Crist

Brain Crist

1600275600

How to Build a Pokedex React App with a Slash GraphQL Backend

Frontend developers want interactions with the backends of their web applications to be as painless as possible. Requesting data from the database or making updates to records stored in the database should be simple so that frontend developer can focus on what they do best: creating beautiful and intuitive user interfaces.

GraphQL makes working with databases easy. Rather than relying on backend developers to create specific API endpoints that return pre-selected data fields when querying the database, frontend developers can make simple requests to the backend and retrieve the exact data that they need—no more, no less. This level of flexibility is one reason why GraphQL is so appealing.

Even better, you can use a _hosted _GraphQL backend—Slash GraphQL (by Dgraph). This service is brand new and was publicly released on September 10, 2020. With Slash GraphQL, I can create a new backend endpoint, specify the schema I want for my graph database, and—voila!—be up and running in just a few steps.

The beauty of a hosted backend is that you don’t need to manage your own backend infrastructure, create and manage your own database, or create API endpoints. All of that is taken care of for you.

In this article, we’re going to walk through some of the basic setup for Slash GraphQL and then take a look at how I built a Pokémon Pokédex app with React and Slash GraphQL in just a few hours!

#development #web developement #databases #graph databases #reactjs #database design #database architecture #pokemon #graph databases in the cloud #dgraph

Mikel  Okuneva

Mikel Okuneva

1599897600

Data Migration From JanusGraph to Nebula Graph - Practice at 360 Finance

Speaking of graph data processing, we have had experience in using various graph databases. In the beginning, we used the stand-alone edition of AgensGraph. Later, due to its performance limitations, we switched to JanusGraph, a distributed graph database. I introduced details on how to migrate data in my article “Migrate tens of billions of graph data into JanusGraph (only in Chinese)”. As the data size and the number of business calls grew, a new problem appeared: Each query consumed too much time. In some business scenarios, a single query took up to 10 seconds, and with increase of the data size, a more complicated single query needed two or three seconds. These problems had seriously affected the performance of the entire business process and the development of related businesses.

The architecture design of JanusGraph determines that a single query is time-consuming. The core reason is that its storage depends on the external storage, and JanusGraph cannot control the external storage well. In our production environment, an HBase cluster is used, which makes it impossible for all queries to be pushed down to the storage layer for processing. Instead, data can only be queried from HBase to the JanusGraph Server memory and then filtered accordingly.

#database #tutorial #graph database #database performance #nebula graph #graph database adoption

Tyrique  Littel

Tyrique Littel

1603573200

Discover GraphQL by Building a Headless Blog with Apollo and Slash GraphQL

In this article, I’m going to explore graph databases/GraphQL by building a proof of concept blog powered by the recently launched Slash GraphQL - a hosted GraphQL backend.

Graphs are a fascinating way to model the information in a system where the relationships (edges) between pieces of data (nodes) are first-class entities of the system. This is a different approach to the more common relational database (RDBMS) model, where the relationships between records are implied. For example, “this user owns this post because user.id == post.user_id”.

Graphs are most useful when the relationships among the data are as important as the data itself. For instance:

  • Knowledge graphs
  • Recommendation engines
  • Supply chain management
  • Social media

In general, graphs offer more descriptive queries, greater flexibility in adapting your model, and better performance when traversing relationships. _GraphQL _(which we will be working with in this article) is a data manipulation and query language that gives you the power of graph through an API.

Working with graphs can feel a little unfamiliar if you’re used to working with RDBMSs, so in this article, I’ll try to show you some of the basics. I’m going to build a GraphQL-powered blog. The relationships between Users, Posts, and Comments (user has many posts, post has many comments, user has many comments) highlight the power of GraphQL, as you’ll see below. As usual, I’m not trying to produce finished, production-ready code here—just simple examples to show you the underlying concepts.

You can find all the example code for this article here.

Dgraph

For this article, I’m going to use Dgraph, a native GraphQL database (as opposed to a GraphQL layer on top of a relational datastore) written in Go, and designed for large-scale application with high-availability and transaction support. If you’re interested in learning more about the project, there are some good introductory videos on their YouTube Channel. Using a native GraphQL database over a GraphQL layer is usually preferred, as it offers easier data modeling, deep join speed, and solves the n+1 problem.

Rather than installing Dgraph on a server, or running it locally via docker, I’m going to use their hosted GraphQL backend service Slash GraphQL. The free tier (at the time of writing) allows 10,000 operations per month, which is plenty for our purposes.

Let’s get started.

Creating a GraphQL Backend on Slash GraphQL

In order to use Slash GraphQL, you need to log in using your GitHub or Google account, and you’ll then see the web interface with options to create and manage your backends, as well as many links to tutorials and other documentation, which I recommend exploring.

Dgraph recently released slash-graphql, a hosted GraphQL backend. With a hosted backend, you don’t need to manage your own infrastructure, create and manage your own database, or create API endpoints. That’s all taken care of for you, so it’s a great choice for our project. Because I strongly prefer working on the command-line, I’m going to use that for this article. But you can do everything via the web interface, if you prefer.

Slash Graph dashboard

You can install the command-line tool by running:

npm install --global slash-graphql

Create a Backend

Using slash-graphql to manage Slash GraphQL backends feels a lot like using the Heroku command-line tool to manage Heroku applications, so if you’re comfortable with that, you should find this quite familiar.

To create a Slash GraphQL backend via the command-line, we first have to log in:

slash-graphql login

This will prompt you to confirm that a code on your terminal matches a code shown on the web page that the command will open. Once you have done this, you can issue commands to Slash GraphQL.

Now we can create our GraphQL backend like this:

slash-graphql create-backend blog

This will output the “endpoint” of your backend, which usually takes around 20 seconds to create. This is the URL we will use to interact with our GraphQL database. You’ll be using this throughout this article, so please make a note of it.

#graph database #graphql #vue #database architecture #web programming #database application development #apollo #graph query languages #dgraph

Dock  Koelpin

Dock Koelpin

1595808240

Graph Databases, GraphQL and Dgraph [Tutorial]

In the first quarter of 2020, I had the experience of being in a short term advisory role for a company that had built a type of Linkedin product for a specific vertical market. They had done a valiant job, but it was clear that scalability was a problem and they had created a problem for themselves by having used MongoDB as the datastore.

The client knew this and wanted to do a rebuild using the neo4j, but despite its popularity, I am not a big fan of it as an implementation of a graph database for a variety of reasons, one of which is that it is written in Java. This led me to hunt around for alternatives which ultimately led me to Dgraph, which I initially liked because it was written in golang but I was won over by just how well the system is built and thought out.

In the process of thinking about Dgraph, they released a private beta for their managed GraphQL backend service called Slash GraphQL. This really caught my attention as it allowed me to just dive in and try out some ideas without having to deal with learning how to install the system. So with that as some preview, I went ahead and signed up for the private beta, I was immediately accepted, I don’t know if there is any real threshold to get access other than they want your information, but a small price to pay.

Let’s cover a bit of the ecosystem before we dive in. While GraphQL has become something of a standard for working with Graph databases, that isn’t what it was developed to do, so Dgraph set out to make something that was more specifically suited by modifying GraphQL into GraphQL± and I assume the ± in the name is in reference to the fact that they added and removed features from it.

When working with Dgraph you can use either GraphQL or GraphQL±, but keep in mind that those languages are really targeted to backend developers and this new Slash GraphQL environment is targeted at full-stack developers.

So with all that said, let’s get into it when we first login we see the following screen:

The quickstart video is definitely worth a few minutes to get a sense of what is going on, in addition, you should read the in-process, quick start guide for Slash GraphQL.

The quick start guide has some specific examples and a working ToDo app at the end that you can check out, but I wanted to build something from scratch. I gave it a lot of thought as to various clever things I could do, but ultimately I decided to use the example in the quick start to provide consistency. If you are familiar with GraphQL, then the syntax is easy.

If you aren’t, but know how to code and know JSON, then it’s not too hard to figure out. This is probably a good time to note some distinguishing characteristics of Dgraph.

Dgraph doesn’t not run on top of another DBMS, it manages data in a way the is optimized for how GraphQL stores and retrieves data. That means you don’t have to deal with creating the schema for the underlying DBMS and then doing it in GraphQL and try to keep them in sync whenever you need to make a change. That means that Slash GraphQL empowers you to only deal with GraphQL schemes without knowing anything about the database.

With that said, let’s create our schema:

#graphql #lowcode #nocode #pros-of-graphql #tutorial-for-beginners #tutorial #learn-to-code #design