Uncovering Xcode Indexing

TL;DR

I use Xcode every day, but I know nothing about Xcode. In this article, I will explore what indexing is and how it works. At the end of the article, I have written a simple command-line tool to query the information stored in indexing DataStore.

Image for post

Summary

1\. Xcode Indexing
  1.1\. What indexing is

2\. How indexing work
  2.1\. Unit file and record file
  2.2\. Using IndexStore to query DataStore
  2.3\. Xcode Indexing Log
3\. Extend Reading

Xcode Indexing

What indexing is

Indexing support a range of IDE features like code navigation, syntax highlighting, code auto-completion, jump-to-definition, find usages, refactor. You use these features every day as an iOS developer.

Image for post

How indexing works

When Xcode loads a project that hasn’t been build, it will start indexing the project.

Image for post

Image for post

Activity monitor is a handy tool to help you find out the running processes, the Xcode and SKAgent pop up from the list after I open the project.

Image for post

So, the question is what **SKAgent is? **The SKAgent is the component of SourceKit framework runs the compilation commands to generate the indexing information. The SKAgent executable binary locates at /Applications/Xcode.app/Contents/SharedFrameworks/SourceKit.framework/Versions/A/XPCServices/com.apple.dt.SKAgent.xpc/Contents/MacOS/com.apple.dt.SKAgent

Image for post

The SKAgent is a XPC Service operated by SourceKit uses the XPCmechanism to communicate with Xcode.

Image for post

#ios #indexing #clang #xcode

What is GEEK

Buddha Community

Uncovering Xcode Indexing

Uncovering Xcode Indexing

TL;DR

I use Xcode every day, but I know nothing about Xcode. In this article, I will explore what indexing is and how it works. At the end of the article, I have written a simple command-line tool to query the information stored in indexing DataStore.

Image for post

Summary

1\. Xcode Indexing
  1.1\. What indexing is

2\. How indexing work
  2.1\. Unit file and record file
  2.2\. Using IndexStore to query DataStore
  2.3\. Xcode Indexing Log
3\. Extend Reading

Xcode Indexing

What indexing is

Indexing support a range of IDE features like code navigation, syntax highlighting, code auto-completion, jump-to-definition, find usages, refactor. You use these features every day as an iOS developer.

Image for post

How indexing works

When Xcode loads a project that hasn’t been build, it will start indexing the project.

Image for post

Image for post

Activity monitor is a handy tool to help you find out the running processes, the Xcode and SKAgent pop up from the list after I open the project.

Image for post

So, the question is what **SKAgent is? **The SKAgent is the component of SourceKit framework runs the compilation commands to generate the indexing information. The SKAgent executable binary locates at /Applications/Xcode.app/Contents/SharedFrameworks/SourceKit.framework/Versions/A/XPCServices/com.apple.dt.SKAgent.xpc/Contents/MacOS/com.apple.dt.SKAgent

Image for post

The SKAgent is a XPC Service operated by SourceKit uses the XPCmechanism to communicate with Xcode.

Image for post

#ios #indexing #clang #xcode

Shawn  Durgan

Shawn Durgan

1598750220

MongoDB Indexes: Deep Dive, Understanding Indexes.

Getting a performance boost with the best usage of indexes, by understanding what’s the data structure, how it works’s/stored, how is it loaded into memory. How Query optimization make’s decision to select indexes.

_Basic understanding of indexes is required i.e what are indexes, index types, creating them. _https://docs.mongodb.com/manual/indexes/

  • Data Structure
  • Storage on disk
  • Memory Allocation.

Data Structure

Index on a filed/fields is stored in order that we specify using B-Tree data structure. Stored in ordered Let see what does it mean’s and how it help’s.

  • Index is created on the value of the filed referencing to the actual document stored.

Image for post

snipped from: MognoDB university

  • Using B-Tree indexes significantly reduces the number of comparison to find the document.

Image for post

snipped from: MognoDB university

  • Likewise in below picture we can see with index(sky blue line) even adding document still limit the number of document examined in comparison to without index/collscan.

Image for post

snipped from: MognoDB university

Storage On Disk

Let’s see/visualize how the index are stored on disk. Index stored on disk is managed by the database storage engine itself.

  • Uses prefix Index compression- Repeated prefix value is not written, let us see example to understand what it mean’s.
db.getCollection("movieTicket")
.ensureIndex({"showDate":1, "seatNo":1, "status":1});

How the index ({“showDate”:1, “seatNo”:1, “status”:1}) is stored on disk.

Image for post

showDate_1_seatNo_1_status_1

#index #mongodb #indexing #mongo

Edison  Stark

Edison Stark

1598535540

How Indexes Work in Nebula Graph - DZone Database

Why Indexes Are Needed in a Graph Database

Indexes are an indispensable function in a database system. Graph databases are no exception.

An index is actually a sorted data structure in the database management system. Different database systems adopt different sorting structures.

Popular index types include:

  • B-Tree index
  • B±Tree index
  • B*-Tree index
  • Hash index
  • Bitmap index
  • Inverted index

Each of them uses their own sorting algorithms.

A database index allows efficient data retrieval from databases. Despite of the query performance improvement, there are some disadvantages of indexes:

  • It takes time to create and maintain indexes, which scales with dataset size.
  • Indexes need extra physical storage space.
  • It takes more time to insert, delete, and update data because the index also needs to be maintained synchronously.

Taking the above into consideration, Nebula Graph now supports indexes for more efficient retrieves on properties.

This post gives a detailed introduction to the design and practice of indexes in Nebula Graph.

Core Concepts to Understand Indexes in Nebula Graph

Below is a list of common Nebula Graph index terms we use across the post.

  • Tag: A label associated with a list of properties. Each vertex can associate with multiple tags. Tag is identified with a TagID. You can regard tag as a node table in SQL.
  • Edge: Similar to tag, edge type is a cluster of properties on edges. You can regard edge type as an edge table in SQL.
  • Property: The name-value pairs on tag or edge. Its data type is determined by the tag or edge type.
  • Partition: The minimum logical storage unit of Nebula Graph. A StorageEngine can contain multiple partitions. Partition is divided into leader and follower. We use Raft to guarantee data consistency between leader and follower.
  • Graph space: A physically isolated space for a specific graph. Tags and edge types in one graph are independent with those in another graph. A Nebula Graph cluster can have multiple graph spaces.
  • Index: Index in this post refers specifically to the index of ~~ ~~tag or edge type properties. Its data type depends on tag or edge type.
  • TagIndex: An index created for a tag. You can create multiple indexes for the same tag. Cross-tag composite index is yet to be supported.
  • EdgeIndex: An index created for an edge type. Similarly, you can create multiple indexes for the same edge type. Cross-edge-type composite index is yet to be supported.
  • Scan Policy: The policy to scan indexes. Usually, there are multiple methods to scan indexes to execute one query statement, but the scan policy itself gets to decide which method to use ultimately.
  • Optimizer: Optimize query conditions, such as sorting, splitting, and merging sub-expression nodes of the expression tree of the where clause. It’s used to obtain higher query efficiency.

What’s Required for Indexes to Work in a Graph Database

There are two typical ways to query data in Nebula Graph, or more generally in a graph database:

  1. One is starting from a vertex, retrieving its (N-hop) neighbors along certain edge types.
  2. Another is retrieving vertices or edges which contain specified property values.

In the latter scenario, a high-performance scan is needed to fetch the edges or vertices as well as the property values.

In order to improve the query efficiency of property values, we’ve implemented indexes in Nebula Graph. By sorting the property values of edges or vertices, users can quickly locate a certain property and avoid full scan.

Here’s what we found are required for indexes to work in a graph database:

  • Supporting indexes for properties on tags and edge types.
  • Supporting analysis and generation of index scanning strategy.
  • Supporting index management such as create index, rebuild index, show index, etc.

How Indexes Are Stored in Nebula Graph

Below is a diagram of how indexes are stored in Nebula Graph. Indexes are a part of Nebula Graph’s Storage Service so we place them in the big picture of its storage architecture.

Seen from the above figure, each Storage Server can contain multiple Storage Engines, each Storage Engine can contain multiple Partitions.

Different Partitions are synchronized via Raft protocol. Each Partition contains both data and indexes. The data and indexes of the same vertex or edge will be stored in the same Partition.

#tutorial #graph database #index #database indexes #nebula graph #database

Charity  Ferry

Charity Ferry

1626142933

How to Use PostgreSQL's Automatic Index Recommendations

In our last blog, we learned about the Need and Usage of Hypothetical Indexes in PostgreSQL. We can now “check” easily in a live environment, determine if some particular index will be helpful or not, and figure out how we get to know which index to test. To do this, you’ll also need in-depth knowledge of indexing and experience in Postgresql. However, in PostgreSQL, we can get an automatic recommendation of indexes for specific queries by using three extensions hypog, pg_stat_statements, and pg_qualstats.

#postgresql #automatic index recommendations #automatic #index

SQL Server Indexes Management Using Index Manager for SQL Server

When talking about SQL Server performance tuning and queries enhancement, the first thing to consider is the SQL Server Index. It serves to accelerate reading data from underlying tables by providing quick access to the requested rows. Thus, it won’t need to scan all the table’s records.

The SQL Server index provides those fast search capabilities due to the B-Tree structure of the index. This structure makes it possible to move through the table rows based on the index key quickly and retrieve the requested records at once. It won’t need to read the whole table.

SQL Server indexes’ types

Among the main types, we pay attention to the clustered and non-clustered indexes.

The **Clustered index **sorts the actual data in the data pages according to the clustered index key values. It stores the data at the “leaf” level of the index, ensuring the possibility to create only one clustered index on each table. The clustered index is created automatically when a Primary Key constraint appears on the heap table.

The **Non-clustered index **contains the index key value and a pointer to the rest of the row columns in the main table. It is the “leaf” level of the index, with the ability to create up to 999 non-clustered indexes on each table.

#SQL Server #indexes #query performance #sql server