Michio JP

Michio JP

1634975643

Vector AI | A Platform for Building Vector Based Applications

 Vector AI is a framework designed to make the process of building production grade vector based applications as quickly and easily as possible. Create, store, manipulate, search and analyse vectors alongside json documents to power applications such as neural search, semantic search, personalised recommendations recommendations etc.

 

Features

  • Multimedia Data Vectorisation: Image2Vec, Audio2Vec, etc (Any data can be turned into vectors through machine learning)
  • Document Orientated Store: Store your vectors alongside documents without having to do a db lookup for metadata about the vectors.
  • Vector Similarity Search: Enable searching of vectors and rich multimedia with vector similarity search. The backbone of many popular A.I use cases like reverse image search, recommendations, personalisation, etc.
  • Hybrid Search: There are scenarios where vector search is not as effective as traditional search, e.g. searching for skus. Vector AI lets you combine vector search with all the features of traditional search such as filtering, fuzzy search, keyword matching to create an even more powerful search.
  • Multi-Model Weighted Search: Our Vector search is highly customisable and you can peform searches with multiple vectors from multiple models and give them different weightings.
  • Vector Operations: Flexible search with out of the box operations on vectors. e.g. mean, median, sum, etc.
  • Aggregation: All the traditional aggregation you'd expect. e.g. group by mean, pivot tables, etc
  • Clustering: Interpret your vectors and data by allocating them to buckets and get statistics about these different buckets based on data you provide.
  • Vector Analytics: Get better understanding of your vectors by using out-of-the-box practical vector analytics, giving you better understanding of the quality of your vectors.

Quick Terminologies

  • Models/Encoders (aka. Embedders) ~ Turns data into vectors e.g. Word2Vec turns words into vector
  • Vector Similarity Search (aka. Nearest Neighbor Search, Distance Search)
  • Collection (aka. Index, Table) ~ a collection is made up of multiple documents
  • Documents (aka. Json, Item, Dictionary, Row) ~ a document can contain vectors, text and links to videos/images/audio.

QuickStart

Install via pip! Compatible with any OS.

pip install vectorai

If you require the nightly version due to on-going improvements, you can install the nightly version using:

pip install vectorai-nightly

Note: while the nightly version will still pass automated tests, it may not be stable.

Check out our quickstart notebook on how to make a text/image/audio search engine in 5 minutes: quickstart.ipynb

from vectorai import ViClient, request_api_key

api_key = request_api_key(username=<username>, email=<email>, description=<description>, referral_code="github_referred")

vi_client = ViClient(username=username, api_key=api_key)

from vectorai.models.deployed import ViText2Vec
text_encoder = ViText2Vec(username, api_key)

documents = [
    {
        '_id': 0,
        'color': 'red'
    },
    {
        '_id': 1,
        'color': 'blue'
    }
]

# Insert the data
vi_client.insert_documents('test-collection', documents, models={'color': text_encoder.encode})

# Search the data
vi_client.search('test-collection', text_encoder.encode('maroon'), 'color_vector_', page_size=2)

# Get Recommendations
vi_client.search_by_id('test-collection', '1', 'color_vector_', page_size=2)

Access Powerful Vector Analytics

Vector AI has powerful visualisations to allow you to analyse your vectors as easily as possible - in 1 line of code.

vi_client.plot_dimensionality_reduced_vectors(documents, 
    point_label='title', 
    dim_reduction_field='_dr_ivis', 
    cluster_field='centroid_title', cluster_label='centroid_title')

View Dimensionality-Reduced Vectors

vi_client.plot_2d_cosine_similarity(
    documents,
    documents[0:2],
    vector_fields=['use_vector_'],
    label='name',
    anchor_document=documents[0]
)

Compare vectors and their search performance on your documents easily! 1D plot cosine simlarity

Why Vector AI compared to other Nearest Neighbor implementations?

  • Production Ready: Our API is fully managed and can scale to power hundreds of millions of searches a day. Even at millions of searches it is blazing fast through edge caching, GPU utilisation and software optimisation so you never have to worry about scaling your infrastructure as your use case scales.
  • Simple to use. Quick to get started.: One of our core design principles is that we focus on how people can get started on using Vector AI as quickly as possible, whilst ensuring there is still a tonne of functionality and customisability options.
  • Richer understanding of your vectors and their properties: Our library is designed to allow people to do more than just obtain nearest neighbors, but to actually experiment, analyse, interpret and improve on them the moment the data added to the index.
  • Store vector data with ease: The document-orientated nature for Vector AI allows users to label, filter search and understand their vectors as much as possible.
  • Real time access to data: Vector AI data is accessible in real time, as soon as the data is inserted it is searchable straight away. No need to wait hours to build an index.
  • Framework agnostic: We are never going to force a specific framework on Vector AI. If you have a framework of choice, you can use it - as long as your documents are JSON-serializable!

Using VectorHub Models

VectorHub is Vector AI's main model repository. Models from VectorHub are built with scikit-learn interfaces and all have examples of Vector AI integration. If you are looking to experiment with new off-the-shelf models, we recommend giving VectorHub models a go - all of them have been tested on Colab and are able to be used in as little as 3 lines of code!

Schema Rules for documents (BYO Vectors and IDs)

Ensure that any vector fields contain a '_vector_' in its name and that any ID fields have the name '_id'.

For example:

example_item = {
    '_id': 'James',
    'skills_vector_': [0.123, 0.456, 0.789, 0.987, 0.654, 0.321]
}

The following will not be recognised as ID columns or vector columns.

example_item = {
    'name_id': 'James',
    'skillsvector_': [0.123, 0.456, 0.789, 0.987, 0.654, 0.321]
}

How does this differ from the VectorAI API?

The Python SDK is designed to provide a way for Pythonistas to unlock the power of VectorAI in as few lines as code as possible. It exposes all the elements of an API through our open-sourced automation tool and is the main way our data scientists and engineers interact with the VectorAI engine for quick prototyping before developers utilise API requests.

Note: The VectorAI SDK is built on the development server which can sometimes cause errors. However, this is important to ensure that users are able to access the most cutting-edge features as required. If you run into such issues, we recommend creating a GitHub Issue if it is non-urgent, but feel free to ping the Discord channel for more urgent enquiries.

Building Products with Vector AI

Creating a multi-language AI fashion assistant: https://fashionfiesta.me | Blog

Demo

Do share with us any blogs or websites you create with Vector AI!

Download Details:
 

Author: vector-ai
Download Link: Download The Source Code
Official Website: https://github.com/vector-ai/vectorai 
License: Apache-2.0 License

#Python 

Vector AI | A Platform for Building Vector Based Applications
Ray  Patel

Ray Patel

1634882040

Python Print JSON Pretty And Example Code

In this tutorial, we'll learn How to Use JSON dump() method with indent parameter to print JSON pretty in Python. You should use the optional argument indent.

#python #json #Python 

Python Print JSON Pretty And Example Code
Ray  Patel

Ray Patel

1634874120

Python JSON Dumps Indent And Example Code

In this tutorial, we'll learn Python JSON Dumps Indent And Example Code

Python JSON method used indent parameter to specify the spaces that are used at the beginning of a line. If the indent parameter is not used By default it doesn’t use indentations and writes all data on a single line, which is not readable.

#python #Python #json 

Python JSON Dumps Indent And Example Code
Ray  Patel

Ray Patel

1634866671

How to Import JSON in Python And Example Code

In this tutorial, we'll share How to Import JSON in Python And Example Code

Python has a built-in json module, which can be used to work with JSON data. See below the syntax of Import the json module:

JSON can store Lists, bools, numbers, tuples and dictionaries.

The JSON module is mainly used to convert the python dictionary above into a JSON string that can be written into a file.

#python #Python #json 

How to Import JSON in Python And Example Code
Osiki  Douglas

Osiki Douglas

1634836920

Ethical Hacking with Python: Beginners to Advanced Level

In this tutorial, we'll share Ethical Hacking with Python: Beginners to Advanced Level

  1. Strings in Python
  2. Math in Python
  3. Variables & Methods
  4. Methods
  5. Type casting in Python
  6. Incrementing A Variable
  7. Functions in Python
  8. Boolean in Python
  9. Relational & Boolean Operators
  10. Conditional Statements — if-else scenario
  11. Lists in Python — mutable — that is it can be changed
  12. Tuples
  13. Looping
  14. Importing
  15. Strings
  16. Dictionaries
  17. Sockets

#python #Python 

 

Ethical Hacking with Python: Beginners to Advanced Level
Ray  Patel

Ray Patel

1634702580

Python Developer Learning Roadmap with Resources for Beginner in 2021

Python Learning Roadmap 2021: Because of its extreme flexibility, Python is one of the most popular programming languages among data scientists, software engineers, and developers. Python can be used for a variety of tasks, including software development, web development, web scraping, data research, machine learning, artificial intelligence, competitive programming, and more. Our custom-made Learning Paths will take your Python skills to the next level with an accelerated, hands-on study plan, whether you’re a beginner, intermediate, or advanced Pythonista.

Why become a Python developer?

How to become a Python developer?

#python #Python 

Python Developer Learning Roadmap with Resources for Beginner in 2021
Ray  Patel

Ray Patel

1634698260

4 Easy Method for Get File Size By Python in 2021

There are different ways to get file size in Python. We will be using the OS module and the pathlib module to check the file size. The OS module in Python comes as built-in, and it provides various utility methods to interact with operating system features.

In this tutorial, we'll share 4 Easy Method for Get File Size By Python in 2021

os.path.getsize()

os.stat()

seek() and tell()

path.stat().st_mode

#python #Python 

4 Easy Method for Get File Size By Python in 2021
Ray  Patel

Ray Patel

1634690760

Python Tutorial: Python Classmethod Decorator & Examples Code

In this tutorial, we'll learn Python Classmethod Decorator & @Classmethod Examples Code in 2021

Using classmethod decorator you can define a class method in Python. You can call this method using ClassName.MethodName().

The first parameter must be cls, which can be used to access class attributes and only access the class attributes but not the instance attributes.

#python #Python 

Python Tutorial: Python Classmethod Decorator & Examples Code
Ray  Patel

Ray Patel

1634683380

Python Tutorial: Python Print Object As A String & Examples Code

In this tutorial, we'll learn Python Print Object As A String & Examples Code in 2021

Use str() and __repr()__ methods to print objects as a string in Python. The __str__ method is what gets called happens when you print it, and the __repr__ method is what happens when you use the repr() function (or when you look at it with the interactive prompt).

#python #Python 

Python Tutorial: Python Print Object As A String & Examples Code
Ray  Patel

Ray Patel

1634676000

Python Tutorial: Python Print Object Of Class & Examples Code

In this tutorial, we'll learn Python Print Object Of Class & Examples Code in 2021

Use anyone from the repr() Method or Using the str() Method or Adding New Class Method to print object in Python. A class is like a blueprint while an object is a copy of the class with actual values.

#python #Python 

Python Tutorial: Python Print Object Of Class & Examples Code
Ray  Patel

Ray Patel

1634668680

Python Tutorial: Python Print Methods Of Object & Examples Code

In this tutorial, we'll learn Python Print Methods Of Object & Examples Code in 2021

The simplest method is to use dir(object_name) to get all the methods available for that object and use the print method to print it.

#python #Python 

Python Tutorial: Python Print Methods Of Object & Examples Code
Ray  Patel

Ray Patel

1634653980

Python Tutorial: How to Convert Python list to JSON & Examples Code

In this tutorial, we'll learn How to Convert Python list to JSON & Examples Code for beginner in 2021.

Use json.dumps() function to convert list to JSON in Python. This function takes a list as an argument and returns a JSON String.

#Python #Python #json 

Python Tutorial: How to Convert Python list to JSON & Examples Code
Ray  Patel

Ray Patel

1634646600

Python Tutorial: Python Print Object As JSON & Examples Code

Learn Python Print Object As JSON & Examples Code in 2021

Use JSON package and print method to print object as JSON in Python. json.dumps() converts Python object into a json string. Every object has an attribute that is denoted by dict and this stores the object’s attributes in Python.

#python #Python #json 

Python Tutorial: Python Print Object As JSON & Examples Code

Autoscaling Lifecycle Hooks: Save EC2 Logs Automatically to S3 Bucket

Have you already lost logs from your cloud server? Then, check out this solution and never lose the logs again.

https://www.bitslovers.com/auto-scaling-lifecycle-hooks/

Goal

This project contains files to create a solution using Autoscaling Lifecycle Hook, to save and storage the EC2 instance logs on a S3 Bucket, before terminate the instance.

To achieve that goal we use the folling resources and services on AWS:

What you need

  • Autoscaling Group
  • IAM Role and Policies
  • S3 Buckets
  • Lambda Function
  • CloudWatch Rule
  • Systems Manager (SSM) Document
  • Run Command


#linux  #ubuntu  #cloudcomputing   #coding  #dev  #docker  #fedora   #RedHat #Python  #opensource  #machinelearning  #devops  #javascript 
#java  #serverless  #aws #cloud 
 

Autoscaling Lifecycle Hooks: Save EC2 Logs Automatically to S3 Bucket
Ray  Patel

Ray Patel

1634639280

Python Tutorial: Print Response Body in Python & Examples Code

Learn Print Response Body in Python & Examples Code in 2021

Simple use requests.get() method to get all body content and use response.json() to get JSON data.

#python #Python 

Python Tutorial: Print Response Body in Python & Examples Code