Facebook is one of data richest companies in the world. Facebook’s Groknet is a state-of-art product recognition system. This model is trained on 7 datasets across several commerce verticals.
Facebook’s Marketplace has became a great platform for selling and buying products across the globe. When user uploads about a product, description of product includes colour, price, location, brand , year of production etc. Users may not feel good when description is missing completely or partially. So, Facebook decided to use image uploaded by user to augment missing information.
Groknet can analyze image and predict :
- Object category : “bar stool,” “scarf,” “area rug,” …
- Home attributes: object color, material, decor style,
- Fashion attributes: style, color, material, sleeve length, …
- Vehicle attributes: make, model, external color, decade,
- Search queries: text phrases likely used by users to find the product on Marketplace Search,
- Image embedding: 256-bit hash used to recognize exact products, find and rank similar products, improve search quality.
Groknet recognizing all products on image
Groknet is trained on human annotations, user-generated tags, and noisy search engine interaction data.
Data Preparation
GrokNet aimed to solve a large number of computer vision tasks. Dataset consists of 89 million Facebook Marketplace images from 7 different datasets. It uses 80 categorical loss functions and 3 embedding losses.
Training data of 89 million images
Object Categories
object categories are from an internal human-annotated dataset of Marketplace images, with one of 566 labels such as “chair”, “bracelet”, and “bicycle” .
Attributes (Fashion, Home, and Vehicles)
Multi label annotations for attributes of fashion, human and vehicles are used in training.
For home products, colour and material are some attributes.
Product Identities (Fashion, Home, and Vehicles)
Different images of same product are going to have same id in dataset.
#image-classification #image-recognition #facebook-algorithm #facebookai #augmentation #algorithms
