How can a search be conducted for the visual attributes of products that were not included in the product's metadata? To clarify further, imagine having millions of fashion products without any information about their color or category. You have the following basic data model:
{
"imageFile": "images/7475.jpg",
"price": 15.66,
"discountPercentage": 7,
"avgRating" : 3.47
}Furthermore, our users wish to perform a search such as "green shirts" and our objective is to retrieve the products where the corresponding image (e.g., images/7475.jpg) portrays a green shirt. This can be easily achieved using Vector Search.
Multi-Lingual examples here (English | Turkish | Italian | Arabic | Spanish) | French
Consider this search query: On the right side, you'll find a sample entry from a database that lacks metadata like the product's "color" or "category." Nevertheless, our search appears to be successful as it matches the visual attributes of the product.
In simpler terms, the application transforms the given texts into vectors, which are then sent to MongoDB Atlas. Using Atlas Search, the system compares these vectors with those in the collection and identifies the most similar vectors to the given one.
Another example:
- Kaggle provides a product data set catalogs:
- If you have any other product catalog images, you can also leverage that.
- You extract the images under this encoder folder such as
encoder/images/- You have jpg files such as
encoder/images/1.jpg,encode/images/abc.jpeg
- You have jpg files such as
There are 2 models evaluated.
- Default: clip-ViT-L-14: Works very well for english language searches (768 dimensions)
- sentence-transformers/clip-ViT-B-32: Works well for multi-lingual searches (512 dimensions)
If you don't want to use the default one, please don't forget to change the number of dimensions too in the config.py file.
- Python Version: Python 3.9 or later
- Python Packages (install with
pip install):- sentence-transformers
- transformers
- Pillow
- pymongo
- MongoDB: MongoDB (Atlas or Enterprise Advanced or Community) with Vector Search enabled
- Database connection
- Please make the necessary changes to the
config/config.pyfile by updating the database connection string, as well as the details of the database and collection.
- Please make the necessary changes to the
- Model
- Choose either multilingual or only-english model and don't forget to set proper number of dimensions in the
config.pytoo- Multi-lingual model encodes the images in 512 dimensions
- Choose either multilingual or only-english model and don't forget to set proper number of dimensions in the
You have already extracted the images under this project structure such as encoder/images/.
Switch to encoder/
And run the encoder_and_loader.py by providing the folder such as images/
$ python encoder_and_loader.py images/It will download the pre-trained model first and then will create worker threads (It will run on 4 processes by default, you can configure this in the config.py), and these processes will go through all the files under the images/ folder and load the vectors inside the MongoDB collection.
The process may require a considerable amount of time, which is dependent on the hardware resources available on this utility and database server distance.
Once the process is finished, you can verify the collection using the instructions provided below.
Switch to webapp/ folder and run flask_server.py.
$ python flask_server.pyThis web application has 2 pages:
For a simple product search, open a browser and navigate to http://localhost:5010/.
For advanced search (multiple conditions), navigate to http://localhost:5010/advanced.
