💻Combine keyword search with vector search

Tutorial on combining keyword search with vector search

There are sometimes search cases where pure vector search often does not provide the best solution. For example - we can take the simple case of product SKUs in retail e-commerce. Product SKUs are the name of the products.

1. Identifying strength & weaknesses of vector search

Below, we use an example of vector search where an individual searches for an SKU. However, the search results encode the letters and fail to realize/return the right SKU. We also attach a code example using the Vector AI client for those interested in trying this out.

Python SDK

search_results = vi_client.search(
    dataset_id,
    text_encoder.encode('R170NZKAXSA'), 'name_vector_', page_size=3)
vi_client.show_json(search_results, selected_fields=['_id', 'name', 'sku'],
    image_fields=['image_url'], image_width=150)

Result from pure vector search

2. Identifying strengths & weaknesses of traditional search

3. Combing both for a hybrid search

In these situations, our search should properly return the right value when given the SKU. In turn, hybrid search can return the right result.

Python SDK

search_results = vi_client.hybrid_search(collection_name, 'R170NZKAXSA',
      text_encoder.encode('R170NZKAXSA'),
      fields=['name_vector_'], 
      text_fields=['name'],
      traditional_weight=0.015,
      page_size=3)
vi_client.show_json(search_results, selected_fields=['_id', 'name', 'sku'],
    image_fields=['image_url'], image_width=150)

Hybrid search example

From above, we realize that the value of hybrid search allows us to match items/products when exact values are known by the searcher.