Scout

Scout is a lightweight, developer-friendly service that computes, stores, and indexes sentence embeddings, serving them through a RESTful interface. Think of it as a mashup of an embedding model, a vector database, and a querying service, all in a single, easy-to-use package. To use scout, simply pass us whole sentences (or paragraphs). Scout will automatically compute embeddings for each sentence and index them. When you want to query, send another sentence and Scout will return sentences that are most semantically similar to the query sentence.

Scout's embedding model supports 50+ languages and allows querying using either Exemplar Support Vector Machine (Exemplar-SVM) or cosine similarity. Exemplar-SVM is an alternative to cosine similarity for ranking that can perform better (at the cost of computing an SVM for each query):

Random note on k-Nearest Neighbor lookups on embeddings: in my experience much better results can be obtained by training SVMs instead. Not too widely known.

Short example: BINGO4D

Works because SVM ranking considers the unique aspects of your query w.r.t. data.

— Andrej Karpathay (@karpathy)

Scout can be used to power semantic search or as a pre-filtering step to reduce prompt sizes for GPT and other costly LLM inputs. Scout can also be used for topic clustering, recommendation systems, or a variety of other natural language processing (NLP) tasks.

Scout employs distiluse-base-multilingual-cased-v2, a 512-dimensional multilingual sentence embedding model. Remarkably, inputs in different languages are mapped close in vector space, allowing for applications across languages. The 53 supported languages are: ar, bg, ca, cs, da, de, el, en, es, et, fa, fi, fr, fr-ca, gl, gu, he, hi, hr, hu, hy, id, it, ja, ka, ko, ku, lt, lv, mk, mn, mr, ms, my, nb, nl, pl, pt, pt-br, ro, ru, sk, sl, sq, sr, sv, th, tr, uk, ur, vi, zh-cn, zh-tw. This model is based upon Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks.

Scout is used at HomeVision as a backend to a number of internal tools and pipelines. HomeVision pipelines collectively process millions of pages of appraisal text monthly using NLP, computer vision, and machine learning.

Note: This server is itself a running instance of scout, so feel free to make and create indices and try it out. Just be aware that this server may be restarted at any time, wiping out any data. ¯\_(ツ)_/¯

API Documentation

POST /index/{index_name}

Creates an index named index_name

Parameters

Name	Description
`index_name`	Name of the index to create
body	Optional `POST` body containing an array of `TextBody` objects to index. If missing, an empty index will be created.

Responses

HTTP Code	Response
`200`	Returns `IndexResponse`

Example

curl -H "Content-Type: application/json" -d '[{"id": "hamlet", "text": "To be, or not to be: that is the question."}, {"id": "julius_caesar", "text": "Friends, Romans, countrymen, lend me your ears."}]' https://goscout.online/index/shakespeare

GET /index/{index_name}

Reads an index named index_name

Parameters

Name	Description
`index_name`	Name of the index to read

Responses

HTTP Code	Response
`200`	Returns `IndexResponse`

Example

curl https://goscout.online/index/shakespeare

PUT /index/{index_name}

Updates an index named index_name

Parameters

Name	Description
`index_name`	Name of the index to read
body	Required `PUT` body containing an array of `TextBody` objects to index. These text bodies will be appended to the index.

Responses

HTTP Code	Response
`200`	Returns `IndexResponse`

Example

curl -H "Content-Type: application/json" -X PUT -d '[{"id": "henry_v", "text": "Once more unto the breach, dear friends, once more."}]' https://goscout.online/index/shakespeare

DELETE /index/{index_name}

Deletes an index named index_name

Parameters

None

Responses

HTTP Code	Response
`200`	Returns `IndexResponse`

Example

curl -X DELETE https://goscout.online/index/shakespeare

GET /index/{index_name}/query?q={query}&n={num
          results}&method={method}

Queries an index named index_name

Parameters

Name	Description
`index_name`	Name of the index to read
`q`	Required query parameter of text to query against `index_name`
`n`	Optional query param to set number of returned results (default: `3`)
`method`	Optional query param to set the method. Valid options are `svm` for Exemplar SVM, or `cosine` for Cosine similarity. (default: `svm`)

Responses

HTTP Code	Response
`200`	Returns an array of `SearchResult`

Example

curl https://goscout.online/index/shakespeare/query?q=romans&n=2

API Schema

TextBody

Represents a sentence to be embedded. The id attribute is an arbitrary string, meaningful only to the client. The text attribute can be a sentence or paragraph to be embedded.

Example

{
  "id": "hamlet",
  "text": "To be, or not to be: that is the question."
}

SearchResult

Represents a result from a query. In addition to the fields from TextBody, the score attribute is a float that represents how well matched the query is to the result.

Example

{
  "id": "hamlet",
  "text": "To be, or not to be: that is the question."
  "score": 0.87
}

ErrorResponse

Returned by the API when encountering an error. The error attribute is a message with more information about an error. The status code associated with this response will always be non-200.

Example

{
  "ok": false,
  "error": "An error has occurred"
}

IndexResponse

Returned by CRUD action on an index. The index attribute is the name of the index and the size attribute is the size of the index at the time of the action.

Example

{
  "index": "shakespeare",
  "size": 1431
}

Source Code, Technical Notes, Installation

The source code for scout can be found at https://github.com/homevision/rs-scout. Scout is written in Rust (built using v1.69.0) and targets x86_64 architectures. Note: you must be able to build PyTorch's C++ bindings as this is a required dependency. Scout's Exemplar-SVM querying is powered by liblinear, the same library used to power fast SVM training in scikit-learn. For the time being, rs-scout does not compile on Apple Silicon.

Installation/Execution

Download repository: git clone git@github.com:HomeVision/rs-scout.git && cd rs-scout
Download and convert model weights using rust-sbert to ./models
Start server: cargo run
Test server: curl http://localhost:8000

Questions, Comments, or Feedback Welcome

Please find me on twitter at @vincentchu.