Embeddings Gemini
Modern text embedding with the Gemini API
The Gemini API will now offer a new experimental Gemini Embedding text model (gemini-embedding-exp-03-07).
This embedding model, which was trained on the Gemini model itself, has inherited Gemini’s linguistic comprehension and subtle context, which makes it suitable for a variety of applications. This new embedding model achieves the top rank on the Massive Text Embedding Benchmark (MTEB) Multilingual leaderboard, outperforms its previous state-of-the-art model (text-embedding-004), and has new features including greater input token length!
Embeddings in the Gemini API
Several embedding models that produce cutting-edge embeddings for words, phrases, code, and sentences are supported by the Gemini API. The generated embeddings can then be applied to a variety of tasks, including text categorisation, clustering, and semantic search.
Describe embeddings Gemini
Texts with similar meanings have “closer” embeddings because embeddings capture semantic meaning and context. For instance, because they both describe a similar environment, the sentences “I took my dog to the vet” and “I took my cat to the vet” would have embeddings in the vector space that are near to one another.
Embeddings can be used to compare and comprehend the relationships between various texts. For instance, you can deduce that the words “cat” and “dog” have comparable meanings, contexts, or both if their embeddings are near to one another. Many typical AI use cases are made possible by this.
Use cases
Numerous typical AI use cases make use of text embeddings, including:
- Information retrieval: Given a segment of input text, you can utilise embeddings to extract semantically related material.
- Clustering: Hidden trends can be found by comparing sets of embeddings.
- Vector database: It is typical practice to store embeddings in a vector database as you move various embedding use cases to production.
- Classification: Documents can be categorised by training a model with embeddings.
Embedding models
Three models that produce text embeddings are available through the Gemini API:
- Gemini-embedding-exp-03-07
- Text-embedding-004
- Embedding-001
In the upcoming months, Google anticipates releasing upgraded versions of the Gemini embedding model.
Google’s best text embedding model yet
Embeddings Gemini model has been developed to be exceptionally general, exhibiting outstanding performance in a variety of fields, such as search, science, finance, and law. It functions well right out of the box and doesn’t require a lot of fine-tuning for particular activities.
In order to provide a thorough benchmark for model comparison, the MTEB (Multilingual) leaderboard scores text embedding models across a variety of tasks, including retrieval and classification. With a mean (task) score of 68.32, Google’s Gemini Embedding model outperforms the next rival model by a margin of +5.81.
What is the purpose of embeddings?
The ability of LLMs to comprehend the meaning of text is essential for developing intelligent retrieval augmented generation (RAG) and recommendation systems as well as for classifying text. In addition to lowering costs and latency, embeddings are frequently essential for creating more effective systems that outperform keyword matching algorithms in general. Embeddings use numerical data representations to capture context and semantic meaning. The embeddings of data with similar semantic meanings are more closely related. Numerous uses are made possible by embeddings, such as:
- Effective Retrieval: By comparing the embeddings of queries and documents, you may locate pertinent documents in big databases, such as enterprise search or legal document retrieval.
- RAG stands for retrieval-augmented generation. By accessing and integrating contextually relevant information into a model’s context, you can improve the resulting text’s quality and relevancy.
- Clustering and Categorisation: Assemble related texts into groups to find themes and patterns in your data.
- Classification: Use tools like sentiment analysis or spam detection to automatically group texts according to their content.
- Text Similarity: Recognise duplicate text to facilitate activities like plagiarism detection and web page deduplication.
Get Gemini Embedding started
Google’s new experimental Embeddings Gemini model is now available to developers via the Gemini API. The current embed_content endpoint is compatible with it.
from google import genai
client = genai.Client(api_key="GEMINI_API_KEY")
result = client.models.embed_content(
model="gemini-embedding-exp-03-07",
contents="How does alphafold work?",
)
print(result.embeddings)Apart from enhanced quality in every aspect, Embeddings Gemini also offers:
- There is an 8K token restriction on input: You can now embed lengthy passages of text, code, or other data because we’ve increased the context length from earlier models.
- 3K dimensions are the output dimensions: high-dimensional embeddings that have nearly four times as many tokens than earlier models.
- Matryoshka Representation Learning (MRL): MRL enables you to scale down to your preferred storage cost by truncating the original 3K dimensions.
- Increased language support: We now offer more than 100 languages, which is a twofold increase.
- Unified model: This model outperforms its earlier code-specific, English-only, and task-specific multilingual models.
This version offers you an early look into Embeddings Gemini capabilities, even if it is still in an experimental phase with restricted capacity.
