What is embedding? The method that turns words into vectors while keeping their meaning
Embedding converts words, phrases and sentences into number vectors that preserve semantic proximity. The article explains how the method works, the difference between Word2Vec and BERT, Turkish models, and their use in semantic search and chatbots.
Nurullah Okatan · Content Writer
· Updated: · 13 min read

Embedding is the method of converting a word, sentence or document into a sequence of numbers, that is, a vector, in a way that preserves its meaning. Because semantically similar texts receive vectors that are close to each other, this method is used in semantic search, chatbots and document matching.
How does embedding turn text into numbers?
An embedding model takes a text and converts it into a list of hundreds of numbers. This list is called a vector. According to the TurkEmbed4Retrieval paper, the embedding method transforms words, phrases and sentences into mathematical vectors. The model determines these numbers during training. It looks at which words appear together with which other words across millions of texts.

Each number in the vector indicates a position along an axis. In this way, every text lands on a point in a multidimensional space. This representation has fewer dimensions than sparse tables that represent text through word counts, and it carries continuous values.
Closeness in the space means closeness in meaning. Semantically similar words are placed close together, and words with different meanings are placed far apart. "Invoice" and "payment" fall close together, while "invoice" and "camping tent" remain far apart. The closeness of two vectors is usually measured by cosine similarity. As the angle between the vectors decreases, the meanings of the two texts also come closer.
An embedding model can take both words and sentences as input. First-generation models such as Word2Vec assign vectors to individual words. Newer models convert a sentence, paragraph or document segment into a single vector. In search and chatbot projects, paragraph-level vectors are mostly used. Since the user's question is also a full sentence, the question and the document segment are compared at the same level.
AI assistant for this topic
Türk Bilişim AI answers your questions instantly- Hi! Ask whatever you're curious about.
The difference between Word2Vec and BERT embeddings
Word2Vec gives the same vector to a word regardless of the sentence in which it appears. BERT, by contrast, recalculates the vector of a word each time by looking at the surrounding words. According to Chris McCormick's article on BERT embedding training, this is the core difference between the two models. In Word2Vec the representation stays fixed, while in BERT it changes depending on context.
According to the Airbyte guide, Word2Vec is a neural network-based model introduced by Google researchers in 2013. During training, the model learns a single vector for each word and stores it like a dictionary. At use time, it looks up this dictionary and does not take the rest of the sentence into account. That is why the word "yüz" receives the same numbers in the sentences "yüz lira" and "yüzünü yıka".
For BERT, the dimension and length limits are clear. According to the Airbyte guide, BERT-Base produces 768-dimensional vectors and BERT-Large produces 1024-dimensional vectors. According to the same guide, BERT processes sequences of up to 512 tokens, and each position takes its own learned vector. A longer document is split into parts and each part is converted into a vector.
A developer who examines BERT's output sees 13 layers. According to McCormick's explanation, the first element is the input embedding, and the remaining elements are the outputs of BERT's 12 layers. When extracting a word vector, the outputs of the last few layers are often combined.
| Feature | Word2Vec | BERT |
|---|---|---|
| Word representation | The same vector in every context | A vector that changes according to surrounding words |
| Origin | Google researchers, 2013 | Contextual language model |
| Vector dimension | Chosen during training | Base 768, Large 1024 |
| Input length | Single word | Up to 512 tokens |
| Output structure | A single word table | Input embedding and 12 layers, 13 elements in total |
Embedding models developed for Turkish texts
One of the models developed for Turkish is TurkEmbed4Retrieval. The model was trained for the task of finding the most suitable text for a question. According to the TurkEmbed4Retrieval paper, the 305-million-parameter GTE-multilingual-base was chosen as the base model. The authors justify this choice by the model's performance. The model was not built from scratch; it was trained with Turkish data on top of a multilingual model.
One of the datasets used in training is MS-Marco-TR. According to the same paper, this dataset is the Turkish translation of the MS-Marco dataset and contains 1 million query-text pairs. Each query is paired with one positive and one negative text. The model thus learns to keep the correct answer close to the question and the irrelevant text far away.
The structure of Turkish requires this adaptation. A single word such as "evlerimizden" carries information that English expresses with several words. General multilingual models split such words into many small pieces. A model trained on Turkish data is tuned to lead users who write the same question with different suffixes to the same answer. Other articles on similar models and methods can be found in the Artificial Intelligence & Automation category.
How is embedding used in semantic search and chatbots?
Keyword search checks whether the words in the query appear verbatim in the document. Semantic search, on the other hand, converts the query and the documents into vectors and then retrieves the vectors closest to the question. A user who writes "where is my cargo" can thus find a page that only contains "shipment tracking". Chris McCormick's BERT article also finds embedding vectors useful for query expansion, semantic search and information retrieval.

A chatbot finds the answer in the organization's documents through the following steps:
- Contracts, product manuals and FAQ pages are split into paragraph-sized chunks.
- Each chunk is converted into a vector with the embedding model and written to a vector database.
- When the user asks a question, the question is also converted into a vector with the same model.
- The database returns the chunks closest to the question.
- The language model writes the answer based on these chunks and shows the source.
This architecture is called Retrieval-Augmented Generation (RAG). The chatbot does not generate its answer from the language model's general knowledge. It retrieves it from the institution's current documents. When a document changes, only the vector of that section is recalculated; the rest of the system is left untouched.
Türk Bilişim, which is considered one of the best AI Chatbot & Customer Assistant companies in Turkey, has been developing artificial intelligence solutions since 2022. The company's work in this field covers chatbots, automation and content generation. Document-based question-answering flow is the subject of the company's AI Chatbot & Customer Assistants service. The cases where a ready-made model does not recognize the institution's terms, and the connection of the model to existing systems, are addressed under the heading Custom AI Models & Integration.
What a company should clarify before starting an embedding project
The first decision is which data will be converted into vectors. Documents that answer the questions customers ask most often come first: product manuals, return conditions, technical documentation. Records containing personal data, expired price lists and internal correspondence should be left out. Every text converted into vectors may one day appear in a chatbot answer.
Embedding lowers storage costs by compressing large data structures. Even so, every unnecessary document indexed is reprocessed with each update and adds noise to search results.
The criterion in model selection is the language in which users ask their questions. If queries are mostly in Turkish, a model adapted to Turkish aims for more accurate matching of suffixed words. If documents in different languages are together in the same database, a multilingual model matches them all in a single space. Türk Bilişim projects can be carried out in Turkish, English, Arabic and Russian. The company's artificial intelligence solutions page presents the services in this field together.
For selection, a small test set is prepared from the institution's real customer questions. The correct document for each question is marked by hand. Two candidate models are tested with the same set. The model that brings the correct document to the top positions is chosen.
Looking for Corporate AI Training?Frequently asked questions
Are embedding and tokenization the same thing?
No. Tokenization splits the text into small pieces (tokens) that the model can process. Embedding converts these pieces and then the whole text into a numerical vector; tokenization always comes first.
Does an embedding model understand Turkish suffixes correctly?
The model processes suffixes by splitting them into sub-parts. How well it captures the relationship between "faturamı" and "fatura" depends on how much Turkish data the model has seen. Models trained on Turkish data are tuned to learn this relationship.
Where are embedding vectors stored?
They are stored in a vector database or in a database extension that supports vector search. The source document and text fragment are also kept next to each vector, so that the chatbot answer can be linked to its document.
Is it necessary to train a custom embedding model on the institution's data?
In most projects it is not necessary; a ready-made model is enough to convert the institution's documents into vectors. When domain terms are intensive, additional training is done on top of the ready-made model with the institution's question-answer pairs, as in TurkEmbed4Retrieval.
Türk Bilişim · Free Quote
Get a free quote for your Automatic SEO Article Generation project
Leave your number and we will get back to you right away. The call is free.
Your request is saved on this device
We can't reach our server right now. Your request will be sent automatically once the connection is back. To talk right away:

Share this article
Comments
Comments
No comments yet.
















