RAG is one of those AI concepts that sounds simple when you first hear about it:

Retrieve relevant information and give it to an LLM.

But I wanted to understand what actually happens between those two steps.

• What happens when a document is uploaded?

• How is the text extracted?

• How should the document be split?

• What exactly is an embedding?

• How does the system decide which pieces of the document are relevant?

• And finally, how does that retrieved information make its way into the prompt sent to the LLM?

Instead of learning all of this purely through diagrams and tutorials, I decided to build something I could interact with.

That became my RAG Playground.

Why I Built a RAG Playground

I've been exploring AI engineering and wanted to move beyond simply calling an LLM API.

Calling an LLM is relatively straightforward.

The more interesting part, for me, was understanding everything that happens around the LLM.

RAG is a good example.

A typical explanation looks something like this:

Documents
   ↓
Embeddings
   ↓
Vector Database
   ↓
Retrieve Relevant Documents
   ↓
LLM
   ↓
Answer

That's useful, but it hides a lot of implementation details.

For example:

• How to handle different types of files?
• How do we split a document into useful chunks?
• How are those chunks represented as vectors?
• How do we compare a user's question with those vectors?
• How do we decide which chunks are relevant?
• How do we pass those chunks to the LLM?

I wanted to be able to see these steps instead of treating RAG as a black box.

So I decided to build a playground where the pipeline itself becomes the product.

The Idea

The idea behind the playground is straightforward:

Make the RAG pipeline visible and interactive.

Instead of having an application that simply does this:

Question → AI → Answer

I wanted the user to be able to follow the process:

Question
   ↓
Retrieval
   ↓
Relevant Chunks
   ↓
Context
   ↓
LLM
   ↓
Answer

And the document side follows a similar flow:

Document
   ↓
Text Extraction
   ↓
Chunking
   ↓
Embeddings
   ↓
Vector Search

Putting those two parts together gives us the complete RAG workflow.

The RAG Playground

The playground is organized around the major stages of the RAG pipeline:

Add Documents
      ↓
Extract Text
      ↓
Chunking
      ↓
Embeddings
      ↓
Semantic Search
      ↓
Assistant

Each stage represents something that actually happens inside a RAG application.

The goal isn't just to get an answer.

The goal is to understand how the answer was produced.

Step 1: Add Documents

The first step is giving the system some information to work with.

The playground allows users to upload documents that can then be processed by the RAG pipeline.

The supported document formats include:

• PDF
• DOCX
• TXT
• Markdown

The uploaded file itself isn't what the retrieval system searches.

The application first needs to turn the file into usable text.

That's where the next step comes in.

Step 2: Extract Text

A document can have a lot of information inside it, but an LLM or embedding system needs that information in a usable text representation.

So the first processing step is:

File
 ↓
Readable Text

Different file types require different extraction approaches.

For PDFs, I used:

pdf2json

For DOCX files, I used:

Mammoth

TXT and Markdown files can be handled directly as text.

The important thing about this stage is that it establishes the input for everything that follows.

If text extraction isn't correct, everything downstream is affected.

A retrieval problem can sometimes start much earlier than retrieval itself.

Step 3: Chunking

Once the document has been converted into text, we have another problem.

A document can be very large.

We don't necessarily want to treat the entire document as one piece of information.

Instead, we split the document into smaller pieces called chunks.

For example:

Document
   │
   ├── Chunk 1
   ├── Chunk 2
   ├── Chunk 3
   ├── Chunk 4
   └── ...

The playground currently uses a default chunk size of around 1,000 characters.

The reason chunking matters is that retrieval happens at the chunk level.

When the user asks a question, the system isn't searching the entire document as one giant object.

It is looking for the chunks that are most relevant to that question.

Why Chunking Matters

This was one of the areas where building the playground helped me understand RAG better.

It is tempting to think of chunking as just a preprocessing step.

It isn't.

Chunking directly affects retrieval quality.

Imagine a document contains this:

Chapter 1
...
Chapter 2
...
Chapter 3
...

If the chunks are too large, a retrieved chunk may contain a lot of unrelated information.

If the chunks are too small, useful context might be separated across different chunks.

So there is a trade-off.

Too Large
   ↓
More irrelevant context

Too Small
   ↓
Less complete context

The "right" chunk size depends on the type of content and the retrieval strategy.

Building the playground made this much more tangible than simply reading about chunking.

Step 4: Embeddings

Now we have chunks. But how do we search them based on meaning?

This is where embeddings come in.

An embedding is a numerical representation of text.

Conceptually:

"Retrieval augmented generation"
                ↓
        Embedding Model
                ↓
    [0.12, -0.34, 0.78, ...]

The resulting vector represents the semantic characteristics of the text.

Once text has been converted into vectors, we can compare vectors mathematically.

For this playground, I intentionally used a deterministic local embedding implementation:

local-hash-embedding

The implementation produces normalized 384-dimensional vectors.

The goal here wasn't to build the world's best embedding system.

The goal was to make the underlying mechanics easy to experiment with.

Using a local deterministic implementation also keeps the playground focused on the RAG workflow instead of requiring a separate external embedding service.

Step 5: Semantic Search

Now we have:

Document
   ↓
Chunks
   ↓
Embeddings

The next question is:

How do we find the chunks that are relevant to the user's question?

Suppose the user asks:

What is Retrieval-Augmented Generation?

The question itself is converted into an embedding.

We can then compare that vector against the vectors of the document chunks.

The playground uses cosine similarity for this comparison.

Conceptually:

User Question
      ↓
Question Embedding
      ↓
Compare Against Chunk Embeddings
      ↓
Similarity Scores
      ↓
Relevant Chunks

A higher similarity means that the vectors are more closely aligned.

The current implementation uses a cosine similarity threshold of 0.1 and retrieves a limited number of relevant chunks.

This is the part of RAG that is often described simply as:

"Search the vector database."

But building it made me realize that there are several decisions hiding inside that sentence.

What Does "Relevant" Actually Mean?

This is one of the interesting questions in RAG.

The system doesn't inherently understand that a chunk is "correct."

It calculates similarity between representations.

For example:

Question
    ↓
Vector A

Chunk 1
    ↓
Vector B

Chunk 2
    ↓
Vector C

Then we calculate:

similarity(Vector A, Vector B)
similarity(Vector A, Vector C)

The chunks with stronger similarity become candidates for retrieval.

That means retrieval quality depends on several things:

• The quality of the document text
• The chunking strategy
• The embedding representation
• The similarity calculation
• The similarity threshold
• The number of chunks retrieved

So when a RAG answer is wrong, the problem isn't necessarily the LLM.

The wrong information may have been retrieved before the LLM even received the request.

Step 6: Building the Context

Retrieving chunks isn't enough.

We now need to give those chunks to the LLM.

The application takes the relevant chunks and constructs context for the model.

Conceptually:

User Question
      +
Retrieved Context
      ↓
      LLM
      ↓
Generated Answer

In the current playground implementation, up to 5 retrieved chunks are included in the context sent to LLM.

The retrieved chunks are also associated with source information so that the generated response can point back to where the information came from.

This gives us a more complete RAG pipeline:

Document
   ↓
Extract Text
   ↓
Chunk
   ↓
Embed
   ↓
Search
   ↓
Retrieve
   ↓
Build Context
   ↓
LLM (Gemini)
   ↓
Answer

And that's the part I really wanted to understand.

RAG Is Not Just "LLM + Vector Database"

Before building this project, it was easy to think about RAG as something like:

LLM + Vector Database = RAG

After building the pipeline, I see it differently.

The vector search is only one component.

A RAG application is really a chain of multiple stages:

Document
   ↓
Extraction
   ↓
Chunking
   ↓
Embeddings
   ↓
Retrieval
   ↓
Context Construction
   ↓
LLM
   ↓
Answer

Every stage can affect the final result.

For example:

Bad extraction

The document text is incomplete.

↓

Bad chunks

Important information is split poorly.

↓

Poor embeddings

Semantically relevant content isn't represented effectively.

↓

Poor retrieval

The wrong chunks are selected.

↓

Poor context

The LLM receives irrelevant or incomplete information.

↓

Poor answer

The user gets a bad response.

This changed how I think about debugging RAG systems.

Instead of immediately asking:

"Why did the LLM give a bad answer?"

I can ask:

"At which stage did the information become incorrect or irrelevant?"

That's a much more useful way to think about the system.

Making the Pipeline Visible

One of the main reasons I built a playground instead of just another RAG chatbot was observability.

Normally, a user sees:

User
 ↓
Question
 ↓
AI
 ↓
Answer

Everything in between is hidden. I wanted to expose it.

The playground lets you understand:

Question
   ↓
Semantic Search
   ↓
Retrieved Chunks
   ↓
Context
   ↓
Assistant
   ↓
Answer

This is useful for learning, but it is also useful for debugging.

Suppose the answer is wrong.

Instead of only looking at the final response, we can inspect:

1. What text was extracted?
2. What chunks were created?
3. What embeddings were generated?
4. Which chunks were retrieved?
5. What similarity scores did they have?
6. What context was sent to the model?
7. What did the model generate?

That makes RAG much less mysterious.

Metadata Matters Too

While working on retrieval, I also started thinking more about metadata.

A vector by itself isn't very useful from a user-experience perspective.

Once a chunk is retrieved, we also want to know where it came from.

Metadata can help connect the retrieved chunk back to its source.

This becomes especially useful when displaying sources alongside an answer.

Instead of simply saying:

Here is the answer.

the application can provide additional context about the retrieved information.

This is important because one of the benefits of RAG is not only generating an answer, but making that answer more grounded in external information.

What Building This Changed for Me

The biggest takeaway wasn't one particular technology. It was the mental model.

Before building the playground, I understood RAG conceptually.

After building it, I could trace the entire lifecycle of information.

                DOCUMENT
                   │
                   ▼
            TEXT EXTRACTION
                   │
                   ▼
                CHUNKS
                   │
                   ▼
              EMBEDDINGS
                   │
                   ▼
             VECTOR SEARCH
                   │
                   ▼
           RELEVANT CONTEXT
                   │
                   ▼
                  LLM
                   │
                   ▼
                ANSWER

And that made RAG feel much less like magic.

From Understanding RAG to Building AI Systems

This project started as a playground. But it became more than that.

It gave me a better understanding of what building AI-powered applications actually involves.

It's not just about choosing an LLM. It's about building the system around the model.

Data
 ↓
Processing
 ↓
Retrieval
 ↓
Context
 ↓
Model
 ↓
Application
 ↓
User Experience

That's the part of AI engineering that I find most interesting.

Final Thoughts

Building the RAG Playground helped me move from knowing what RAG is to understanding how RAG actually works.

I can now look at a RAG application and break it down into individual stages:

Document
   ↓
Extract
   ↓
Chunk
   ↓
Embed
   ↓
Retrieve
   ↓
Build Context
   ↓
Generate
   ↓
Answer

And that's exactly what I wanted when I started this project.

Sometimes the best way to understand a technology isn't to read another explanation.

It's to build it.

Try the RAG Playground

If you'd like to see the workflow yourself, you can try the RAG Playground

You can upload a document, follow the processing pipeline, explore the retrieved context, and see how that context is used to generate the final response.

What's Next?

There are still plenty of things I want to experiment with:

• Better embeddings
• Smarter chunking
• Hybrid retrieval
• Reranking
• RAG evaluation
• Better observability
• Realtime AI processing
• More production improvements

I'll be writing about those experiments as I continue building.