What is RAG?

RAG (Retrieval-Augmented Generation) retrieves relevant knowledge first, then lets the LLM generate answers based on retrieved context. Compared to direct LLM answering, RAG provides:

• Up-to-date, accurate knowledge
• Reduced hallucinations
• Private knowledge base support

Architecture

Document Upload → Text Extraction → Text Chunking → 
Embedding → Milvus Vector Store
                                 ↓
User Question → Query Embedding → Similarity Search (Top-K) → 
Prompt Augmentation → LLM Generation

Step 1: Dependencies

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-milvus-store-spring-boot-starter</artifactId>
</dependency>

Step 2: Document Loading with Apache Tika

@Component
public class DocumentLoader {
    private final Tika tika = new Tika();

    public String extractText(MultipartFile file) throws IOException {
        return tika.parseToString(file.getInputStream());
    }
}

Step 3: Smart Text Chunking

Chunk size and overlap strategy directly impact retrieval quality:

private static final int CHUNK_SIZE = 500;
private static final int OVERLAP = 50;

Step 4: Vector Storage with Milvus

public void storeDocument(String text, Map<String, Object> metadata) {
    List<Document> chunks = textSplitter.split(text, metadata);
    vectorStore.add(chunks);  // auto embedding + write
}

Step 5: RAG Query Pipeline

public String ask(String question) {
    // 1. Retrieve relevant docs
    List<Document> docs = vectorStoreService.search(question, 5);
    
    // 2. Build augmented prompt
    String context = docs.stream()
        .map(Document::getContent)
        .collect(Collectors.joining("\n---\n"));
    
    // 3. Generate with LLM
    return chatClient.call(buildPrompt(context, question));
}

Optimization Tips

1. Adjust chunk size by document type (200-300 for code, 500-800 for articles)
2. Combine keyword search (BM25) with vector search for better recall
3. Use Cross-Encoder for result re-ranking
4. Cache frequent queries to reduce LLM calls

Summary

Spring AI provides out-of-the-box RAG support. Combined with Milvus, you can quickly build an intelligent Q&A system. Core flow: Load → Chunk → Embed → Store → Retrieve → Augment → Generate.

Return to the blog index for more entries.