Notes

Parent Document Retriever

• Parent Document Retriever
• Original documents are split up
• Embeddings are created from the split up documents
• Large split documents can produce inaccurate answers
• Another strategy is to split up the documents further
  • Parent
  • Parent Split
  • Child Split
  • Embeddings from Child Split Docs

Naive RAG

• Challenges
  • Not enough to simply give context
  • Low precision
    • Chunks are not all relevant
    • Hallucinations
    • Correct context may be lost
  • Low recall
    • Lacking context
  • Outdated information
  • Bad response generation
    • Hallucination
    • Bias
    • Non-relevant response
• How to fix
  • Data
    • Can we make the data pipeline better?
    • Better chunking
  • Embeddings
  • Retrieval
    • Can be do better than top k?
  • Synthesis
    • What can the LLM be used for other than generation?
• Measuring Performance
  • Evaluation
    • Both in isolation and e2e
  • Retrieval metrics
  • Make sure items returned actually answer the question
  • Possibilities
    • Labeled datasets
    • Human feedback
    • Generated
  • Evaluators
    • MRR
    • Precision@K
    • NDCG
  • Evaluation E2E
    • Evaluation of final response
    • Create dataset
      • Input
      • Output - ground-truth answer
    • Run through RAG pipeline
    • Collect metrics
• Optimizing
  • Table Stakes
    • Better parsers
    • Chunk sizes
      • Can have an outsized impact
      • More tokens ≠ better performance
    • Hybrid search
    • Metadata filters
      • Add structured context to chunks
      • Raw semantic search is low precision
      • Are there metadata tags we can infer?
        • “Can you tell me the risk factors in 2021?”
        • year = 2021
        • Look for chunks with that year, remove others
  • Advanced Retrieval
    • Reranking
    • Recursive retrieval
    • Embedded tables
    • Small-to-big
      • Embed text at the sentence-level, then expand that window during LLM synthesis
      • Avoids lost in the middle problems
      • Alternatively, embed a smaller reference to the parent chunk. Use the parent for synthesis.
  • Fine-tuning
    • Embeddings
      • Generate a synthetic query dataset from raw text chunks
      • Can either fine tune the base model itself or an adapter model on top of it
      • Avoid reindexing of document corpus
    • LLM
      • Generate a synthetic dataset from raw chunks
  • Agentic Behavior
    • Routing
    • Query Planning
    • Multi-document agents

Links

Advanced RAG 02 - Parent Document Retriever

Building Production-Ready RAG Applications: Jerry Liu