This article reviews the paper REFRAG: Rethinking RAG based Decoding.
RAG is one of the first methods considered when applying LLMs to services. The key advantage of RAG is its ability to leverage domain knowledge without model fine-tuning.
However, as the knowledge base grows, longer contexts must be fed as
