colpali library

ColPali is a Python library for multi-modal RAG, driving superior document retrieval by processing entire document pages as images to capture visual layout and text.

ColPali (Contextualized Late Interaction Over PaliGemma) is a powerful, open-source Python library designed to revolutionize Retrieval-Augmented Generation (RAG) for complex documents. It bypasses traditional, brittle OCR/text-extraction pipelines; instead, it treats each document page as an image, feeding it directly to a Vision-Language Model (VLM) like PaliGemma-3B. This approach generates detailed multi-vector embeddings (typically 128-dimension) that holistically capture text, tables, and visual structure. This visual-first indexing, coupled with ColBERT-like late interaction scoring, delivers significantly enhanced retrieval accuracy, especially for visually-rich PDFs, outperforming text-only methods on benchmarks like ViDoRe. Deploy it directly with vector databases such as Qdrant or Elastic Search for efficient, production-ready document understanding.

https://github.com/illuin-tech/colpali