The developer created a local RAG system to index and search through a personal collection of text files. The project requirements included local-only processing without cloud services and low hardware requirements, specifically targeting systems with 4 GB of RAM.
The initial implementation used Python, the 'intfloat/multilingual-e5-base' model for vector generation, and FAISS for vector storage. Metadata was stored in JSON files. To improve performance, the system was later modified to run as an HTTP server, allowing external clients to send queries independently of the indexing scripts.
The architecture evolved to offload vector generation to Ollama, which runs on a separate network server. This allowed for testing different models and decoupling the processing logic from the Python environment.
Seeking to replace Python, the developer migrated the system to Perl. The final setup utilizes SQLite with the 'SQLite::VecDB' extension for storing vectors and metadata, while Ollama continues to handle vector calculations via API requests.
