Designing a corporate GenAI platform requires defining goals, user roles, and security requirements before selecting specific technologies. Architects should focus on non-functional requirements like scalability, auditability, and fault tolerance rather than immediate implementation details.
The system architecture utilizes a modular monolith approach with .NET 10 for business logic and security, while Python handles AI-specific tasks like embeddings and retrieval. This separation allows for independent scaling of resource-intensive AI workloads without the overhead of a distributed system.
For data storage, the project uses PostgreSQL with the pgvector extension instead of a dedicated vector database. This choice simplifies infrastructure by keeping metadata, access control lists, and embeddings in one place, reducing complexity until measurable performance limits are reached.
Security is managed by ensuring authorization occurs on the backend before data reaches the LLM. An LLM Gateway acts as a central point for policy enforcement, managing token usage and data transmission restrictions, while RabbitMQ handles asynchronous document processing tasks.
To prevent architectural drift, the team uses Architecture Decision Records (ADR). These documents capture the context, alternatives, and consequences of key choices, such as the selection of RabbitMQ over Kafka, ensuring that future changes are based on documented reasoning rather than assumptions.
