OpenAI operates an internal initiative known as Project Lily, which involves human reviewers assessing ChatGPT conversation logs to improve model performance. These reviewers, referred to as prompt auditors, evaluate responses for quality, tone, and adherence to guidelines, such as avoiding anthropomorphism or fabricated personal experiences.
Reviewers receive an hourly wage exceeding $50 for this task. According to the project documentation, the process involves reviewing anonymized logs, though OpenAI acknowledges that personal data may occasionally remain visible. Some conversation samples provided to reviewers include user memory summaries containing interests, context, and location data.
Project Lily focuses on response quality rather than factual accuracy or safety monitoring, which are handled by separate teams. While OpenAI states in its FAQ that human review is used for model optimization, the company does not explicitly detail the manual reading process in its help documentation, even after recent updates.
Data collection settings for product improvement are enabled by default for most ChatGPT users, though they are disabled for enterprise, business, and education accounts. Deleting chat history does not remove data that has already been processed and stored in training datasets, as the deletion mechanism is not retroactive.
