Project Lily is putting an uncomfortable question back in front of ChatGPT users: who can see what you tell an AI?
According to a September 2026 report by 404 Media, OpenAI has used hundreds of contractors to review real, anonymized and filtered ChatGPT conversations as part of an internal effort known as Project Lily. The work focuses heavily on identifying and reducing AI sycophancy—the tendency of ChatGPT to agree with, flatter or excessively validate users.
The revelation is significant, but it is not the same as saying OpenAI secretly gives people unrestricted access to everyone’s private chats.
What is Project Lily?
Project Lily is an OpenAI model-improvement project in which human reviewers examine selected ChatGPT interactions and assess how well the AI responds.
The reported goal includes making ChatGPT less sycophantic and less likely to simply tell users what they want to hear.
Human feedback remains an important part of improving large language models. Reviewers can identify whether an answer is useful, accurate, appropriately cautious or excessively agreeable—patterns that are difficult to measure from automated tests alone.
OpenAI’s public policy already says that content from consumer services such as ChatGPT may be used to improve its models unless users opt out.
Can humans read your ChatGPT chats?
Potentially, yes—if the conversation is eligible for model improvement.
OpenAI says it uses privacy safeguards before conversations are incorporated into training data. Its Privacy Filter is designed to detect and mask personal information, and OpenAI says it is applied to user conversations when “Improve the model for everyone” is enabled.
But filtering is not the same thing as making a conversation completely anonymous.
That distinction matters because people increasingly use ChatGPT for highly personal subjects, including relationships, finances, work problems and other sensitive questions.
OpenAI itself advises users not to share sensitive information they would not want used or reviewed.
How to stop ChatGPT conversations being used for training
ChatGPT users can turn off model-improvement sharing.
Go to:
Settings → Data Controls → Improve the model for everyone → Off
OpenAI says that once the setting is disabled, new conversations are not used to train its models.
Another option is Temporary Chat. OpenAI says Temporary Chats are not used to improve its models, do not appear in chat history and do not create memories, although they are retained for up to 30 days for safety purposes.
Why Project Lily matters
The controversy is less about whether OpenAI is allowed to improve its models using user feedback and more about how clearly users understand the human layer behind AI.
ChatGPT can feel like a private conversation with software. In reality, consumer AI services operate inside a much larger ecosystem of data collection, filtering, evaluation and human review.
Project Lily therefore offers a useful rule for AI users:
If you would not want a human reviewer to potentially encounter something, do not put it into a consumer AI chatbot.
The technology may look conversational. The infrastructure behind it is not.
