An experiment analyzed how conversation history influences CRM recommendations in ChatGPT, Perplexity, and Alice AI. The study involved 45 dialogues, with each of five scenarios repeated three times per model. All sessions started in new chats, and the final question regarding CRM selection for a Russian company remained consistent.
The researcher tested various contexts, including no history, specific business requirements (small company or long sales cycles), and user-initiated mentions of specific CRM products like BPMSoft. The results were evaluated using the Jaccard index to compare the overlap of suggested CRM lists against a baseline.
Data showed that providing context significantly altered the composition of recommended CRM systems compared to baseline responses. For instance, when BPMSoft was mentioned by the user, it appeared in all final recommendations across all three models, often shifting the ranking of previously suggested tools like Bitrix24.
The study highlights that AI recommendations are highly sensitive to preceding dialogue. The researcher notes that measuring brand visibility requires distinguishing between organic suggestions and those prompted by user input, as combining these metrics can skew results regarding how often a brand is recommended without prior mention.
