The author suggests evaluating LLMs for professional tasks by defining specific acceptance criteria and using real-world, imperfect data. For coding tasks, he recommends testing models on existing projects to see if they can fix bugs without breaking functionality or changing interfaces. For analytical tasks, the author advises using datasets with known errors to verify the model's ability to handle data cleaning and logic.
Anthropic's Fable 5.1 is priced at $10 per million input tokens and $50 per million output tokens. Opus 5 costs $5 and $25, while Sonnet 5 costs $2 and $10 respectively. The cost of reading cache for Fable has been reduced to $0.25 per million tokens. The author notes that while Fable is more expensive, it may save time on manual corrections.
For analytical tasks, the author compares GPT-5.6 Sol and GPT-6 Astra. Astra is priced at $10 per million input tokens and $50 per million output tokens, while Sol costs $4 and $20. The Sol pricing is promotional and guaranteed until November 21, 2026. Both models feature a context window of 1.05 million tokens.
The author highlights the use of Gemini 3.6 Flash for processing multimodal data, including audio, video, and PDF files. For monitoring social media discussions, he suggests using X Search with Grok 4.6. He emphasizes that evaluation should be based on a controlled set of known data to verify the model's accuracy in classification and retrieval.
