LLM-box vs Thinking-box

Thinking-box introduces a user-level intervention framework that scaffolds deliberate user engagement with distorted information in Large Language Model (LLM)-powered conversational search. While technical mitigations effectively reduce hallucination rates, deceptive errors inevitably persist within confident narratives, posing significant recognition challenges for daily information seekers. To address this risk at the user interface level, this system utilizes a sentence-level reflective prompt mechanism derived from a formalized taxonomy of consistent hallucination patterns. Rather than delivering automated correctness verdicts on behalf of users, Thinking-box operates as a non-judgmental checking tool—introducing constructive friction through suggestive checkpoints that prompt line-by-line verification. A within-subjects user study ($N=16$) confirms that this localized, uniform disruption of reading flow significantly improves a user's ability to filter distorted claims. Although this thorough inspection demands higher cognitive load and task completion time compared to assertive model-level feedback mechanisms (such as LLM-as-a-judge approaches), Thinking-box effectively preserves user agency, prevents tunnel-vision reliance on automated helpers, and fosters long-term, internalized habits of independent information validation.