When the People Building AI Start Worrying, Should We Stop Thinking?

On September 8, 2026, a British researcher named Jacob Coxon announced on X that he was leaving Anthropic. He had spent three years doing pretraining research at both OpenAI and Anthropic, and he said neither company was acting responsibly in its race toward ever more powerful systems (ARA, 2026). Within hours, colleagues from inside the industry were echoing his unease. Evan Hubinger, who leads alignment work at Anthropic, put the chance of catastrophic outcomes above 10% (ARA, 2026), and National Public Radio reported the resignation as a warning that the industry may be building tools humans will soon lose control of (NPR, 2026).

These are personal estimates, not settled facts, and risk assessment for AI has no agreed method yet. But they raise a more immediate question that matters for Pakistan: while experts debate distant dangers, what is AI already doing to the way we think?

What the Experts Rank as Urgent

A 2026 study by MIT FutureTech and the University of Queensland asked 272 international experts to rate 24 AI risks over a five-year horizon. They placed dangerous capabilities, competitive pressure between firms, weapons and cyberattacks, concentrated power, and false or misleading information among the most severe (Burnham, 2026). The information sector was judged especially vulnerable, and the study noted a troubling mismatch: those best placed to address the risks are rarely the people most exposed to them. For ordinary users, then, the first line of defence is often their own judgment.

The Optimist's Condition

Bill Gates is far from an AI pessimist. In 2023, he argued that the risks are real but manageable, comparing AI to cars, which brought speed limits and safety rules rather than a ban (Gates, 2023). Yet his argument rests on a condition: users must stay informed. He warned that AI can state falsehoods with confidence and absorb bias from its training data, so people must check its output and their own assumptions. He also described the ability to tell true from false as one of the most important skills today (Gates, 2023). In 2026, he went further, acknowledging that AI can now match and even surpass human cognition (Gates, 2026).

That raises an uncomfortable question. If managing AI depends on critical thinking, what happens when we hand that thinking over to AI?

What Research Says About Thinking

Microsoft Research and Carnegie Mellon University surveyed 319 knowledge workers who used generative AI weekly. People who trusted the tool more applied less critical effort to its answers, while those confident in their own skills scrutinized the output more (The Register, 2025). Their role also shifted from doing the task to supervising it, which only works if one still understands the task. A Swiss study reported alongside this research found that younger users, aged 17 to 25, used AI most and scored lowest on critical thinking tests, while higher education appeared to protect against the effect (The Decoder, 2025).

The Pakistani Picture

Pakistani studies, though small, point in the same direction.

A professor at FAST, Muhammad Abbas, noticed growing uncritical reliance on AI among his students and surveyed 494 Pakistani university students three times. The findings linked heavy ChatGPT use with procrastination, memory loss, and weaker academic performance, and students under high academic stress used it more (Malay Mail, 2024).

A study of 51 literature undergraduates found that 78.4% used AI tools daily or several times a week, and 27.5% relied only on AI summaries without reading the original texts. Students themselves admitted their independent critical voice had weakened and their writing had become more generic (Journal of Research in Humanities, Punjab University).

A survey of 400 students aged 18 to 24 found that 86% used AI tools in daily life, and heavily dependent users reported larger declines in critical thinking and problem-solving (UMRJ, LUMHS).

Research presented in 2025 on high school and university students in Pakistan likewise found over-reliance and uncertainty among institutions about AI's role in learning (Khalid et al., 2025).

To be fair, the evidence is not one-sided. A study of 352 Pakistani university students, published in Scientific Reports, found that ChatGPT improved academic performance, particularly for students struggling with rote learning (ScienceBlog, 2025). The problem, then, is not the tool but how it is used. Most of these Pakistani samples are small and rely on self-reporting, so they indicate trends rather than proof.

What We Can Do

Meeting this challenge requires more than telling students to "use AI responsibly," because the evidence suggests that over-reliance is driven by pressure as much as by habit. Pakistani research found that students under high academic stress were more likely to turn to ChatGPT (Malay Mail, 2024), which implies that exam-heavy, deadline-driven systems push young people toward shortcuts. Individual discipline therefore matters, but it is not enough. Students can build better habits by drafting their own outline or opinion before consulting AI, treating every answer as a claim to verify rather than a fact, reading sources instead of summaries, and using AI to understand ideas rather than to write for them. Yet these habits will only last if institutions reward them. Gates notes that some teachers already ask students to fact-check AI-generated text (Gates, 2023), but this approach assumes teachers have the time, training, and small classes to design such tasks, which is a hopeful assumption for many Pakistani classrooms where rote learning and large enrollments are common. Assessment, then, must change first: tasks that demand personal analysis, oral explanation, or critique of an AI's answer make unthinking reliance far less rewarding. Without such reform, telling children to think for themselves while grading them on what AI can produce in seconds sends them a contradictory message, and they will respond rationally to the incentives they are given.

Whether or not the most dramatic warnings prove correct, one danger is already visible and entirely within our control. Regulation and international coordination belong to governments and companies, but the habit of questioning belongs to each of us. A society that stops asking "How do you know?" will struggle to govern any technology, however powerful.