How does GPT-5's AIME and HMMT performance compare to GPT-4 on hard problems

AIME (American Invitational Mathematics Examination) Performance

GPT-5 sets a new standard in AIME performance, reaching 94.6% accuracy on the 2025 competition without any auxiliary tools or code execution. This is a significant improvement over available GPT-4 variants, which achieved far lower accuracy rates on analogous problems. The near-perfect score achieved by GPT-5 underscores a qualitative transformation: the model now solves not only standard, well-posed AIME problems but also handles the trickier, less routine questions that typically stump both exam candidates and prior AI models.

GPT-4 models, in contrast, struggled to surpass the 75-80% range on hard AIME problems and performed with considerably less reliability on newer problem types that demand multi-layered reasoning and creative deduction. The adoption of deeper chain-of-thought processing in GPT-5 allows for more robust solution steps, error checking, and alignment with the multi-step rationales required by AIME's toughest questions.

Hard Problem Handling

Where GPT-5 truly distinguishes itself is in the solution of “hard” AIME problems, which historically involve multiple hidden steps, clever tricks, and require flexible, non-linear thought. GPT-5's "thinking" or “deep reasoning” mode—activated for prompts flagged as complex—enables longer, more detailed intermediate reasoning. This approach dramatically increases answer reliability and replicates the kinds of strategies seen among human math olympiad medalists, rather than the more template-driven pattern-matching of previous models.

In both controlled evaluations and real-world user reports, GPT-5 demonstrates an ability to correctly navigate the pitfalls, dead ends, and subtle misdirections presented by the hardest AIME problems. Hallucination rates—instances where the model confidently provides incorrect answers—have dropped substantially, implying improved self-evaluation and error mitigation processes.

HMMT (Harvard-MIT Mathematics Tournament) Performance

GPT-5's HMMT results further cement its reputation for mathematical prowess. Across multiple problem sets, including the 2025 HMMT, GPT-5 consistently achieves above 90% accuracy, with the highest-end versions moving close to perfect performance on entire rounds. On the most complex HMMT questions—traditionally a graveyard for even top students—GPT-5's "Pro" version routinely solves every problem when allowed to use code or explicit tool use, and tops 93% accuracy with reasoning alone.

GPT-4's historical HMMT outcomes generally hovered around the 70% accuracy mark and dropped further for the especially intricate problems, frequently producing lapses in logical rigor, missing key edge cases, or failing to integrate constraints across multiple variables. GPT-5, by contrast, can be prompted to explain its full step-by-step rationale, exhibits non-trivial checking behavior, and systematically avoids classically fatal errors, such as misapplying combinatorial identities, overcounting, or mishandling bounding arguments.

Hard Problems: Qualitative Shifts

GPT-5's greatest advance over GPT-4 is in questions located at the extreme end of HMMT's difficulty spectrum, including combinatorial construction, invariants, and deep geometry problems. Here, the earlier GPT-4 would either guess, stall, or offer plausible-but-incomplete reasoning. GPT-5, benefiting from improved context retention and explicit problem-type awareness, not only completes the full logical chain but also offers alternative solutions and, when prodded, is capable of critiquing, refactoring, and even producing nonstandard but valid arguments—something never observed in GPT-4's mathematical outputs.

Architectural and Methodological Improvements Driving Gains

The stark performance gap between GPT-4 and GPT-5 on AIME and HMMT hard problems derives not simply from scale, but from deep architectural and training differences.

- Dynamic Reasoning Modes: GPT-5 implements multiple internal modes ("Default," "Thinking," "Pro"), with a routing system that selects computational resources based on prompt complexity. Thus, HMMT-hard and AIME-hard triggers invoke more memory, greater depth, and iterative refinement passes than previous models.

- Extended Context: GPT-5's much larger context window supports reviewing, cross-referencing, and recombining multiple problem parts simultaneously, a clear advantage for multi-part problems and those requiring synthesis of several strategies.

- Continuous Learning and Tool Integration: Coupled with real-time reinforcement from user correction and self-critique, GPT-5 further improves with exposure—closing performance gaps even on rare or previously unsolved problem types.

- Lower Hallucination Rates and Self-Evaluation: Enhancements to GPT-5's internal confidence estimation produce substantially fewer false positives, especially on problems where stepwise verification is possible. The model is much more likely to “admit” when it is uncertain or to flag questionable steps.

- Superior Multi-step Reasoning: GPT-5's advances are especially evident in questions requiring layered deductions, including intermediate assertions unsupported by direct input, guessing-and-checking, and construction-based arguments (which defeated GPT-4 and earlier models far more often).

Benchmark Status and Limitations

According to leading independent benchmarking authorities, GPT-5 currently leads all OpenAI models on OTIS Mock AIME, GPQA Diamond, and FrontierMath benchmarks, with its "medium" variant achieving ~87% on carefully curated AIME-like datasets and its "Pro" configuration leading nearly all competitors on edge-case, high-difficulty mathematical reasoning scenarios.

However, GPT-5's lead is not universal or absolute. On certain specialized tasks, focused models—such as Anthropic's latest Claude variants—have edged ahead in narrowly optimized benchmarks, though not by statistically significant margins on AIME or HMMT per se. In these rare cases, the edge is attributed to targeted architectural tweaks rather than general mathematical intelligence. Nonetheless, GPT-5 remains the strongest all-around performer, especially when flexibility and adaptability on hard, unseen problems are required.

Comparison with GPT-4: Beyond Scores

GPT-4 represented a major leap over its predecessor on prior AIME and HMMT tasks, pushing AI toward parity with strong high school competitors. Notably, GPT-4 could sometimes produce near-perfect AIME sets in simulation but was inconsistent, requiring careful prompt engineering and often failing to generalize from one problem format to the next. It frequently tripped on “trap” questions requiring more than surface-level comprehension or overlooked subtle logical interdependencies.

GPT-5 does not simply improve score percentages but fundamentally alters the character of its solutions, offering:

1. Human-Like Scrutiny: GPT-5 queries its own reasoning, proposes multiple solution routes, and volunteers counterexamples—capabilities far beyond the reflexive, single-path approach in most GPT-4 runs.
2. Reduced User Supervision: Even on new or adversarial problem types, GPT-5 requires much less guidance, solving independently rather than relying on continuous re-prompting or nudging as GPT-4 often did.
3. Transfer to Related Domains: The same depth of mathematical comprehension enables superior performance on graduate-level GPQA benchmarks, coding tasks, and multimodal math (e.g., diagram analysis), outpacing GPT-4's largely lexical and pattern-driven reasoning.
4. Stability and Repeatability: Solutions to hard problems are robust across multiple runs, with far less sensitivity to prompt phrasing, user context, or sequencing—in contrast to GPT-4's inconsistent behavior under small prompt shifts or longer sessions.

Examples of Hard Problem Effectiveness

On concrete 2025 AIME and HMMT problems released publicly, GPT-5 demonstrates:

- Accurate application of unfamiliar theorems, sometimes not explicitly named or labeled in the question text
- Formal proof-style explanation, including outlining all edge cases, verifying solution bounds, and making explicit reference to constraints
- Flexible adaptation to mixed-format problems—including geometry with embedded combinatorics or algebraic inequalities nested within number theory frameworks

Hard combinatorial geometry—an area where GPT-4 often got stuck—becomes tractable for GPT-5 through context-aware diagram synthesis (either textual or visual, depending on modality).

Remaining Gaps and Nuances

While GPT-5's overall benchmark supremacy on hard AIME and HMMT questions is clear, a few nuances remain:

- On *ultra-novel* problem types, where no clear precedent exists in training data, solution quality can revert to stepwise guesswork. Still, the baseline is consistently above GPT-4's best outputs, and partial progress is more frequent and interpretable.
- GPT-5 occasionally produces longer, more detailed answers which, while correct, may contain unnecessary steps—a byproduct of its robustness mechanisms. This sometimes introduces ambiguity in auto-graded environments. However, for human-read assessment (as in olympiad marking), the depth is overwhelmingly positive.
- “Mini” and “nano” GPT-5 models show expected accuracy declines (~83-85% AIME accuracy for “mini,” ~60-70% for “nano”), but both still outperform GPT-4 class models of similar cost or latency envelope, reaffirming scalable architecture advantages.

Broader Implications

GPT-5's capacity on AIME and HMMT “hard” problems illustrates the maturity of language models as domain experts—not merely search engines or pattern matchers, but as true partners in advanced math. It foreshadows their utility across academic research, STEM education, automated theorem proving, and as aids for real-world problem solving where rigor and creativity are equally required.