At Topliner, we use AI to assess candidate relevance for executive search projects. Specifically, we rely on GPT-4o, because, well… at the time it was among the sharpest knives in the drawer. And to be fair, it mostly works. Mostly. mostly The problem? Every now and then, GPT-4o goes rogue. It decides that a perfectly relevant candidate should be tossed aside, or that someone utterly irrelevant deserves a golden ticket. It’s like flipping a coin, but with a fancy API. Predictability is out the window, and in our line of work, that’s unacceptable. So, I started wondering: is it time to move on? Ideally, the new model should be available on Microsoft Azure (we’re already tied into their infrastructure, plus shoutout to Microsoft for the free tokens - still running on those, thanks guys). But if not, any other model that gets the job done would do. Here’s what matters to us: Accuracy – Top priority. If we run the same candidate profile through the system twice, the model should not say “yes” once and “no” the next time. Predictability and correctness are everything. Speed – If it thinks too long, the whole pipeline slows down. GPT-4o’s ~1.2 seconds per response is a pretty good benchmark. Cost – Ideally cheaper than GPT-4o. If it’s a lot cheaper, even better. Accuracy – Top priority. If we run the same candidate profile through the system twice, the model should not say “yes” once and “no” the next time. Predictability and correctness are everything. Accuracy Speed – If it thinks too long, the whole pipeline slows down. GPT-4o’s ~1.2 seconds per response is a pretty good benchmark. Speed Cost – Ideally cheaper than GPT-4o. If it’s a lot cheaper, even better. Cost a lot Recently, I stumbled upon xAI’s new Grok-4 Fast Reasoning model, which promised speed, affordability, and smart reasoning. Naturally, I put it to the test. Grok-4 Fast Reasoning The Setup I designed a test around one “problem candidate profile” - a case where GPT-4o typically fails. The prompt asked the model to decide if a candidate had ever held a role equivalent to “CFO / Chief Financial Officer / VP Finance / Director Finance / SVP Finance” at SpaceX (with all the expected variations in title, scope, and seniority). problem candidate profile Here’s the prompt I used: Evaluate candidate's eligibility based on the following criteria. Evaluate whether this candidate has ever held a role that matches or is equivalent to 'CFO OR Chief Financial Officer OR VP Finance OR Director Finance OR SVP Finance' at 'SpaceX'. Consider variations of these titles, related and relevant positions that are similar to the target role(s). When making this evaluation, consider: - Variations in how the role title may be expressed. - Roles with equivalent or similar or close or near scope of responsibilities and seniority level. - The organizational context, where titles may reflect different levels of responsibility depending on the company's structure. If the candidate's role is a direct or reasonable equivalent to the target title(s), set targetRoleMatch = true. If it is unrelated or clearly much below the intended seniority level, set targetRoleMatch = false. Return answer: true only if targetRoleMatch = true. In all other cases return answer: false. Candidate's experience: [here is context about a candidate] Evaluate candidate's eligibility based on the following criteria. Evaluate whether this candidate has ever held a role that matches or is equivalent to 'CFO OR Chief Financial Officer OR VP Finance OR Director Finance OR SVP Finance' at 'SpaceX'. Consider variations of these titles, related and relevant positions that are similar to the target role(s). When making this evaluation, consider: - Variations in how the role title may be expressed. - Roles with equivalent or similar or close or near scope of responsibilities and seniority level. - The organizational context, where titles may reflect different levels of responsibility depending on the company's structure. If the candidate's role is a direct or reasonable equivalent to the target title(s), set targetRoleMatch = true. If it is unrelated or clearly much below the intended seniority level, set targetRoleMatch = false. Return answer: true only if targetRoleMatch = true. In all other cases return answer: false. Candidate's experience: [here is context about a candidate] Simple in theory, but a surprisingly effective way to separate models that understand nuance from those that hallucinate or guess. I ran the experiment across 9 different models, including: 9 different models All the latest OpenAI releases: GPT-4o, GPT-4.1, GPT-5 Mini, GPT-5 Nano, GPT-5 (August 2025), plus o3-mini and o4-mini. xAI’s Grok-3 Mini and Grok-4 Fast Reasoning. All the latest OpenAI releases: GPT-4o, GPT-4.1, GPT-5 Mini, GPT-5 Nano, GPT-5 (August 2025), plus o3-mini and o4-mini. All the latest OpenAI releases: GPT-4o, GPT-4.1, GPT-5 Mini, GPT-5 Nano, GPT-5 (August 2025), plus o3-mini and o4-mini. xAI’s Grok-3 Mini and Grok-4 Fast Reasoning. xAI’s Grok-3 Mini and Grok-4 Fast Reasoning. Final Comparison Across All Models 📊 Performance Ranking (by average response time): Performance Ranking (by average response time): Azure OpenAI GPT-4o: 1.26s (avg), 0.75-1.98s (range), 1/10 correct (10%), $12.69 per 1000 req Azure OpenAI o4-mini: 2.68s (avg), 1.84-3.53s (range), 10/10 correct (100%), $5.47 per 1000 req xAI Grok-4 Fast Reasoning: 2.83s (avg), 2.39-4.59s (range), 10/10 correct (100%), $0.99 per 1000 req OpenAI GPT-4.1: 3.58s (avg), 2.66-5.05s (range), 0/10 correct (0%), $10.80 per 1000 req Azure OpenAI o3-mini: 4.23s (avg), 2.56-5.94s (range), 10/10 correct (100%), $5.53 per 1000 req xAI Grok-3 Mini: 5.65s (avg), 4.61-6.99s (range), 10/10 correct (100%), $1.47 per 1000 req OpenAI GPT-5 Nano: 8.04s (avg), 6.46-10.44s (range), 10/10 correct (100%), $0.29 per 1000 req OpenAI GPT-5 Mini: 9.7s (avg), 5.46-20.84s (range), 10/10 correct (100%), $1.37 per 1000 req OpenAI GPT-5 2025-08-07: 13.98s (avg), 9.31-21.25s (range), 10/10 correct (100%), $6.62 per 1000 req Azure OpenAI GPT-4o: 1.26s (avg), 0.75-1.98s (range), 1/10 correct (10%), $12.69 per 1000 req Azure OpenAI o4-mini: 2.68s (avg), 1.84-3.53s (range), 10/10 correct (100%), $5.47 per 1000 req xAI Grok-4 Fast Reasoning: 2.83s (avg), 2.39-4.59s (range), 10/10 correct (100%), $0.99 per 1000 req OpenAI GPT-4.1: 3.58s (avg), 2.66-5.05s (range), 0/10 correct (0%), $10.80 per 1000 req Azure OpenAI o3-mini: 4.23s (avg), 2.56-5.94s (range), 10/10 correct (100%), $5.53 per 1000 req xAI Grok-3 Mini: 5.65s (avg), 4.61-6.99s (range), 10/10 correct (100%), $1.47 per 1000 req OpenAI GPT-5 Nano: 8.04s (avg), 6.46-10.44s (range), 10/10 correct (100%), $0.29 per 1000 req OpenAI GPT-5 Mini: 9.7s (avg), 5.46-20.84s (range), 10/10 correct (100%), $1.37 per 1000 req OpenAI GPT-5 2025-08-07: 13.98s (avg), 9.31-21.25s (range), 10/10 correct (100%), $6.62 per 1000 req 🎯 Accuracy Ranking (by correctness percentage): Accuracy Ranking (by correctness percentage): Azure OpenAI o4-mini: 10/10 correct (100%), 2.68s avg response, $5.47 per 1000 req xAI Grok-4 Fast Reasoning: 10/10 correct (100%), 2.83s avg response, $0.99 per 1000 req Azure OpenAI o3-mini: 10/10 correct (100%), 4.23s avg response, $5.53 per 1000 req xAI Grok-3 Mini: 10/10 correct (100%), 5.65s avg response, $1.47 per 1000 req OpenAI GPT-5 Nano: 10/10 correct (100%), 8.04s avg response, $0.29 per 1000 req OpenAI GPT-5 Mini: 10/10 correct (100%), 9.7s avg response, $1.37 per 1000 req OpenAI GPT-5 2025-08-07: 10/10 correct (100%), 13.98s avg response, $6.62 per 1000 req Azure OpenAI GPT-4o: 1/10 correct (10%), 1.26s avg response, $12.69 per 1000 req OpenAI GPT-4.1: 0/10 correct (0%), 3.58s avg response, $10.80 per 1000 req Azure OpenAI o4-mini: 10/10 correct (100%), 2.68s avg response, $5.47 per 1000 req xAI Grok-4 Fast Reasoning: 10/10 correct (100%), 2.83s avg response, $0.99 per 1000 req Azure OpenAI o3-mini: 10/10 correct (100%), 4.23s avg response, $5.53 per 1000 req xAI Grok-3 Mini: 10/10 correct (100%), 5.65s avg response, $1.47 per 1000 req OpenAI GPT-5 Nano: 10/10 correct (100%), 8.04s avg response, $0.29 per 1000 req OpenAI GPT-5 Mini: 10/10 correct (100%), 9.7s avg response, $1.37 per 1000 req OpenAI GPT-5 2025-08-07: 10/10 correct (100%), 13.98s avg response, $6.62 per 1000 req Azure OpenAI GPT-4o: 1/10 correct (10%), 1.26s avg response, $12.69 per 1000 req OpenAI GPT-4.1: 0/10 correct (0%), 3.58s avg response, $10.80 per 1000 req 💰 Cost Efficiency Ranking (by average cost per 1000 requests): Cost Efficiency Ranking (by average cost per 1000 requests): OpenAI GPT-5 Nano: $0.29 per 1000 req, 10/10 correct (100%), 8.04s avg response xAI Grok-4 Fast Reasoning: $0.99 per 1000 req, 10/10 correct (100%), 2.83s avg response OpenAI GPT-5 Mini: $1.37 per 1000 req, 10/10 correct (100%), 9.7s avg response xAI Grok-3 Mini: $1.47 per 1000 req, 10/10 correct (100%), 5.65s avg response Azure OpenAI o4-mini: $5.47 per 1000 req, 10/10 correct (100%), 2.68s avg response Azure OpenAI o3-mini: $5.53 per 1000 req, 10/10 correct (100%), 4.23s avg response OpenAI GPT-5 2025-08-07: $6.62 per 1000 req, 10/10 correct (100%), 13.98s avg response OpenAI GPT-4.1: $10.80 per 1000 req, 0/10 correct (0%), 3.58s avg response Azure OpenAI GPT-4o: $12.69 per 1000 req, 1/10 correct (10%), 1.26s avg response OpenAI GPT-5 Nano: $0.29 per 1000 req, 10/10 correct (100%), 8.04s avg response xAI Grok-4 Fast Reasoning: $0.99 per 1000 req, 10/10 correct (100%), 2.83s avg response OpenAI GPT-5 Mini: $1.37 per 1000 req, 10/10 correct (100%), 9.7s avg response xAI Grok-3 Mini: $1.47 per 1000 req, 10/10 correct (100%), 5.65s avg response Azure OpenAI o4-mini: $5.47 per 1000 req, 10/10 correct (100%), 2.68s avg response Azure OpenAI o3-mini: $5.53 per 1000 req, 10/10 correct (100%), 4.23s avg response OpenAI GPT-5 2025-08-07: $6.62 per 1000 req, 10/10 correct (100%), 13.98s avg response OpenAI GPT-4.1: $10.80 per 1000 req, 0/10 correct (0%), 3.58s avg response Azure OpenAI GPT-4o: $12.69 per 1000 req, 1/10 correct (10%), 1.26s avg response 🏆 Overall Leaderboard (Speed + Cost + Accuracy): Overall Leaderboard (Speed + Cost + Accuracy): 🥇 xAI Grok-4 Fast Reasoning: 93.1/100 overall├── Speed: 88/100 (2.83s avg)├── Cost: 94/100 ($0.99 per 1000 req)└── Accuracy: 100/100 (10/10 correct) 🥈 xAI Grok-3 Mini: 82.5/100 overall├── Speed: 65/100 (5.65s avg)├── Cost: 90/100 ($1.47 per 1000 req)└── Accuracy: 100/100 (10/10 correct) 🥉 Azure OpenAI o4-mini: 80.9/100 overall├── Speed: 89/100 (2.68s avg)├── Cost: 58/100 ($5.47 per 1000 req)└── Accuracy: 100/100 (10/10 correct) OpenAI GPT-5 Nano: 78.8/100 overall├── Speed: 47/100 (8.04s avg)├── Cost: 100/100 ($0.29 per 1000 req)└── Accuracy: 100/100 (10/10 correct) Azure OpenAI o3-mini: 76.1/100 overall├── Speed: 77/100 (4.23s avg)├── Cost: 58/100 ($5.53 per 1000 req)└── Accuracy: 100/100 (10/10 correct) OpenAI GPT-5 Mini: 70.5/100 overall├── Speed: 34/100 (9.7s avg)├── Cost: 91/100 ($1.37 per 1000 req)└── Accuracy: 100/100 (10/10 correct) Azure OpenAI GPT-4o: 42.5/100 overall├── Speed: 100/100 (1.26s avg)├── Cost: 0/100 ($12.69 per 1000 req)└── Accuracy: 10/100 (1/10 correct) OpenAI GPT-5 2025-08-07: 42.2/100 overall├── Speed: 0/100 (13.98s avg)├── Cost: 49/100 ($6.62 per 1000 req)└── Accuracy: 100/100 (10/10 correct) OpenAI GPT-4.1: 38.1/100 overall├── Speed: 82/100 (3.58s avg)├── Cost: 15/100 ($10.80 per 1000 req)└── Accuracy: 0/100 (0/10 correct) OpenAI GPT-5 Nano: 78.8/100 overall├── Speed: 47/100 (8.04s avg)├── Cost: 100/100 ($0.29 per 1000 req)└── Accuracy: 100/100 (10/10 correct) Azure OpenAI o3-mini: 76.1/100 overall├── Speed: 77/100 (4.23s avg)├── Cost: 58/100 ($5.53 per 1000 req)└── Accuracy: 100/100 (10/10 correct) OpenAI GPT-5 Mini: 70.5/100 overall├── Speed: 34/100 (9.7s avg)├── Cost: 91/100 ($1.37 per 1000 req)└── Accuracy: 100/100 (10/10 correct) Azure OpenAI GPT-4o: 42.5/100 overall├── Speed: 100/100 (1.26s avg)├── Cost: 0/100 ($12.69 per 1000 req)└── Accuracy: 10/100 (1/10 correct) OpenAI GPT-5 2025-08-07: 42.2/100 overall├── Speed: 0/100 (13.98s avg)├── Cost: 49/100 ($6.62 per 1000 req)└── Accuracy: 100/100 (10/10 correct) OpenAI GPT-4.1: 38.1/100 overall├── Speed: 82/100 (3.58s avg)├── Cost: 15/100 ($10.80 per 1000 req)└── Accuracy: 0/100 (0/10 correct) ⚡ Overall Statistics: Overall Statistics: 🏃♂️ Fastest individual response: 0.75 seconds (Azure OpenAI GPT-4o)🐌 Slowest individual response: 21.25 seconds (OpenAI GPT-5 2025-08-07)🎯 Most accurate model: OpenAI GPT-5 Nano (100%)❌ Least accurate model: OpenAI GPT-4.1 (0%)💰 Most expensive model: Azure OpenAI GPT-4o ($12.69 per 1000 req)💎 Most cost-effective model: OpenAI GPT-5 Nano ($0.29 per 1000 req)💵 Total cost for all tests: $0.452 And the winner is…. xAI Grok-4 Fast Reasoning (The Star of the Show) Accuracy: 10/10 (100%) Speed: 2.83s average (2.39s fastest, 4.59s slowest) Cost: $0.99 per 1000 requests Accuracy: 10/10 (100%) Accuracy: Speed: 2.83s average (2.39s fastest, 4.59s slowest) Speed: Cost: $0.99 per 1000 requests Cost: Cheap, accurate, and reasonably fast. Not the absolute fastest (that crown goes to GPT-4o), but considering GPT-4o answered correctly only 1 out of 10 times, I’ll take slightly slower for way more reliable. absolute fastest 1 out of 10 times Key Takeaways GPT-4o is fast but unreliable for this task. Great at sprinting, terrible at staying in its lane. Grok-4 Fast Reasoning hits the sweet spot: cheap, fast enough, and dead-on accurate. Azure’s o4-mini is also strong (100% accuracy, decent speed) but over 5x more expensive than Grok-4. GPT-5 Nano is ridiculously cheap, but you’ll wait 8+ seconds for every answer, which breaks our workflow. GPT-4o is fast but unreliable for this task. Great at sprinting, terrible at staying in its lane. Grok-4 Fast Reasoning hits the sweet spot: cheap, fast enough, and dead-on accurate. Azure’s o4-mini is also strong (100% accuracy, decent speed) but over 5x more expensive than Grok-4. GPT-5 Nano is ridiculously cheap, but you’ll wait 8+ seconds for every answer, which breaks our workflow. Where We Go From Here A year ago, GPT-4o was one of the most advanced and reliable options. We built big chunks of our product around it. But time moves fast in AI land. What was cutting-edge last summer looks shaky today. This little experiment with Grok-4 was eye-opening. Not only does it give us a better option for candidate evaluation, but it also makes me want to revisit other parts of our application where we blindly trusted GPT-4o. Moral of the story: don’t get too attached to your models. The landscape shifts, and if you don’t keep testing, you might wake up one day realizing your AI is confidently giving you the wrong answers… in record speed. So yes, GPT-4o, thank you for your service. But it looks like Grok-4 Fast Reasoning is taking your seat at the table.
