ChatGPT Goes Ghibli, Google Gets Smarter, and Microsoft Embeds Knowledge at Scale

Hello AI Enthusiasts! Welcome to the twelfth edition of "This Week in AI Engineering"! "This Week in AI Engineering" ChatGPT's 4o brings powerful native image generation that sparked the viral "Ghibli effect," and Tencent unveils the world's first ultra-large Hybrid-Transformer-Mamba MoE model, Google's Gemini 2.5 Pro achieves state-of-the-art performance with remarkable reasoning capabilities, Microsoft's KBLaM integrates knowledge bases with linear scaling efficiency.Plus, we'll cover Anthropic's new "think" tool dramatically improving Claude's complex reasoning abilities, alongside must-know tools to make developing AI agents and apps easier. ChatGPT 4o Image Generation & The Ghibli Art Style ChatGPT 4o Image Generation & The Ghibli Art Style OpenAI has released a new image generation system built directly into GPT-4o, representing a significant advancement beyond DALL-E by integrating image creation capabilities directly into the language model. This native multimodal approach delivers more precise, useful, and context-aware image generation. image generation system image generation system Technical Capabilities Technical Capabilities Text Rendering: Unparalleled accuracy in generating images with text elements, enabling effective visual communication Multi-turn Generation: Maintains visual consistency across iterations when refining images through conversation Enhanced Instruction Following: Handles 10-20 different objects in a single image with proper relationships (versus 5-8 in competing systems) In-context Learning: Analyzes uploaded reference images and incorporates their visual elements into new generations World Knowledge Integration: Leverages GPT-4o's knowledge base to create more intelligent, factually accurate images Text Rendering: Unparalleled accuracy in generating images with text elements, enabling effective visual communication Text Rendering Multi-turn Generation: Maintains visual consistency across iterations when refining images through conversation Multi-turn Generation Enhanced Instruction Following: Handles 10-20 different objects in a single image with proper relationships (versus 5-8 in competing systems) Enhanced Instruction Following In-context Learning: Analyzes uploaded reference images and incorporates their visual elements into new generations In-context Learning World Knowledge Integration: Leverages GPT-4o's knowledge base to create more intelligent, factually accurate images World Knowledge Integration The "Ghibli Effect" Trend The "Ghibli Effect" Trend The release has sparked a viral trend known as the "Ghibli effect," with users transforming photos into art inspired by Studio Ghibli's distinctive animation style. The trend exploded after GPT-4o's March 25th launch, with users sharing creations under hashtags like #GhibliStyle and #AIGhibli. Visual Characteristics: Soft watercolor backgrounds, expressive characters, and pastoral scenes reminiscent of films like Spirited Away and My Neighbor Totoro High-Profile Participation: OpenAI CEO Sam Altman changed his profile picture to a Ghibli-style portrait, while Elon Musk called it "the theme of the day" on X (formerly Twitter) Widespread Adoption: Users are transforming everything from selfies to iconic pop culture moments into Ghibli-inspired art Democratized Creativity: The tool allows anyone to create visually compelling artwork without requiring artistic skills Visual Characteristics: Soft watercolor backgrounds, expressive characters, and pastoral scenes reminiscent of films like Spirited Away and My Neighbor Totoro Visual Characteristics Spirited Away My Neighbor Totoro High-Profile Participation: OpenAI CEO Sam Altman changed his profile picture to a Ghibli-style portrait, while Elon Musk called it "the theme of the day" on X (formerly Twitter) High-Profile Participation Widespread Adoption: Users are transforming everything from selfies to iconic pop culture moments into Ghibli-inspired art Widespread Adoption Democratized Creativity: The tool allows anyone to create visually compelling artwork without requiring artistic skills Democratized Creativity Safety and Technical Implementation Safety and Technical Implementation Content Provenance: All generated images include C2PA metadata to identify them as AI-created Deliberative Alignment: Uses a reasoning LLM trained on human-written safety specifications Content Moderation: Blocks inappropriate content with safeguards against deepfakes and misuse Rendering Time: Due to enhanced detail capabilities, images take up to one minute to generate Content Provenance: All generated images include C2PA metadata to identify them as AI-created Content Provenance Deliberative Alignment: Uses a reasoning LLM trained on human-written safety specifications Deliberative Alignment Content Moderation: Blocks inappropriate content with safeguards against deepfakes and misuse Content Moderation Rendering Time: Due to enhanced detail capabilities, images take up to one minute to generate Rendering Time Availability Availability Current Access: Available to Plus, Pro, Team, and Free users as the default image generator in ChatGPT Coming Soon: Enterprise, Edu, and API access in the coming weeks DALL-E Access: Still available through a dedicated DALL-E GPT for those who prefer it Current Access: Available to Plus, Pro, Team, and Free users as the default image generator in ChatGPT Current Access Coming Soon: Enterprise, Edu, and API access in the coming weeks Coming Soon DALL-E Access: Still available through a dedicated DALL-E GPT for those who prefer it DALL-E Access Despite its advancements, OpenAI acknowledges limitations in areas like cropping, hallucinations, precise graphing, multilingual text rendering, and editing precision, which they plan to address through future model improvements. Google Gemini 2.5 Pro Achieves State-of-the-Art Performance Google Gemini 2.5 Pro Achieves State-of-the-Art Performance Google has introduced Gemini 2.5, starting with an experimental version of Gemini 2.5 Pro that showcases significantly improved reasoning abilities and benchmark performance. This "thinking model" leverages advanced reasoning techniques to analyze problems more thoroughly before responding. Gemini 2.5 Gemini 2.5 Benchmark Performance Benchmark Performance Humanity's Last Exam: Achieves 18.8% accuracy without tools, establishing state-of-the-art performance on this challenging benchmark Scientific Reasoning: 84.0% on GPQA Diamond single-attempt benchmark, outperforming OpenAI o3-mini (79.7%) and Claude 3.7 Sonnet (78.2%) Mathematical Reasoning: 86.7% on AIME 2025 and 92.0% on AIME 2024, surpassing all competitors on single attempts MMRC Long Context: 94.5% on 128K context window tests, demonstrating superior long-context comprehension Humanity's Last Exam: Achieves 18.8% accuracy without tools, establishing state-of-the-art performance on this challenging benchmark Humanity's Last Exam Scientific Reasoning: 84.0% on GPQA Diamond single-attempt benchmark, outperforming OpenAI o3-mini (79.7%) and Claude 3.7 Sonnet (78.2%) Scientific Reasoning Mathematical Reasoning: 86.7% on AIME 2025 and 92.0% on AIME 2024, surpassing all competitors on single attempts Mathematical Reasoning MMRC Long Context: 94.5% on 128K context window tests, demonstrating superior long-context comprehension MMRC Long Context Technical Capabilities Technical Capabilities Extended Context Window: Ships with 1 million token context (2 million coming soon) Multimodal Processing: Native handling of text, audio, images, video and code repositories Code Generation: 70.4% on LiveCodeBench v5 and 63.8% on SWE-Bench Verified with custom agent setup Global Performance: 89.8% on Global MMLU (Lite) tests showing strong multilingual capabilities Extended Context Window: Ships with 1 million token context (2 million coming soon) Extended Context Window Multimodal Processing: Native handling of text, audio, images, video and code repositories Multimodal Processing Code Generation: 70.4% on LiveCodeBench v5 and 63.8% on SWE-Bench Verified with custom agent setup Code Generation Global Performance: 89.8% on Global MMLU (Lite) tests showing strong multilingual capabilities Global Performance Availability Availability Current Access: Available now in Google AI Studio and in the Gemini app for Gemini Advanced users Coming Soon: Vertex AI integration in coming weeks with production pricing Leaderboard Position: Currently ranks #1 on LMArena by a significant margin Current Access: Available now in Google AI Studio and in the Gemini app for Gemini Advanced users Current Access Coming Soon: Vertex AI integration in coming weeks with production pricing Coming Soon Leaderboard Position: Currently ranks #1 on LMArena by a significant margin Leaderboard Position The model represents Google's strategic focus on building reasoning capabilities directly into their models rather than adding them as external components. Gemini 2.5 Pro can tackle complex tasks including visual reasoning (81.7% on MMMU) and image understanding (69.4% on Vibe-Eval), making it particularly well-suited for the development of capable, context-aware AI agents. Microsoft KBLaM: Efficient Knowledge Integration for LLMs with Linear Scaling Microsoft KBLaM: Efficient Knowledge Integration for LLMs with Linear Scaling Microsoft Research has introduced Knowledge Base-Augmented Language Model (KBLaM), a novel approach that efficiently integrates structured external knowledge into pre-trained language models without requiring separate retrieval systems or expensive retraining. Knowledge Base-Augmented Language Model (KBLaM) Knowledge Base-Augmented Language Model (KBLaM) Technical Architecture Technical Architecture Key-Value Vector Encoding: Transforms knowledge triples (entity, property, value) into continuous vector representations using pre-trained sentence encoders with lightweight adapters Rectangular Attention Mechanism: Implements specialized attention where language tokens attend to knowledge tokens but not vice versa, enabling efficient integration Linear Scaling: Memory usage and computation time scale linearly with knowledge base size rather than quadratically as with traditional in-context learning Key-Value Vector Encoding: Transforms knowledge triples (entity, property, value) into continuous vector representations using pre-trained sentence encoders with lightweight adapters Key-Value Vector Encoding Rectangular Attention Mechanism: Implements specialized attention where language tokens attend to knowledge tokens but not vice versa, enabling efficient integration Rectangular Attention Mechanism Linear Scaling: Memory usage and computation time scale linearly with knowledge base size rather than quadratically as with traditional in-context learning Linear Scaling Performance Metrics Performance Metrics Knowledge Capacity: Stores over 10,000 knowledge triples (equivalent to 200,000 text tokens) on a single GPU Time Efficiency: Maintains constant time-to-first-token across increasing knowledge base sizes, while RAG approaches show exponential slowdown Memory Usage: Exhibits linear memory growth as knowledge base expands, compared to quadratic growth in traditional approaches Base Model Extension: Achieves these improvements while extending a base model with only 8K token context length Knowledge Capacity: Stores over 10,000 knowledge triples (equivalent to 200,000 text tokens) on a single GPU Knowledge Capacity Time Efficiency: Maintains constant time-to-first-token across increasing knowledge base sizes, while RAG approaches show exponential slowdown Time Efficiency Memory Usage: Exhibits linear memory growth as knowledge base expands, compared to quadratic growth in traditional approaches Memory Usage Base Model Extension: Achieves these improvements while extending a base model with only 8K token context length Base Model Extension Core Advantages Core Advantages Dynamic Updates: Allows modifying individual knowledge triples without retraining or recomputing the entire knowledge base Improved Interpretability: Attention weights provide visibility into which knowledge is being utilized for each response Enhanced Reliability: System learns to refuse answering questions when necessary information is absent from its knowledge base Reduced Hallucinations: Structured knowledge representation helps prevent incorrect information generation Dynamic Updates: Allows modifying individual knowledge triples without retraining or recomputing the entire knowledge base Dynamic Updates Improved Interpretability: Attention weights provide visibility into which knowledge is being utilized for each response Improved Interpretability Enhanced Reliability: System learns to refuse answering questions when necessary information is absent from its knowledge base Enhanced Reliability Reduced Hallucinations: Structured knowledge representation helps prevent incorrect information generation Reduced Hallucinations Microsoft has released KBLaM's code and datasets to the research community and plans integration with the Hugging Face transformers library. Tencent Hunyuan-T1: First Ultra-Large Hybrid Transformer-Mamba MoE Model Tencent Hunyuan-T1: First Ultra-Large Hybrid Transformer-Mamba MoE Model Tencent has officially released Hunyuan-T1, a significant upgrade from their T1-preview version introduced in February. This reasoning-focused model is built on their TurboS fast-thinking base architecture, making it the world's first ultra-large-scale Hybrid-Transformer-Mamba MoE (Mixture of Experts) model. Hunyuan-T1 Hunyuan-T1 Technical Architecture Technical Architecture Hybrid Architecture: First-of-its-kind combination of Transformer and Mamba architectures in a MoE framework TurboS Base: Leverages the TurboS fast-thinking foundation with enhanced long-text capture capabilities Reinforcement Learning: 96.7% of compute resources focused on RL-based post-training to improve reasoning Curriculum Learning: Gradually increased data difficulty while expanding context length for improved efficiency Hybrid Architecture: First-of-its-kind combination of Transformer and Mamba architectures in a MoE framework Hybrid Architecture TurboS Base: Leverages the TurboS fast-thinking foundation with enhanced long-text capture capabilities TurboS Base Reinforcement Learning: 96.7% of compute resources focused on RL-based post-training to improve reasoning Reinforcement Learning Curriculum Learning: Gradually increased data difficulty while expanding context length for improved efficiency Curriculum Learning Performance Metrics Performance Metrics Knowledge Benchmarks: 87.2 on MMLU-PRO (second only to OpenAI's o1), 69.3 on GPQA-Diamond Reasoning: Exceptional 93.1 on DROP F1, outperforming GPT-4.5 (84.7) and comparable to DeepSeek R1 (92.2) Mathematics: 96.2 on MATH-500, nearly matching o1's 96.4 and approaching DeepSeek R1's 97.3 Chinese Language Tasks: 91.8 on CEval and 90.0 on CMMLU, tied with DeepSeek R1 Code Generation: 64.9 on LiveCodeBench, competitive with o1 (63.4) and DeepSeek R1 (65.9) Knowledge Benchmarks: 87.2 on MMLU-PRO (second only to OpenAI's o1), 69.3 on GPQA-Diamond Knowledge Benchmarks Reasoning: Exceptional 93.1 on DROP F1, outperforming GPT-4.5 (84.7) and comparable to DeepSeek R1 (92.2) Reasoning Mathematics: 96.2 on MATH-500, nearly matching o1's 96.4 and approaching DeepSeek R1's 97.3 Mathematics Chinese Language Tasks: 91.8 on CEval and 90.0 on CMMLU, tied with DeepSeek R1 Chinese Language Tasks Code Generation: 64.9 on LiveCodeBench, competitive with o1 (63.4) and DeepSeek R1 (65.9) Code Generation Core Advantages Core Advantages Processing Speed: 2x faster decoding than comparable models under equivalent deployment conditions Long-Text Processing: Mamba architecture optimizes processing of long sequences with reduced computational overhead Training Stability: Combined self-rewarding and reward model approach improved training stability by over 50% Alignment Performance: 91.9 score on ArenaHard, demonstrating strong instruction-following capabilities Processing Speed: 2x faster decoding than comparable models under equivalent deployment conditions Processing Speed Long-Text Processing: Mamba architecture optimizes processing of long sequences with reduced computational overhead Long-Text Processing Training Stability: Combined self-rewarding and reward model approach improved training stability by over 50% Training Stability Alignment Performance: 91.9 score on ArenaHard, demonstrating strong instruction-following capabilities Alignment Performance Hunyuan-T1 demonstrates particularly strong performance in DROP F1 (reading comprehension), Chinese language understanding, and mathematical reasoning tasks, establishing itself as a leading reasoning model that competes directly with OpenAI's o1 and DeepSeek R1. Anthropic's "Think" Tool Boosts Claude's Complex Tool Use Capabilities Anthropic's "Think" Tool Boosts Claude's Complex Tool Use Capabilities Anthropic has introduced a new "think" tool for Claude 3.7 that significantly enhances the model's performance on complex tasks involving sequential tool calls, policy adherence, and multi-step decision-making. "think" tool for Claude 3.7 "think" tool for Claude 3.7 Technical Implementation Technical Implementation Simple JSON Structure: Implemented as a standard tool with a straightforward schema that accepts a "thought" string parameter Self-Contained Process: Doesn't access external information or modify databases—just provides space for structured thinking Integration Method: Works alongside existing tools in standard tool-calling frameworks Implementation Overhead: Minimal code changes required to integrate into existing Claude deployments Simple JSON Structure: Implemented as a standard tool with a straightforward schema that accepts a "thought" string parameter Simple JSON Structure Self-Contained Process: Doesn't access external information or modify databases—just provides space for structured thinking Self-Contained Process Integration Method: Works alongside existing tools in standard tool-calling frameworks Integration Method Implementation Overhead: Minimal code changes required to integrate into existing Claude deployments Implementation Overhead Performance Metrics Performance Metrics Airline Domain: 0.584 pass^1 score with "Think + Prompt" versus 0.332 baseline (76% improvement) Consistent improvement across multiple trials: 0.444 at k=2, 0.384 at k=3, 0.356 at k=4, and 0.340 at k=5 Significantly outperforms both Extended Thinking (0.412 at k=1) and "Think" without prompt (0.404 at k=1) Retail Domain: 0.812 pass^1 score with "Think" tool alone versus 0.783 baseline Maintains advantage through k=5 (0.626 vs 0.583 baseline) Surpasses Extended Thinking (0.770 at k=1, dropping to 0.548 at k=5) SWE-Bench: 1.6% average improvement in software engineering tasks (statistically significant: p < .001, d = 1.47) Airline Domain: 0.584 pass^1 score with "Think + Prompt" versus 0.332 baseline (76% improvement) Consistent improvement across multiple trials: 0.444 at k=2, 0.384 at k=3, 0.356 at k=4, and 0.340 at k=5 Significantly outperforms both Extended Thinking (0.412 at k=1) and "Think" without prompt (0.404 at k=1) Airline Domain Consistent improvement across multiple trials: 0.444 at k=2, 0.384 at k=3, 0.356 at k=4, and 0.340 at k=5 Significantly outperforms both Extended Thinking (0.412 at k=1) and "Think" without prompt (0.404 at k=1) Consistent improvement across multiple trials: 0.444 at k=2, 0.384 at k=3, 0.356 at k=4, and 0.340 at k=5 Significantly outperforms both Extended Thinking (0.412 at k=1) and "Think" without prompt (0.404 at k=1) Retail Domain: 0.812 pass^1 score with "Think" tool alone versus 0.783 baseline Maintains advantage through k=5 (0.626 vs 0.583 baseline) Surpasses Extended Thinking (0.770 at k=1, dropping to 0.548 at k=5) Retail Domain Maintains advantage through k=5 (0.626 vs 0.583 baseline) Surpasses Extended Thinking (0.770 at k=1, dropping to 0.548 at k=5) Maintains advantage through k=5 (0.626 vs 0.583 baseline) Surpasses Extended Thinking (0.770 at k=1, dropping to 0.548 at k=5) SWE-Bench: 1.6% average improvement in software engineering tasks (statistically significant: p < .001, d = 1.47) SWE-Bench Key Differences from Extended Thinking Key Differences from Extended Thinking Extended Thinking: Occurs before response generation begins; plans an approach before taking action "Think" Tool: Used during response generation; processes new information after tool calls Use Case Separation: Extended thinking for upfront planning; "think" tool for sequential decision making Implementation: Extended thinking is a Claude feature; the "think" tool is developer-implemented Extended Thinking: Occurs before response generation begins; plans an approach before taking action Extended Thinking "Think" Tool: Used during response generation; processes new information after tool calls "Think" Tool Use Case Separation: Extended thinking for upfront planning; "think" tool for sequential decision making Use Case Separation Implementation: Extended thinking is a Claude feature; the "think" tool is developer-implemented Implementation Best Implementation Practices Best Implementation Practices System Prompt Integration: Place complex guidance in the system prompt rather than the tool description Targeted Use Cases: Most effective for tool output analysis, policy-heavy environments, and sequential decision making System Prompt Integration: Place complex guidance in the system prompt rather than the tool description System Prompt Integration Targeted Use Cases: Most effective for tool output analysis, policy-heavy environments, and sequential decision making Targeted Use Cases The "think" tool represents a low-risk, high-reward addition to Claude implementations that can dramatically improve performance on complex tasks with minimal implementation complexity, with graphics clearly showing performance advantages maintained across multiple trial runs when compared to baseline, extended thinking, and unprompted "think" approaches. Tools & Releases YOU Should Know About Tools & Releases YOU Should Know About Chat2DB is an AI-powered SQL client and database management tool. It uses AI to generate optimized SQL queries from natural language, enabling users to gain fast insights from their databases. It supports various databases, whether local or cloud-based, relational or non-relational, offering a centralized management interface. It enhances data security by processing queries locally and encrypting data. Chat2DB is designed for data analysts, developers, and database administrators who need an efficient, secure, and user-friendly way to interact with databases, analyze data, and manage schemas. Chat2DB is an AI-powered SQL client and database management tool. It uses AI to generate optimized SQL queries from natural language, enabling users to gain fast insights from their databases. It supports various databases, whether local or cloud-based, relational or non-relational, offering a centralized management interface. It enhances data security by processing queries locally and encrypting data. Chat2DB is designed for data analysts, developers, and database administrators who need an efficient, secure, and user-friendly way to interact with databases, analyze data, and manage schemas. Chat2DB Chat2DB Goast.ai is an AI-powered tool designed to automate bug fixing for software engineering teams. It integrates with platforms like Sentry and GitHub to analyze errors in real-time, pinpoint root causes, and generate code fixes. Goast creates pull requests for developers to review, saving time and improving productivity. It's ideal for engineering teams seeking to streamline their debugging process, reduce time spent on error resolution, and focus on building new features. Goast.ai is an AI-powered tool designed to automate bug fixing for software engineering teams. It integrates with platforms like Sentry and GitHub to analyze errors in real-time, pinpoint root causes, and generate code fixes. Goast creates pull requests for developers to review, saving time and improving productivity. It's ideal for engineering teams seeking to streamline their debugging process, reduce time spent on error resolution, and focus on building new features. Goast.ai Goast.ai Corgea is an AI-powered Static Application Security Testing (SAST) platform that helps modern development teams detect and fix code vulnerabilities. It employs AI to identify business logic and code flaws, reduce false positives, and generate code fixes automatically. Corgea uses natural language policies to tailor vulnerability detection and offers features like SLA management, blocking rules, and developer-friendly integrations. It supports multiple languages and aims to protect codebases from start to finish, ensuring data security and compliance. Corgea is designed for DevSecOps teams looking to streamline security and improve code quality. Corgea is an AI-powered Static Application Security Testing (SAST) platform that helps modern development teams detect and fix code vulnerabilities. It employs AI to identify business logic and code flaws, reduce false positives, and generate code fixes automatically. Corgea uses natural language policies to tailor vulnerability detection and offers features like SLA management, blocking rules, and developer-friendly integrations. It supports multiple languages and aims to protect codebases from start to finish, ensuring data security and compliance. Corgea is designed for DevSecOps teams looking to streamline security and improve code quality. Corgea Corgea Mage is an AI-powered platform designed for e-commerce businesses and marketers. It helps users create high-quality, AI-generated product photos without the need for expensive photoshoots. By simply providing product images or descriptions, Mage generates professional, styled visuals suitable for ads, websites, and social media. It's mainly for online store owners, designers, and marketers who want to enhance product visuals quickly and affordably. In AI terms, Mage leverages generative AI (likely diffusion models) to synthesize realistic, creative, and branded product images tailored to the user’s needs. Mage is an AI-powered platform designed for e-commerce businesses and marketers. It helps users create high-quality, AI-generated product photos without the need for expensive photoshoots. By simply providing product images or descriptions, Mage generates professional, styled visuals suitable for ads, websites, and social media. It's mainly for online store owners, designers, and marketers who want to enhance product visuals quickly and affordably. In AI terms, Mage leverages generative AI (likely diffusion models) to synthesize realistic, creative, and branded product images tailored to the user’s needs. Mage Mage And that wraps up this issue of "This Week in AI Engineering", brought to you by jam.dev— your flight recorder for AI apps! Non-deterministic AI issues are hard to repro, unless you have Jam! Instant replay the session, prompt + logs to debug ⚡️ This Week in AI Engineering jam.dev jam.dev Thank you for tuning in! Be sure to share this with your fellow AI enthusiasts and follow for more weekly updates!