I built a Gen AI chatbot to search through a vault of internal documents for a financial services firm to enable users with a tool to search, scan, and summarise documents faster. Creating up to 80% efficiency, I adopted Retrieval-Augmented Generation (RAG), embedded documents into the vault (group of directories), and adopted Titan after comparing LLMs like Claude, Titan, and Llama-3 to deliver context-aware, accurate answers for document search. For a three sprints work of comparing models, tweaking pipelines, trying to optimise performance, and tuning prompts—it worked pretty well on Titan model (Titan works the best amongst the listed models as it is designed for text model and my use case sits well under its capability scope). All the users I showcased and demoed LOVED it. It saved hours of searching and scanning on documents for FAQs and standard T&Cs for financial product documents (terminology heavy and full of footnotes etc.). While I am proud of the result of my work, I had one question in my mind: Am I 100% sure that this is the solution with the best price and performance – especially in the age of AI? More specifically, “How on earth am I going to compete with Amazon Q pricing?” The Problem With Success When you build something that works well to test and validate an idea, it feels like a great achievement —until you realize the bar is not just “working.” Although customers are overly cautious about using GenAI in highly regulated industries such as financial services, it still doesn’t seem enough for me to charge a higher price than enterprise-ready solutions like Amazon Q, Microsoft Copilot, and Google Gemini. Recognising that my solution with RAG that are increasing confidence on responses, eliminating hallucinations, and quality with many guardrails, I wasn’t satisfied that it is outperforming what's already out there. Enterprise-ready platforms are cheap, well-looking, and deeply integrated with ecosystems that businesses already live in. Amazon Q Business, for example, can index your S3 docs, handle access control, and costs next to nothing compared to the infrastructure I’d need to support high-volume GenAI queries on Titan or Claude. In my case, Amazon Q Business Enterprise charges $0.264 per hour for one unit (20K document or 200Mb extracted text) whereas I was calculating my cost per transaction per document as $0.23. I was proud of the chatbot I built. But in terms of scaling and cost-efficiency? I was suddenly in a very tight spot. Hard Decision: Build vs. Buy in the GenAI Era This is not a new dilemma, but with GenAI, things have become more complex. Here’s a summary of what I learned, comparing during the development of in-house chatbot which is similar to AWS Q Business (to some extend): Feature In-house GenAI Chatbot AWS Q Business / Off-The-Shelf ChatBot Control You have full customization (RAG, LLM tuning,prompt engineering) Limited to platform capabilities – but fairly flexible Data Privacy You can enforce custom encryption, anonymization, or add new layers Predefined policies & integrations – needs configuration Cost Significantly higher (especially if you use Bedrock type of environments) Cost-effective for enterprise level solutions Setup Time Weeks of model selection, actual development, QA, iterations Hours, sometimes minutes (can have account support) LLM Model Options Choose your model (Claude, Llama-3, Titan, etc.) Locked into platform choice (Amazon = Titan/Q, Microsoft = GPT-4) Maintenance you have to manage yourself - scaling, uptime, latency tuning Handled by provider Feature In-house GenAI Chatbot AWS Q Business / Off-The-Shelf ChatBot Control You have full customization (RAG, LLM tuning,prompt engineering) Limited to platform capabilities – but fairly flexible Data Privacy You can enforce custom encryption, anonymization, or add new layers Predefined policies & integrations – needs configuration Cost Significantly higher (especially if you use Bedrock type of environments) Cost-effective for enterprise level solutions Setup Time Weeks of model selection, actual development, QA, iterations Hours, sometimes minutes (can have account support) LLM Model Options Choose your model (Claude, Llama-3, Titan, etc.) Locked into platform choice (Amazon = Titan/Q, Microsoft = GPT-4) Maintenance you have to manage yourself - scaling, uptime, latency tuning Handled by provider Feature In-house GenAI Chatbot AWS Q Business / Off-The-Shelf ChatBot Feature Feature In-house GenAI Chatbot In-house GenAI Chatbot AWS Q Business / Off-The-Shelf ChatBot AWS Q Business / Off-The-Shelf ChatBot Control You have full customization (RAG, LLM tuning,prompt engineering) Limited to platform capabilities – but fairly flexible Control Control Control You have full customization (RAG, LLM tuning,prompt engineering) You have full customization (RAG, LLM tuning,prompt engineering) Limited to platform capabilities – but fairly flexible Limited to platform capabilities – but fairly flexible Data Privacy You can enforce custom encryption, anonymization, or add new layers Predefined policies & integrations – needs configuration Data Privacy Data Privacy Data Privacy You can enforce custom encryption, anonymization, or add new layers You can enforce custom encryption, anonymization, or add new layers Predefined policies & integrations – needs configuration Predefined policies & integrations – needs configuration Cost Significantly higher (especially if you use Bedrock type of environments) Cost-effective for enterprise level solutions Cost Cost Cost Significantly higher (especially if you use Bedrock type of environments) Significantly higher (especially if you use Bedrock type of environments) Cost-effective for enterprise level solutions Cost-effective for enterprise level solutions Setup Time Weeks of model selection, actual development, QA, iterations Hours, sometimes minutes (can have account support) Setup Time Setup Time Setup Time Weeks of model selection, actual development, QA, iterations Weeks of model selection, actual development, QA, iterations Hours, sometimes minutes (can have account support) Hours, sometimes minutes (can have account support) LLM Model Options Choose your model (Claude, Llama-3, Titan, etc.) Locked into platform choice (Amazon = Titan/Q, Microsoft = GPT-4) LLM Model Options LLM Model Options LLM Model Options Choose your model (Claude, Llama-3, Titan, etc.) Choose your model (Claude, Llama-3, Titan, etc.) Locked into platform choice (Amazon = Titan/Q, Microsoft = GPT-4) Locked into platform choice (Amazon = Titan/Q, Microsoft = GPT-4) Maintenance you have to manage yourself - scaling, uptime, latency tuning Handled by provider Maintenance Maintenance Maintenance you have to manage yourself - scaling, uptime, latency tuning you have to manage yourself - scaling, uptime, latency tuning Handled by provider Handled by provider So... Does it worth building in-house? Yes. And no. If my clients want to have 100% control and enforce me to apply all known regulations (even if they are not fully applicable and necessary) over document processing, search, and model explainability, building in-house made sense. Because double checking regulatory requirements and bespoke query structures can create value for the business; they could become willing to pay more for the added value. However, if I had to scale quickly? Or if I needed something good enough (85-90% of use cases) without investing a lot of cash upfront, Amazon Q Business suddenly becomes very attractive. You Should Build If: You need stricter control over data and model behaviour You’re required to work within unique workflows such as complex financial documents, multi-source RAG etc. You want to add additional AI features within core product Your organisation is ready to invest upfront for development, maintenance and infra cost You need stricter control over data and model behaviour You’re required to work within unique workflows such as complex financial documents, multi-source RAG etc. You want to add additional AI features within core product Your organisation is ready to invest upfront for development, maintenance and infra cost You Should Buy If: You want quick wins for small scopes and showcases Your use case is relatively standard (e.g. document Q&A, policy navigation etc.) You’re cost-sensitive You’re already within the vendor’s ecosystem (e.g. AWS, Microsoft, Google) You want quick wins for small scopes and showcases Your use case is relatively standard (e.g. document Q&A, policy navigation etc.) You’re cost-sensitive You’re already within the vendor’s ecosystem (e.g. AWS, Microsoft, Google) Closing Thoughts GenAI chatbot creation can be rewarding and create quick wins for the business. It is also relatively easier to start experimenting with GenAI tooling within operations, upskilling the team. But the market moves super fast. Tech giants like AWS are lowering the barrier even further for tools like AWS Q Business. So, it always worth asking, “Should we build or buy?” Cause we’re not just competing with code in this era. We’re competing with commoditized GenAI tools created by giants with billion-dollar infrastructure, talent, and polished UI/UX as well as tooling. And that’s a different kind of battle.
