How much does AI consulting cost?

BlokAI's engagements range from $5,000 to $75,000 depending on scope. Simple automations and LLM integrations start around $5K–$15K. Custom AI agent systems and full data infrastructure projects run $30K–$75K. We scope every project before committing to a price.

What does BlokAI build?

BlokAI builds AI agents, LLM applications, RAG systems, automation pipelines, and data infrastructure for startups and SMBs. We take projects from architecture to production — not demos, not prototypes.

Is BlokAI based in Japan?

Yes. BlokAI is a solo studio run by Kent Tran, based in Tokyo, Japan. We work with clients globally in English.

What is RAG and why does my business need it?

RAG (Retrieval-Augmented Generation) lets AI systems search your own documents, databases, or knowledge bases before answering — instead of relying only on training data. It powers AI customer support, internal search tools, and sales enablement systems that are accurate and up-to-date.

The Stack Behind a Production LLM Application in 2023

"PaLM 2 is our next-generation language model, available today through 25 Google products and the PaLM API — a major step in making AI more accessible to developers." — Google I/O Blog, May 10, 2023

The demos are easy. Production is where it gets interesting.

I see a lot of content about what LLMs can do — the impressive outputs, the capability benchmarks, the creative use cases. What I see very little of is an honest account of what it actually takes to run an LLM-powered application in production at a business scale. This post is that account.

This is the stack we're building with at BlokAI right now, in May 2023, for real client projects. It'll evolve — the ecosystem is moving fast — but this is the current reality.

Layer 1: The Model API

At the foundation, you're almost certainly calling an API to a large language model. The main options right now:

OpenAI API (GPT-4, GPT-3.5-turbo) — strongest capability, well-documented, but costs add up at scale and you're sending data to a third party
Anthropic's Claude — strong alternative, good for long-context tasks, different strengths and failure modes than OpenAI
Azure OpenAI Service — same OpenAI models but hosted on Microsoft infrastructure, better for enterprise compliance requirements, especially relevant for businesses in regulated industries

The choice depends on your compliance requirements, your cost model, and which model actually performs better on your specific task. We test multiple models against the actual task before committing to one.

Layer 2: The Orchestration Framework

You don't call the model API directly in production code. You use an orchestration layer that handles the complex logic around:

Prompt construction (assembling system prompts, retrieved context, user input)
Chain-of-thought structuring (breaking complex tasks into steps)
Memory management (what context to include in each call)
Tool use / function calling (letting the model trigger actions or retrieve data)

LangChain is the dominant framework right now. It's powerful and has a huge ecosystem, but it's also rapidly changing and the abstractions can get leaky. We've found it works well for prototyping and for standard patterns, but complex custom logic sometimes fights the framework.

For simpler use cases, we occasionally write the orchestration directly. Less magic, more control.

Layer 3: The Knowledge Layer (RAG)

This is where most of the business value lives, and also where most projects get into trouble.

Retrieval-Augmented Generation (RAG) is the technique of taking your business documents, chunking them, embedding them into vectors, storing them in a vector database, and then at query time, retrieving the relevant chunks and including them in the prompt context before asking the model to answer.

The components:

Embedding model: Takes text and converts it to a numerical vector representation. OpenAI's text-embedding-ada-002 is the current default for most projects. Fast, cheap, good quality. Vector database: Stores the embeddings and enables semantic similarity search. Options we're using: Pinecone (managed, reliable, costs money), Chroma (local/self-hosted, good for development), Weaviate (self-hosted, feature-rich). The choice depends on data volume, infrastructure preferences, and budget. Chunking strategy: How you split documents before embedding them has a significant impact on retrieval quality. Fixed-size chunking with overlap is the default. Semantic chunking (splitting at sentence or paragraph boundaries) often performs better but requires more upfront work. This is an area worth investing time in.

Layer 4: The Application Layer

The API calls, the user interface, the business logic. This is standard software development. We typically use:

Python for backend AI services (the ecosystem is unmatched)
FastAPI for exposing AI capabilities as APIs
Next.js or similar for frontend if there's a user-facing interface
PostgreSQL for structured application data

Nothing exotic here. The AI stack sits on top of conventional web application infrastructure.

Layer 5: Observability and Evaluation

This layer is the most neglected in early projects and the most important in production.

You need to know:

What prompts are being sent to the model?
What responses are coming back?
Where is the system failing?
Are outputs improving or degrading over time?
What is this costing, per user, per query, per day?

LangSmith (from the LangChain team) is emerging as the observability layer for LLM apps — traces, evaluations, and debugging. Still early but worth using. Alternatives include rolling your own logging (more control, more work). Evaluation is the hardest part. Unlike traditional software, "did the AI give a good response?" is often a qualitative judgment. You need human review, reference datasets, and ideally automated evaluation using a second model as a judge. This is where serious production systems invest significant effort.

Layer 6: Guardrails and Safety

For any customer-facing deployment, you need:

Input filtering — detect and reject off-topic, harmful, or adversarial inputs
Output validation — check that responses meet your quality and safety criteria before returning them
Fallback logic — what happens when the AI can't or shouldn't answer?
Rate limiting — protect against abuse and runaway costs

These aren't optional. Every production system needs them.

The Cost Reality

A rough breakdown for a moderately-used internal tool (100 active users, ~50 queries/day each):

GPT-4 API: ~$200-800/month depending on query complexity
Vector database (Pinecone): ~$70/month for a starter tier
Hosting (assuming cloud): ~$50-200/month depending on complexity
Monitoring/logging: variable

Total infrastructure: roughly $300-1,100/month at this usage scale. For many business applications, this is easily justified by the value delivered. But it's not free, and it scales with usage.

What This Means For Your Build

The stack is more complex than most people expect when they start, and simpler than it looks once you've built it a few times. The patterns are becoming standardised. The tooling is maturing quickly.

What matters most: don't treat this as a technology project. Treat it as a product with a technology layer. The engineering is solvable. The product decisions — what does the user need, how do they interact with it, what does success look like — are where the real work is.

Want to implement AI in your business? Book a 30-minute architecture call at blok-ai.com.