Priyanka Jindal

AI Systems Pricing and Margin Problems

Priyanka Jindal — Sat, 27 Sep 2025 00:12:53 GMT

Over the last decade, two key tenets for cloud-native services and SaaS pricing were: 1/is it easy to understand? and 2/is it cost-following? Being easy-to-understand and cost-following was not a challenge for SaaS applications and even a large majority of infrastructure services. Why? Because infra margins hovered between 80-95%, so contribution/net margins were driven by willingness to pay.

However, adhering to both these tenets for AI Systems Pricing has been incredibly tricky in certain domains, one can say even an unsolved problem yet. I will use the example of Coding Agents (my favorite subject 😉) to walk you through the logic. At the heart of the issue are two fundamental pricing questions:

💲 What is the at-scale marginal cost?
When pricing, you need to know your at-scale marginal infra cost. But with AI Systems, when you change the underlying model in-use, the infrastructure bill of material changes and hence the marginal cost changes. For example: a new reasoning model which has the capability to execute longer running tasks will take more tokens for the same prompt, of-course deliver higher value to the user (through better accuracy or completion), but also cost you more than before.

⚙️ What are the architecture design choices that cause outlier cost scaling scenarios (edge cases, failure scenarios)?
With managed cloud-services or SaaS, these scenarios were predictable; Carefully study the usage patterns and corresponding traffic shape, and then do a bit of sensitivity analysis on the key cost drivers.

With AI Systems, not so predictable. What if the agent goes off rails, or goes into a loop costing you expensive tokens and tool calls, while not delivering on the user prompt.

🤯 What can you do?
Change your price with each model release? Or have a different price for each model? Or charge based on #tokens or #tool calls which makes it difficult to understand? Or charge based on usage like each prompt which makes it not really cost following.

💡 So where is it headed?
One of the yet unexplored levers lies in the Coding Systems architecture choice (beyond of course the model costs themselves coming down over time as competition heats up). All Coding Systems implement a Single Agent architecture today, which is really the best practice for most domains unless sub-tasks can be parallelized. Imagine being able to switch to a multi-agent architecture instead where sub-agents can use a mix of smaller, faster models based on the task complexity. This gives you more control over costs, rather than using the same, expensive reasoning model through the task completion.

I will dive into why Single Agent architecture is used by Coding Systems, and how Multi-Agent move can bring down costs over time, once their technical limitations of caching and routing could be overcome. Stay tuned and follow along if you are interested.

Understanding AI Coding Agents architecture

Priyanka Jindal — Sat, 27 Sep 2025 00:09:53 GMT

To build new AI-powered experiences that are 10x more efficient than developers alone in certain workflows, you need to deeply understand the architecture and toolsets available with AI IDEs and Remote Coding Agents. Why? Because AI IDEs and Agents are rapidly becoming the center of gravity around which all SDLC use-cases need to evolve.

To develop the right intuition, understanding the Coding Agents architecture becomes non-negotiable. The simplest way to understand how the state of the art works is as follows:
⚙️ A data model captures the semantic meaning of the codebase.
🖥️ A UI lets users input a prompt. The prompt is translated into a query against the semantic model to find the “most relevant context”.
🔍 A planning agent takes the relevant context and derives a list of actions to complete.
🧰 A set of tools are triggered to complete the actions. Tools report their status back to the planner, and iterate until finished. The tools include grep, web search, read file, generate code, edit code, terminal commands etc.

Once you understand the architecture, the next inevitable question is the “So What?”. What can you do with it?

1. Differentiate the semantic data model: This is why indexing the whole codebase is superior than just reading the open files.

2. Build UX controls to clarify context: Make it easy for the user to provide context (examples include Rule files, attaching files/folders, retaining workspace memories and commit history).

3. Build UX controls to validate planning: Be transparent and show the user the task list or the queried context. Get user input to correct it.

4. Wire external tools for richer context: MCP is the emerging vehicle for this. Adding MCP Servers for Documentation, Project management (e.g., Linear, JIRA, Github Issues), Code scans and reviews (Github scans, Copilot reviews, CodeRabbit reviews), Database schema (e.g., MongoDB), Logs and metrics (e.g., Datadog), Testing tools (e.g., Playwright) enrich the Agent’s context and aid with better planning and execution.

What’s interesting to note is that some UX levers are entirely controlled by first-party AI IDEs and Remote Coding Agents, while others are available to third-party (3P) developers integrating with them. There are examples of native integrations & partnerships between the two parties, but when such partnerships are not feasible 3P tools have two interesting avenues: MCP Servers and as Background Agents across other key surfaces like PR, CI and project management tools (e.g., a review agent in PR, debugging agent in CI, triage agent in Ticket tracker).

So if you are a builder, its a fascinating time to reimagine workflows across the entire SDLC.

Building AI Agents: Designing the user experience

Priyanka Jindal — Mon, 23 Jun 2025 22:02:24 GMT

To design an agent, you (the developer) have three levers to consider:

What should the model control?
What should the developer (you) control?
What should the end-user control?

Model: How can you shape the model’s personality, reasoning and acting capability?

Tools: System instructions, Reinforcement learning
Key metric: Consistency ( e.g., clarifies assumptions?, selects the right set of tools?)
Type of evals: Human and LLM Graders (since the metric is subjective)
Key problems to solve:
- Which model should I select?
- How do I curate datasets with golden user prompts and reference answers for constructing baseline evals?
- How do I tune system instructions to guide the model reasoning ? How do I analyze and trace model reasoning?
- Can I improve model consistency by modifying its weights? Does my use-case rely on datasets that the general purpose models are not trained on?
- How do I collect user traces or generate synthetic data to apply reinforcement learning to tune the model weights?

Developer: What agent architecture and set of tools will best process the user intent, and enable the model to complete actions?

Tools: Agent architecture and tools design including indexed search, MCP, native tools, memory
Key metric: Accuracy
Type of evals: objective scores (precision@K, recall@K, passed unit test) and LLM graders (e.g., is the PR summary accurate?)
Key problems to solve:
- Which agent architecture is best suited for the use-case? How should I break down the workflow and define sub-agents with its own set of tools and system instructions to insert most relevant information at each point?
- How do I iterate on available tools and tools definition to improve accuracy?
- How do I define metrics and scoring rubrics or objective functions to automate evals?

User experience: What modalities are available to the user? Is the end-user experience responsive or proactive?

Tools: Inputs via speaking (voice), writing (inline, chat, keywords, prompt or rule files), selecting or loading data (files, MCP tool, image upload)
Key metric: Simplicity
Key problems to solve:
- Which inputs should the user be able control?
- Can I deliver a proactive agent with guardrails given my use-case? For example: a code reviewer agent that automatically gets triggered upon each PR is a proactive agent.
- How to solve a MVP use-case completely while keeping the number of input modalities, and the number of user actions required minimal?
- Are the user controls intuitive and visible, or hidden under hard to find settings?

Does this summary match your own experience or learnings while designing AI agents?

Building AI Agents - How to build a Business Case

Priyanka Jindal — Thu, 19 Jun 2025 06:36:08 GMT

How do you define an AI Agent? AI Agents, even though software, are not only about replacing existing software tools to do a job. They are about re-designing the roles and responsibilities of a job itself - this involves re-designing both the software and knowledge work done by a human with the software. This definition of AI Agents is key to evaluating it’s opportunity. Lets break it down further.

Software

Majority AI Agents in news today, however, are integrating with rather than displacing existing software tools. These integrations serve the purpose of fetching data from the source of record and then applying intelligence on top, rather than replacing the source for now. Examples of source of records would be Salesforce for GTM, ServiceNow for customer support, Datadog for observability, Terraform for IaC and so on. Do we expect this to work in the long term? Will customers continue paying for the system of record, and separately for the AI Agent on top?

Knowledge work

While cloud-native software targeted IT budgets ONLY, AI Agents target a combination of IT budgets and knowledge worker & professional services budgets. For reference, knowledge worker’s salaries and professional services budgets combined are at-least a magnitude higher relative to IT software budgets (i.e., in USD trillions). Let’s say, you’re building an AI agent for code security remediation. In this case, the market opportunity isn’t limited to the subscription cost of existing AppSec tools. It also includes the labor and hidden costs—namely, the engineering headcount and developer hours required to triage, fix, and merge the issues surfaced by those tools. An AI agent that automates remediation addresses both the tool cost and operational overhead.

What lessons can we draw from building and selling cloud-native software?

One of the biggest technology waves before AI was cloud computing, which disrupted how software was built and delivered to business users. The cloud computing wave delivered software innovations across each layer of the stack - infrastructure (e.g., AWS, GCP), domain specific platforms across network, security, data (e.g., Cloudflare, Zscaler, Datadog), and of course applications (e.g., Github, Salesforce, ServiceNow).

There is also a lot to learn by comparing and contrasting AI Agents with cloud-native software, including looking at cloud-native software’s entitlement, GTM, and cost drivers.

Entitlement: Targeted IT budgets
GTM and pricing: steal on-prem share, grow volume, and grow the pie itself. Per seat subscription (if used by a human) or usage-based metering (if used programmatically by other machines).
Defining features and cost drivers: Managed software delivered via browser app, desktop app, or CLI; a cloud-native operational model (automatic remote updates, 24x7 support, availability SLAs), and multiple deployment modes for varying networking, security, and governance environments.

So how does it compare to the opportunity size of an AI Agent?

Entitlement: The easiest way to think about it is as a combination of the pre-AI tool cost and the labor cost of the person using the tool. How much the AI Agent chews into existing tools and labor costs will lie on a spectrum and depends on the use-case. For example:
- an AI agent for market research doesn’t yet replace the existing tools where you store information (Google drive), but replaces the labor $ spent on hiring or outsourcing research tasks.
- an AI agent for customer support replaces the online support tools, but also reduces the number of human support agents needed to achieve the same unit of output as before.
- an AI agent for technical documentation replaces both the tools and the technical writer hours needed to use the tool.
Pricing: This is a hotly debated topic within Silicon Valley circles. The breakout use-cases like Coding Agents continue to use cloud-native pricing model (a combination of per seat subscription combined with usage-based metering). Given the steep infrastructure marginal cost of running agents, anything but usage-based pricing will bleed companies red. So I am not yet sold on outcomes based pricing models. This will of-course evolve in the coming years as inference compute gets cheap and new agentic usage patterns breakout.
Defining features and cost drivers: The cloud-native delivery and operational model remains the same for AI Agents. Building AI Agents is much like designing any other complex distributed system. AI Agents need an interface (desktop app, browser app, or a device) and scale well with a cloud-native operational model. Sure, the software infrastructure itself is evolving (with new components being model APIs, frameworks for evals and inference serving, observability and so on), but they still fit into the distributed system engineering discipline.

What are some tension points that you need to be opinionated about?

If your AI Agent relies on data that exists in other software today, which data sources can you displace versus which ones you must integrate with? The ones you integrate with, can you displace them in the future if you are successful with the MVP?
If you are designing an AI Agent to be used by a persona, are you going to project shrinkage in the # personas worldwide in the coming years? How will the role and responsibilities of the persona going to change with AI? Is it likely to grow? Is it likely to merge into other roles? Is the role going to disappear?

To summarize, you will need to work back from what the future of the job you are designing for will look like. And then you can evaluate it’s entitlement, pricing opportunities, and cost drivers by using existing benchmarks from cloud-native software today.

Remote Coding Agents to SWE Orchestrator?

Priyanka Jindal — Tue, 27 May 2025 15:59:47 GMT

Coding Eval Infra (yesterday) -> Remote Coding Agents (today) -> SWE Orchestrator (tomorrow) ?

If you have run Coding Evals and tried any of the Remote Coding Agents launched in the last month (Augment Code, Google Jules, OpenAI Codex), you would have noticed that the product is what you would get if you were to take Coding Evals infra and add a UI to it to run parallel tasks.

This makes me wonder whether the idea started with taking the infra these teams had already built internally (for running coding evals at scale) and productionizing it to see whether a sticky developer behavior outside IDE comes out of it? AWS came out of such an approach.

In the case of Jules and Codex, I have found that initiating a task/agent from the web UI without being able to easily provide additional context, like you can in an IDE with multiple context pinning options, makes for a sub-par experience.
Minus the ability to add rich context, you are restricted to initiating trivial tasks or lumber through manually writing richer prompts. There is a lot of value in running multiple tasks remotely/in the cloud, but a web UI for triggering tasks leaves much to desire.

A more compelling CX for Remote Agents is then to initiate tasks from the IDE, which I believe Augment is promising (even though I have yet to try it since I am on the waitlist). It may also be the direction where Codex may be headed (triggering codex agents from Windsurf?).

An interesting use-case for the web experience though is to overtake the SWE planning stage (could this replace ticketing systems?). Imagine this SWE workflow: 1/ there is Planning (with meeting notes in Granola, Otter.ai, async communication in Slack and Loom, and writing in Notion and Google Docs). Out of this SDLC step comes your goals and breakdown of tasks ready for implementation. Where does it go? 2/In ticketing systems like Linear and JIRA - used for tracking and risk management. And then comes 3/Remote SWE Agents and IDEs - this is where solo builders can maintain their backlog and decide which tasks are suited for SWE Agent vs which is better suited for AI pair programming in IDE.

In this world, why can’t I skip 2 and directly go from 1 to 3? There are so many opportunities across 1 and 3 to simplify. Lesser tools, better CX?

Pitfalls of AI positioning

Priyanka Jindal — Tue, 20 May 2025 20:13:46 GMT

An early learning I had when ‘selling’ my AI-native product was that calling your product an “AI " is usually not a good positioning. Turns out, positioning yourself as AI Counsel or AI Engineer is much less inviting than calling yourself Harvey or Copilot.

Why?

1. It alienates a subset of your target buyers or users. Early majority who may not be ready yet, but will be critical once you establish proof points.

2. Adding an AI Prefix doesn’t automatically communicate YOUR vision of how the workplace would evolve and look different in 10 years. You need to tell the story.

3. Ideally, you are building towards a future which is not only automating repetitive, mundane tasks and only selling productivity boost, but also innovating on experiences that were not possible before without the AI foundation tech stack (LLMs, tool use, agent to agent communication). If you are working on new use-cases with novel CX, using the “AI Persona” leads your audience to think about your TAM incorrectly.

4. Further, what the persona does TODAY, in their existing workflows itself, may vary from company to company. For example in software engineering, roles across Tech Leads, Architects, SDEs, TPMs, Deployment Engineers, GTM PMs have lots of gray boundaries and overlaps. With AI-native SWE and collaboration tools, the responsibilities of each of these persona are sure to evolve. Your job is to define what that future looks like. So ideally, you are working back from the outcomes SWE teams want, NOT from what these role do today and what their boundaries are.

5. Finally, there may be logical reasons for choosing AI + Persona positioning with buyers and decision makers to communicate your business/pricing model. If so, reserving that positioning for the business/pricing model discussion ONLY works better.

Divinely Discontent

Priyanka Jindal — Tue, 08 Apr 2025 01:01:16 GMT

Have you experienced a divinely discontent moment recently with AI?

This last weekend, I tried the Agent mode from Augment to create a fully functional web app prototype. It worked beautifully. It recognized the language and frameworks I already had in my repo. Setup the new project, created both frontend UI and backend integration, configured deployment with my choice of service, and self-corrected when it encountered runtime errors. No debugging required. Brilliant!

And then, I wanted to make tweaks to the app it had produced. Turns out that Agent mode is too slow for these tweaks and I preferred using Chat mode to assist with edits (since Agent mode is still in beta, my guess is that infra optimizations to improve its latency will follow). I switched to Chat mode, but Memories with additional context was in the Agent mode window, not Chat mode. Ugh! For a split second, I reasoned, “why can’t it automatically switch between modes depending on my intent? Another version of model picker story”.

A classic “divinely discontent” moment!

In his 2017 shareholder letter, Jeff Bezos recognized, “One thing I love about customers is that they are divinely discontent. Their expectations are never static – they go up. It's human nature.”

I see it everywhere these days with AI progress. It’s easy to blame AI when it doesn't work or doesn’t meet your expectations. But it's important to recognize when your expectations have evolved.

Self-host models or use APIs?

Priyanka Jindal — Sat, 05 Apr 2025 00:17:44 GMT

If I am an enterprise CTO considering using DeepSeek models, I would imagine the questions making rounds would be: what are the considerations with deploying an open-source model like DeepSeek for my AI projects? How do these considerations change, if any, if the inference-time flops are coming down as significantly?

1/ When kickstarting new AI initiatives, the truth is that the technical bar for calling OpenAI/Claude APIs is much lower for lean-prototyping teams compared to getting entangled in the work of self-hosting models on EKS/GKE. I am excited to see that the DeepSeek launch is going to give momentum to startups/builders (like Replicate, Modal, Baseten) who are simplifying the process of deploying open-source models and differentiating themselves with simple developer experiences.

2/If you start comparing costs of self-hosting a model (e.g., request throughput/GPU) with calling an inference API ($/token) in production environments, the reality is that running models requires engineering systems and operational excellence that most customers, even the largest enterprises won’t and can’t invest in building in-house. Given the status quo, it’s not as simple as spinning up a VM. There is a fair bit of muck involved in setting up and measuring inference-server and GPU-level optimizations, and with infra deployments (auto-scaling, load balancing, reducing latency etc). This reflects in the infra margins for these specialized GPU-hosting services. It’s not clear whether incumbent or specialized Inference-as-a-service providers can establish differentiation here or it will become a race to the bottom with infra margins.

3/If we think of models as a net new primitive just like compute, storage, networking, one implication of DeepSeek is that the “Novelty/R&D investment” Margins as priced in $/token by closed-model providers may eventually need to compete with the Infrastructure Margins of a managed or self-hosted open-source model. This may accrue more value to domain-specific models, not just models, AI solutions that enterprises can plug in their data to get value. It will be interesting to see how this development influences buy vs build decision that each large organization (~1000+ employees) faces.

What has not changed?

Priyanka Jindal — Sat, 05 Apr 2025 00:12:52 GMT

As a builder, a key lesson I have learnt is to always ask the question “what has NOT changed?” rather than only focus on “what has changed?” when designing products.

If you are building an AI app/service/platform that wants to target the enterprise segment, what hasn’t changed is the software infrastructure investment you need to be prepared to make get your product enterprise infra ready. These roadmap investments often have a long lead time. More importantly, you will have to walk through several one-way door CX and architecture decisions that may be the key to your differentiation and compounding it over time.

You will need to build a CX for the customer’s operator/admin that leads to faster onboarding of your product into their org. Providing deployment options (not just a SaaS offering), setting up secure connectivity and onboarding data sources, allowing for role based access for inputs and outputs to your AI system, configuring privacy filters, compliance logging and more - remain as critical as they have been in the cloud-migration decade, and now more so.

My AI-native dev environment

Priyanka Jindal — Sat, 05 Apr 2025 00:10:05 GMT

Lately, I have been in the groove with AI-native engineering tools and iterating on my dev environment. My main motivation is to do hands-on research for a startup concept and arrive at a compelling CX for it, as I iterate on it while going through customer interviews.
For full transparency, I coded heavily in C++ for several years post graduation and am very familiar with cloud-infra, but after I became a PM leader, I skipped many years without writing any serious application code. And despite the hiatus, now I suddenly find myself a more efficient and productive coder than I was at the peak of my undergrad/early SWE career!
In my current setup, I use:

Languages: Python, TypeScript
Tools: Cursor IDE with Claude 3.7 Sonnet, Vercel V0, Github, CLI
Assistant: ChatGPT (including deep research), Gemini for Google Cloud (specially for GCP infra)
Managed services and database: Firebase, Cloud Run, Firestore (NoSQL)
Model APIs: OpenAI APIs
Evals: OpenAI Evals, Ragas

Choices and learnings:
I stayed away from Agentic frameworks as they are confusing and seemed like an overhead. Using Python libraries and model APIs directly, along with simple workflow and agentic patterns (published by Anthropic in Dec’24) accomodate almost all logic I have needed so far.
Once I have thought through the Agentic/workflow logic I want to make progress on, I start with the closest pattern in a credible repo I can find (ChatGPT is able to help here).
I use Cursor (heavily) for sample codebase explanation, syntax error correction, formatting, imports, and local testing. Recently also started using Agent mode to draft complete logic including data models and functions, complete with retry/error handling logic.
I iterate a frontend/UX concept directly in Vercel V0, and get to a reasonable looking first draft, usually within a few prompts. And then I prefer downloading the code and iterating on it within Cursor IDE.
Debugging deployment issues with Cursor has been counter-productive, and I found myself going in circles. Since I am using GCP infra, I have found Gemini for Google Cloud (available across multiple surfaces e.g., Cloud console, Firebase console etc.) to be directionally more helpful in resolving issues faster.
Observations:
While not as useful to me given my flow and current goals, I was curious about engineering collaboration features. I found PR summary and automatic code review features to be better with Copilot, given it's deeper integration into Github platform.
I have found all AI for Code tools to be lacking in cloud-native or system design. Moreover, IDE as a UX for system design thinking and decisions is filled with many friction points (missing context from sources other than codebase, and inability to iterate, share, and collaborate). I have also tried tools other than Cursor (like Copilot and Augment Code) for system design, and found the results to be mostly the same. This is not surprising since the underlying models are trained specifically for code completion/generation/explanation/review tasks.