Engineering the Future: Sai Sreenivas Kodur on Scaling AI Systems That Think, Learn, and Operate at Enterprise Scale
The breakthroughs in AI today aren’t happening in research labs. They happen at 2 AM, when production systems fail, on-call engineers scramble, and decisions need to be made in milliseconds.
Sai Sreenivas Kodur has spent the last decade in those moments. From high-scale search infrastructure to voice analytics platforms and a pioneering AI company for the food and beverage industry, Kodur has worked at the sharp edge of what it means to build AI systems that not only work but endure.
From Systems Research to Scalable Reality
Kodur’s engineering mindset was forged at IIT Madras, where his graduate research blended machine learning with compiler optimization algorithms to improve performance across heterogeneous computing environments.
“The real value wasn’t just the technical depth,” he says. “It was learning how to design systems that solve real constraints across architecture, data, and performance.”
That systems-first framing, treating ML not as magic but as part of a larger machine, became a recurring pattern in his career.
It wasn’t long before he’d be putting those ideas to the test, in production.
Making AI Work in Production
At Myntra and later at Zomato, Kodur led teams that built search and recommendation systems for millions of users. Traffic surged. Catalogs are updated in real time. The margin for error was thin.
“At that scale, it’s not just about a better prediction, it’s about infrastructure,” he explains. “Caching, freshness, indexing logic, these aren’t backend concerns. They are the product experience.”
In one case, a latency misalignment between the model and the cache caused expired items to appear in user feeds. A tiny detail, but in e-commerce, tiny details cost millions.
“That’s when it clicked for me. Scaling AI isn’t about scaling models. It’s about designing the systems around them.”
Serving the Enterprise: Reliability as a Feature
Kodur’s next chapter took him deeper into the enterprise. At Observe.AI, as Director of Engineering, he led platform, analytics, and product engineering just as the company began onboarding major enterprise clients.
Suddenly, the rules changed. Uptime wasn’t a feature; it was a contract. Compliance, observability, and auditability weren’t nice-to-haves; they were essentials. They were table stakes.
“We couldn’t just add features. We had to re-architect the platform to deserve trust,” he says.
The work paid off: his team introduced data observability layers that slashed operational tickets by 60%, redesigned infra to support 10x growth, and supported $15M+ in ARR from new enterprise customers, including Uber, DoorDash, and Swiggy.
“Enterprise AI doesn’t scale by brute force. It scales through clarity. Every layer from the API to the database has to carry the weight.”
Building Spoonshot: A Vertical Intelligence Stack
While at Observe.AI, Kodur also began to see the limitations of general-purpose AI. In sectors like food and beverage, where regulation, science, and sensory data drive decisions, off-the-shelf tools fall short.
So he co-founded Spoonshot, an AI company purpose-built for food innovation.
“We weren’t just analyzing data. We were building a brain for food,” he says.
Spoonshot’s core engine, Foodbrain, ingested over 100TB of alternative data from 30,000+ sources. It mapped ingredients to sensory trends, regulatory data, flavor compounds, and consumer insights, surfacing opportunities that human R&D teams often missed.
“One client spotted an emerging spike in ‘umami’ trends months before it hit retail. That kind of signal isn’t in your sales data, and it’s buried in food science and niche blogs.”
The platform, Genesis, became a trusted tool for companies like Coca-Cola, Heinz, and Pepsico to develop new products faster and with greater confidence.
“Domain-aware AI isn’t just ‘smarter.’ It’s more respectful. It understands the user’s world, not just their data.”
Research That Fixes Real Problems
Kodur’s contributions to AI don’t end at products. He’s also published practical research grounded in day-to-day engineering pain.
His 2025 paper on Debugmate, an AI agent for on-call triaging, tackled a universal developer nightmare: late-night outages and complex system failures.
“Ask any engineer what they dread. It’s not bad code; it’s the moment you’re alone with a vague alert and 10 dashboards. Debugmate was our answer.”
By correlating observability signals, internal system knowledge, and historical tickets, the agent reduced incident load by 77%. Not a theoretical operational relief.
“We weren’t trying to ‘do research.’ We were solving a problem we lived through.”
That ethos practitioner-first, problem-led is a hallmark of Kodur’s approach to AI systems.
Building an AI-Native Organization
In a recent three-part blog series, Kodur mapped out his thinking on what comes next: not just using AI to build software, but reorganizing teams and operating procedures on how software itself gets built with AI in the loop as both builder and operator.
“The old stack was built for human workflows. But today, assistants like Claude and Devin are not just writing code, they’re taking the role of pilots while human engineers are merely co-pilots.”
The challenge? Infrastructure hasn’t caught up.
“AI is now a user of your systems and a maintainer. The abstractions need to change.”
In his view, the AI-native organization needs:
- Self-observing platforms that diagnose and heal themselves
- Developer velocity abstractions that work with generated code
- Governance that assumes iteration is constant, not occasional
“Reliability won’t come from checklists. It will come from how the system is born.”
You can read the whole blog series at aiworldorder.xyz.
What’s Next: Compounding Machines
Looking ahead, Kodur believes that platform engineering will define the next decade of AI, not just as a post facto function, but as the backbone of systems that evolve autonomously.
“We’re not just shipping software anymore. We’re building compounding machines,” he says. “Every model you deploy trains another. Every insight feeds the next. If the platform can’t keep up, the whole thing collapses.”
His vision? A world where infrastructure is self-managing, where AI agents operate systems with accountability, and where every line of code moves us closer to scalable, resilient, domain-aware intelligence.
Final Thought: The Blueprint for AI Engineers
Image by DC Studio on Freepik
If you’re an engineering leader wondering how to architect systems for this new reality where AI isn’t a feature but a participant, Sai Sreenivas Kodur’s journey is more than a biography.
It’s a playbook.
Build for change, not control. Assume the AI is watching. And design your systems like they’ll be inherited by an agent with no context but full access.
Welcome to the AI-native era. Are your systems ready?
Want more stories like this? Explore AI Journ’s archive for practitioner-driven insights on building reliable, scalable, AI-first platforms.
You May Also Like
Atlassian’s Monumental DX Acquisition: Revolutionizing Developer Productivity for a Billion-Dollar Future
China Bans Nvidia’s RTX Pro 6000D Chip Amid AI Hardware Push
