Puneeth Kotha

Classify 1.4M parent entities — Individual, Company, Family Firm, Government — across 120+ countries, where naming conventions are inconsistent and the text is multilingual.

A cascade that spends compute only where it has to: deterministic rules catch the obvious cases at near-perfect precision, an LLM (Claude, batched) resolves the ambiguous middle, and a fine-tuned XLM-RoBERTa handles the multilingual long tail with confidence-threshold abstention.

Hardest part. Calibrating the escalation thresholds — how confident each stage must be before it hands off to the next, more expensive one.

Teams ship agent regressions because observability isn't evaluation — and an LLM judge you can't trust can't grade the build. I wanted one defensible pass/fail that also watches the judge.

Error-analysis first: sample failing traces, embed, and cluster them into a failure taxonomy, then run deterministic code checks plus a calibrated LLM judge. The judge is re-scored against a frozen human anchor set every run (Cohen's κ + Gwet's AC1); the gate reads a Wilson score interval and compares versions with McNemar's paired test.

Hardest part. Keeping the verdict trustworthy on tiny samples — Wilson lower-bound gating and McNemar paired tests so a two-sample wobble can't flip the build, plus κ-drift detection so a drifting judge is caught, not obeyed.

Building complex LLM workflows means a code change for every adjustment — and scripts break silently when an upstream schema shifts. I wanted to describe a workflow in natural language, have it run reliably, and catch its own failures.

An LLM-powered compiler turns a natural-language description into an executable DAG, then a parallel asyncio scheduler runs it with topological batching, 5-check corruption detection (cardinality, required, non-nullable, range, freshness), and failure-classified retries (rate-limit → wait, network → backoff, logic → halt). The whole pipeline runs on a free real-time LLM — Groq by default, swappable to Gemini/Claude/OpenAI/Ollama with one env var.

Hardest part. Self-Heal: auditing a workflow's resilience and running a live monitor → detect → diagnose → recover → verify loop — turning fragile nodes red, applying fix patches, and re-verifying until the reliability score climbs back to green.

A model deployed as a single instance is a bottleneck and a single point of failure. Inference needs to scale, degrade gracefully, and be observable.

A multi-worker FastAPI service behind Nginx with Redis caching, idempotency-key dedup, a circuit breaker (5-failure threshold, half-open recovery), exponential-backoff retries, and graceful shutdown that drains in-flight requests.

Hardest part. Getting the circuit breaker and in-flight draining right so a bad deploy degrades instead of dropping requests.

Financial analytics needs sub-second processing and fault tolerance under bursty, high-throughput load.

A Kafka + Spark streaming pipeline with hybrid PostgreSQL + InfluxDB storage tuned for time-series queries, consumer groups and checkpointing for fault tolerance, and Grafana for observability, with rule-based alerts on price/volume/volatility.