Srini Singarapu

Building systems that keep improving

walk through the diagrams

I spent the summer at Phoebe, where we build AI agents for home-care businesses. One of my goals was to make our top of funnel meaningfully autonomous. this is some writing on how i tried to do this!

the two most important things I learned

design systems to own the outcome

I was trying to work backward from an outcome while still figuring out what the system needed to do

Define a measurable outcome, design the whole process around it, then measure the actual result. Results feed back into the process so the steps and handoffs can improve while the outcome remains the goal.

run every idea as a scientific experiment

I wanted to understand what we were actually testing. what was the hypothesis, why might it work, and what would good or bad look like?

Start with a hypothesis, run a fair test, measure whether the result supports it, then decide what to keep, change or stop. Carry what was learned into the next hypothesis.

our top of funnel

I started with the TAM, our total addressable market. We combined public records, company websites, and providers including Serper, Blitz, Clay, and Deepline to build records of companies, their locations, and the people working there

Then I connected that information to the rest of our outreach. New signals gave us a reason to contact someone. Calls, forms, and replies gave us more context. The system needed to carry that context through the conversation, with deterministic checks and human decisions where they mattered

Public records, company websites and data providers build the TAM. Signals and inbound activity add context. Email and LinkedIn use the relevant contact, signal and account history to start conversations. Incoming calls, forms and replies are matched to companies and people, checked against prior conversations, and followed up after review. Qualified meetings and customers feed the account history.

some terms that were outcome metrics

metric meaning what I used it to understand
CPQL cost per qualified lead Which sources brought relevant demand?
qualified meetings booked meetings with qualified prospects Did that demand become conversations?
CAC customer acquisition cost What did it cost to acquire a customer?
ROAS return on ad spend How much attributed revenue did the ad spend produce?

TAM agent

A new lead rarely arrived with everything we needed. I built the TAM agent to take a lead or company domain and do the investigation: check whether the company or person already existed in our TAM and CRM, work out what was missing, and research it

The agent ran in Modal sandboxes with our harness and could use multiple tools. Browserbase let it navigate websites like a person would - opening pages, following links, reading and parsing what it found. It combined that with independent searches and enrichment providers to collect company details, locations, contacts, and software information

Those findings became proposed additions or updates, with their sources attached. After review and approval, they went into the TAM. The same process could revisit existing records as new information came in, so the TAM kept changing with the businesses in it

A lead or company domain enters a Modal sandbox running our harness. The TAM agent checks the TAM and CRM for existing companies, locations and people. It chooses research steps using Browserbase to navigate and scrape pages, independent search and maps, and enrichment providers including Blitz, Clay and Deepline. Findings become a sourced proposal. The agent can investigate remaining gaps before human review and approval writes the additions or corrections into the TAM.

I brought the same approach to Google Ads. We shipped a self-evolving agent that monitored the account, made changes, measured the results, and learned from them. Google Ads data was joined to CRM outcomes through the click ID, so qualified leads and customers could guide its decisions

It could add negative keywords, pause or re-enable keywords and ads, adjust budgets and bid targets, add keywords, create responsive search ads, and run experiments. A planner produced a typed change plan, a critic challenged it, and the executor applied permitted or approved changes after checking the account state

The learning happened after execution. Results and human rejection reasons became memory for the next plan. Individual action types earned more autonomy through their track record, and lost it after a regression. The agent could propose updates to its own prompts, with replay tests and review before promotion. It also rolled changes back when performance crossed the configured limits

Google Ads account snapshots and search-term classification are joined to qualified CRM outcomes in Snowflake. A planner retrieves memory and produces a typed change plan; a critic keeps, revises or drops it. Code assigns automatic, Slack approval or blocked execution. The executor rechecks the account fingerprint and validates before applying operations. Scored experiments produce lessons and update autonomy by action type. Guardrail failures trigger rollback. Proposed prompt updates pass historical replay and human review before promotion.
Google Ads tools and change operations

The agent used these read tools to investigate the account. Write operations went into a typed plan for the executor

read tool what it retrieved
get_account_snapshot campaigns, ad groups, budgets, bidding, spend, and conversions
run_gaql read-only Google Ads queries
get_search_terms searches, their cost, and qualified outcomes
get_funnel_by clicks through to qualified leads, booked meetings, and customers
get_change_history agent changes and human edits
get_experiments open experiments and scored results
search_memory lessons, rejection reasons, and account facts
get_policy current limits and autonomy levels
change operation what it could do
add_negative_keywords exclude terms at campaign or shared-list scope
pause_entity pause a keyword, ad, or ad group
enable_entity re-enable an entity, including for rollback
adjust_budget change a campaign budget within policy limits
adjust_bid_target change target CPA or ROAS with approval
add_keywords add keywords and match types
create_rsa create a responsive search ad with approval
create_experiment create a campaign experiment with a traffic split and duration

why we changed harnesses

We started with Modal sandboxes + our harness, moved to Vercel Eve, and then to Devin (Cognition). Our team quadrupled during my time there. We needed a shared place where everyone could access the same data, see what the agents had done, and monitor both agents together

We began with Modal sandboxes and our harness, moved to Vercel Eve, then to Devin (Cognition). As the team quadrupled, the TAM and Google Ads agents moved into a shared operating environment with common data, run history and monitoring.

Hypotheses I tested

I wanted to see if the landing page was making it harder for people to book. PostHog showed visitors leaving through the navigation, so we tried fewer links and a form on the page itself. We tracked form completion, booked demos and CPQL to see if it helped

On Meta, we tried different video hooks, webinar clips, UGC-style videos and static ads. One clip got us cheap leads early on, but we couldn't get that result to hold up. We stopped that version. I was trying to understand what we could actually repeat

For the TAM, we compared what Blitz and Clay found and checked the agent's results against company websites. Some of the "phone numbers" were tracking IDs, so we had to fix how we pulled them out. I was also trying to work out who would respond and to what - a case study, a webinar or a demo

Three areas of experimentation. Google Ads landing-page variants were measured with PostHog funnels and session replays, form completion, booking rate and CPQL. Meta creative tests included video hooks, webinar clips, UGC and static ads; a clip version stopped when cheap early leads did not repeat. Outreach tested named accounts, signals and case-study, webinar or demo offers, following replies through to qualified meetings

final thoughts

I think the biggest thing I took from this summer was that my job didn't end when something shipped. I still had to see whether it helped us get customers, and figure out what to test next when it didn't. I want to keep getting better at both - building the system and staying responsible for the outcome

how it works