The evaluation framework practitioners wish they had before signing their first contract.
TL;DR
Most buyers evaluate cold email agencies the wrong way. They look at case studies, Google the agency name, check a few reviews, and make a decision. Then 90 days later they wonder why they are paying $5,000 a month for 2 meetings. This guide covers the 8 criteria that actually separate good agencies from bad ones -- and 10 specific questions to ask in the sales call that will tell you everything you need to know.
Ask the agency how they define your ICP. Vague answers -- 'B2B SaaS companies, 50-500 employees' -- signal template-based execution. Strong agencies define ICP with signals: companies that recently raised Series B, expanded their sales team in the last 90 days, and use Salesforce. That specificity gets 3-5x the reply rate of industry + headcount targeting.
Who owns the domains and mailboxes? This is the highest-stakes question in any agency evaluation. There are two models. First: the agency builds dedicated infrastructure in your name -- domains you own, mailboxes connected to your systems. When you leave, you keep the assets. Second: the agency sends from their shared infrastructure, routing replies through forwarding setups. Your deliverability is tied to their other clients' behavior. One bad client in their pool damages your campaigns.
Do not ask for case studies. Ask for real email examples sent to clients in your industry in the last 60 days. Case studies are curated. Real emails are not. Look for: specific personalization hooks (not 'I noticed you work at X'), a clear and credible value statement, one ask at the end (not a multi-question close), and no corporate language. If they cannot share examples without a lengthy NDA process, they are hiding generic copy.
Agencies that lead with open rates are hiding behind vanity metrics. Open rates measure deliverability, not campaign quality. The only metrics that matter: meetings booked, cost per qualified meeting, and show rate on booked meetings. If an agency cannot tell you their average cost per qualified meeting across clients, they either do not track it or the number is bad.
Ask what tools they use for contact discovery, enrichment, and copy generation. Apollo export + mass send is a 2022 approach. Strong agencies use Clay or equivalent for signal-based targeting, AI enrichment to generate personalized copy at scale, and dedicated sequencing tools with deliverability monitoring. The stack signals how much they invest in doing the work well versus how much they rely on list purchasing.
Domain warmup takes 2-3 weeks minimum. Any agency promising live campaign sends within 2 weeks of kickoff is either skipping warmup or sending from pre-warmed shared infrastructure. The right timeline: Week 1 (ICP workshop + account list), Week 2-3 (domain purchase + DNS + warmup starts), Week 3-4 (copy written + reviewed), Week 4-5 (first sends). That is 4-5 weeks from kickoff to first emails out.
Check three things. First, lock-in length -- 3 months is reasonable; 6-12 months with no performance clause is a risk. Second, performance exit -- can you cancel with 30 days notice if the agency misses an agreed meeting threshold for two consecutive months? Third, data ownership on exit -- do you get your contact list, copy templates, and campaign data when you leave? Some agencies retain this as proprietary. That is unacceptable.
Ask who writes your copy and who manages your campaigns day-to-day. Agencies with 10 clients per account manager are running a volume business, not a quality business. The person who sold you the engagement should be able to name the specific copywriter and campaign manager assigned to your account. If those are different people from the ones on the sales call, ask to meet them before signing.
| Behavior | What It Signals |
|---|---|
| Leads with open rate metrics in reporting | Hiding behind vanity metrics; no meetings to show |
| Cannot share real email examples without lengthy NDA | Generic copy they do not want you to see |
| Promises first sends within 2 weeks | Skipping domain warmup or using shared infrastructure |
| Uses Apollo bulk export as their prospecting method | No signal-based targeting; list quality will be low |
| 12-month contract with no performance clause | Low confidence in their own results |
| Account manager covers 15+ clients | You are not getting dedicated attention |
| No email verification step in their process | High bounce risk; will damage your sender reputation |
| Unclear who actually writes the copy | Copy is written by a junior or outsourced overseas |
| Monthly Budget | What You Get | What You Give Up |
|---|---|---|
| $2,000-3,000/mo | One campaign, basic prospecting, template-based copy | Customization, dedicated team, signal-based targeting |
| $3,500-5,000/mo | 1-2 campaigns, dedicated account manager, some personalization | Deep ICP research, multi-persona coverage |
| $5,000-7,000/mo | Multiple campaigns, signal-based targeting, custom copy per persona | Performance fees, enterprise-tier tooling |
| $7,500+/mo | Full-stack execution: research, infrastructure, copy, optimization, reporting | Nothing at this tier -- this is what a complete engagement looks like |
Budget tier is less important than whether the agency can answer the 10 questions above. A $3,000/mo agency that owns your infrastructure, writes specific copy, and reports on meetings booked is worth more than a $7,000/mo agency that shares domains, sends generic templates, and reports on opens.
Below $200 typically signals low qualification standards or inflated meeting counts. Above $500 is common for enterprise ICP with small total addressable markets and long buying cycles.
A proposal tells you as much about an agency as their sales call does. Here is what to look for.
We build and run these systems embedded with your team.