How to Choose a Performance Creative Agency: 10 Questions and a Scorecard

Choosing a performance creative agency is a procurement decision dressed up as a creative one. The decks you receive will show polished ads, familiar logos and a headline ROAS. None of that tells you whether the agency can run the loop that moves your CAC: brief from data, produce in batches, test in media, read the result, kill or scale, brief again. Production volume has stopped separating agencies.

Forrester reports that nearly nine out of ten marketing agencies now use generative or agentic AI in marketing creation and delivery, so every shortlisted shop can make a lot of ads quickly. What still differs is the testing discipline around that volume and whether media buying sits inside it. This guide gives you ten questions and a weighted scorecard to find that out before you sign.

P.S. If your bottleneck is tested creative volume rather than media settings, see how inBeat's performance creative team builds and tests concepts alongside the buyers who scale them.

inBeat POW: We think most agency pitches answer a question you didn't ask. They show you what they made; you need to know what they learned, and how quickly it changed the next batch. Creative, media and performance are one system. When the people making the ads sit apart from the people buying the media, test results stall in a dashboard and never reach the next brief. Media buying is the laboratory: dark-post the concepts, read the signal, scale only what earned it. As platform targeting narrows, the persona inside the creative does the audience selection that settings no longer can, so the agency you hire has to own both the concept and the readout. Spend that produces no learning is the real waste, and that's exactly what a weak agency choice buys you.

TL;DR: How to choose a performance creative agency

The short version, for the kickoff note you send before any pitch:

• Hire for the loop. The agency must show how a creative test result changed the next brief and the next media allocation, with the artifacts to prove it.
• Sort the three agency types by what they're paid for (assets, media efficiency, or tested creative that scales in media) before you compare a single fee.
• Score every candidate on the same ten questions and a weighted scorecard, so the decision survives procurement, a CFO review and the agency's best presenter.
• Put creative-level data access in the contract: hit rate, CPA by concept and fatigue curves, delivered on a weekly cadence instead of buried in a monthly ROAS summary.
• Treat heavy AI use as neutral. With nearly nine in ten agencies already using generative or agentic AI per Forrester, the differentiator is the human test design around it.
• Price the engagement against MER and CAC movement over a defined period, with a written kill rule for losing concepts and a refresh cadence for winners.

A VP of growth holding three decks can use these six lines to decide which slides are evidence and which are decoration. Portfolio reels and logo walls fall into the second group. A rewritten brief, a kill list with dates and a CPA-by-concept export fall into the first. The rest of this article explains why each bullet holds and how to score it.

Performance creative agency vs creative agency vs performance marketing agency: what's the difference?

The three models differ in what they're paid for, and that commercial incentive predicts how each will behave with your budget far better than the agency's own label does.

Creative or brand agencies are paid for assets and brand direction

A creative or brand agency sells campaigns: a platform idea, a visual system, a set of finished assets, often judged by approval and awards. Its unit of output is the campaign. That's a legitimate purchase when you need a brand position, but the model breaks for paid social because it has no performance data in the room. The team learns from research and reviews; nobody in the room sees CPA by concept, so a losing idea can live for months.

Performance marketing agencies are paid for media efficiency on creative they did not make

A performance marketing agency sells efficiency: lower CPA, higher ROAS, better pacing, measured on creative it typically receives from you or a third party. Its unit of output is the media metric. The model breaks when creative volume dries up. The buyer cannot fix a tired concept with bid changes, and the usual response is to ask the client for more assets and blame the delay.

There's a historical reason this matters.

A Nielsen analysis found that media's effect on sales had risen to 36% from 15% over the preceding 11 years, which Nielsen attributed to better targeting. That study predates today's algorithmic delivery on Meta and TikTok, so it cannot tell you how much of performance now comes from media versus creative.

What it does show is that media execution became more influential as targeting improved. For an agency evaluation, that means creative results should be read alongside the media conditions that produced them rather than treated as proof of creative quality on their own.

Performance creative agencies are paid for creative that is tested in media and scaled on results

A performance creative agency sells tested concepts. Its unit of output is a concept that survived a dark-post test and earned budget, plus the learning from the ones that didn't.

That requires owning, or sitting inside, the media buy. The model breaks when an agency is really one of the first two with a new label, and Frank Advertising's comparison of creative and performance agencies is a reminder of how often the two are still sold separately.

The tell is simple: ask who owns the test plan and who reads the results. If the creative director owns the plan and a separate media vendor reads results into a weekly email, you're buying a creative shop. If the buyer owns the plan and creative arrives on request, you're buying a media shop.

The recommendation:

• if you already run meaningful paid social spend and your constraint is creative volume and learning, the integrated model wins.
• If your constraint is the absence of a brand platform, it doesn't, and hiring for the loop will leave you with well-tested ads for a brand nobody has defined.

Pro tip: Classify each shortlisted agency into one of the three columns with the test-plan question before you score anything else.

How a performance creative agency integrates creative production and media buying

The integration is a seven-step loop, and the honest test of any agency is whether a named artifact changes hands at every step. Draw your current process against these steps and mark where nothing is handed over; those gaps are what you're hiring for.

Step 1 to 3: brief from data, produce in batches, dark-post the concepts

1. Brief from data. The media buyer opens the brief with a readout: which personas and angles are under-served, where CPA is rising, which winners are approaching fatigue. Handoff artifact: a test matrix listing personas, angles and formats with a hypothesis per cell.
2. Produce in batches. The creative team fills the matrix with net-new concepts and variations, including creator and UGC assets where persona matching calls for them. Handoff artifact: a batch sheet mapping every asset to its matrix cell and naming convention.
3. Dark-post the concepts. The buyer launches the batch in a test structure that holds audience and bid steady so creative is the variable. Handoff artifact: a launch log with budgets, start dates and the minimum spend or conversions each concept needs before judgement.

Step 4 to 6: read creative-level signal, kill or scale, whitelist winners

4. Read creative-level signal. The buyer reports CPA by concept against the account control and leaves blended ROAS out of the readout. Handoff artifact: a readout that states which concepts beat control, by how much and at what confidence.
5. Kill or scale. Losers are paused on a written rule; winners get budget. Handoff artifact: a dated kill list and a scale list, each with the reason.
6. Whitelist winners. Creator-led winners move to whitelisted or partnership placements where the creator's own handle carries the persona. Handoff artifact: an updated media plan showing where the scaled budget came from.

Step 7: feed results into the next brief and the media plan

7. Feed results back. The readout becomes the first page of the next brief, and the media plan reallocates toward the angles that earned it. Handoff artifact: the next test matrix, annotated with what the last round proved and disproved.

How often this loop turns, and how big each batch is, depends on spend level and platform. An account spending modestly on one platform might turn the loop every three or four weeks with a handful of concepts; a large multi-platform account may run weekly batches of dozens. Any agency that quotes a fixed number without asking your spend is quoting from its production capacity and has ignored your learning needs.

AI belongs in steps two and, increasingly, four. Given the Forrester adoption figure cited above, AI-assisted production is likely to be a baseline capability across most serious candidates. The differentiator is human judgement in steps one, three and five: who designs the matrix, who decides the minimum spend before a verdict, and who signs the kill list. Ask to see those three artifacts from a live account, with client names redacted if needed.

The cost of leakage between steps is large relative to any fee. R-Advertising cites a Statista projection that global digital ad spend will reach $876 billion by 2026; that's a secondary-reported projection, not an audited figure, but it frames the stakes. Every month a winning concept keeps running past fatigue, or a losing one survives because nobody owned step five, is media spent without learning.

For a published example of creator-led performance marketing, our Booksy campaign generated 1,700% growth in business registrations, a 637% increase in app installs and a 92% lower cost per registration, as shown in the case study snapshot below.

10 questions to ask a performance creative agency before you sign

Send these ten questions to every shortlisted agency 48 hours before the pitch, then score the answers live against the strong, weak and red-flag columns below. Agencies that run the loop described above will find the questions easy; agencies that don't will find them oddly specific.

Questions 1 to 4: testing workflow and creative volume

Q1. How many net-new concepts and variations do you ship per month at my spend level, and who decides the test matrix? You're probing steps one and two. A strong answer asks about your spend and platforms before giving a range, and names the role that owns the matrix. A weak answer gives one fixed number for every client. The red flag is a volume figure with no mention of a matrix at all, which means production isn't tied to a hypothesis.

Q2. What is your rule for calling a loser, and how fast? This is step five and the most revealing risk question. A strong answer states a threshold (spend or conversions reached, CPA versus control) and a maximum number of days before a verdict, plus who approves the pause. A weak answer says it depends on the client. The red flag is that losers are rarely paused because they still deliver reach, which tells you the agency is paid for volume.

Q3. Show me a brief that was rewritten because of test data. Darkroom's guide on what to look for in a performance creative agency frames this as evidence of the process, and that's the right framing: with AI production commoditised, the rewritten brief is the proof of human discipline. Strong: a redacted before-and-after brief with the readout attached. Weak: a verbal story. Red flag: the briefs are the client's responsibility.

Q4. How do you detect and respond to creative fatigue on a winner? Strong answers name the signals (frequency climbing, CPA trending up over a stated window) and the response (a refresh variation briefed before the winner dies). Weak answers mention refreshing creative periodically. Red flag: fatigue is handled by the media team swapping audiences.

Questions 5 to 7: media integration and measurement

Q5. Does the same team buy the media, and if not, how do results reach the creative team? Strong: yes, or a named weekly readout meeting with your incumbent buyer and a shared sheet. Weak: a monthly report that the creative team can read. Red flag: the agency hasn't thought about it.

Q6. Which creative-level metrics do I get access to, in what tool and how often? AppsFlyer's State of Creative Optimization report shows how granular creative performance analysis has become, breaking results down across creative elements, formats and media types. Strong: direct access to CPA by concept, hit rate and fatigue curves in a named dashboard, refreshed at least weekly. Weak: a PDF monthly. Red flag: aggregate ROAS only.

Q7. How do you separate creative performance from audience and bid effects when you read a test? Strong: a test structure that holds audience and bid constant, a minimum spend before verdicts, and an admission of what can't be isolated. Weak: the platform's algorithm sorts it out. Red flag: no structure, only post-hoc reporting.

Questions 8 to 10: proof, pricing and what happens when creative fails

Q8. Walk me through one case study with the raw numbers, the period and what you cannot claim from it. This is the trust question. A strong answer gives the baseline, the period, the spend range, whether the agency owned both creative and media, and at least one limitation volunteered without prompting. A weak answer is a slide with a percentage. The red flag is a result the agency cannot explain the baseline for.

Q9. How is your fee tied to output or outcome, and what is excluded? Strong: a clear structure, a defined review metric and an explicit list of exclusions such as media spend, creator fees or usage rights. Weak: a retainer with a vague deliverables list. Red flag: a performance bonus on blended ROAS the agency doesn't fully control.

Q10. Which industries or audiences have you not worked in, and how would you learn mine? A credible agency names gaps and explains its persona-matching and research method; our own notes on performance creative for NYC DTC brands show how category context changes the angle matrix. Strong: named gaps plus a research plan. Weak: a claim of universal fit. Red flag: a logo wall offered as the answer.

Use the table as the live scoring sheet during the pitch and the prose above as the key for borderline answers.

Performance creative agency scorecard: how to weight and score each criterion

Weight the scorecard to your bottleneck, score each agency against the ten questions, and apply two gates before you read the total. That order stops the best presenter from winning on polish.

Weighting the five criteria by your actual bottleneck

The five criteria map directly to the questions: testing workflow (Q1 to Q4), media integration (Q5 and Q7), measurement access (Q6), proof quality (Q8 and Q10) and commercial fit (Q9). The default weights below are a starting point we'd use for a brand with meaningful paid social spend and a creative bottleneck; they are editorial judgement with no measured basis, and the third column tells you when to move them.

Measurement access deserves heavy weight anywhere budgets have moved decisively online, because the money you're steering is increasingly digital and therefore auditable.

One regional illustration: Andava projects that by the end of 2025 more than 70% of UAE marketing budgets would be dedicated to digital channels. That figure is UAE-specific and shouldn't be read as a global benchmark, but the direction it describes is why a monthly PDF is no longer an acceptable reporting standard.

Scoring rules that stop the best presenter from winning

Score each criterion from 1 to 5 using the strong and weak answers as anchors: a 5 is the strong answer with artifacts shown, a 3 is the strong answer described but not shown, a 1 is the red flag. Three rules keep the sheet honest. First, each stakeholder scores independently before anyone speaks, then you reconcile differences above one point in a 30-minute session. Second, anything scored 4 or 5 must cite an artifact you saw, not a sentence you heard. Third, apply two gates regardless of total: an agency scoring below 3 on testing workflow or below 3 on measurement access is removed. A high total can hide a disqualifying gap, and these two criteria are where the gaps hide.

A hypothetical worked example: two agencies both total 3.9 weighted. Agency A has a stronger portfolio, scores 5 on proof quality, but scores 2 on measurement access because it offers only monthly aggregate reporting. Agency B scores 3 on proof and 4 on measurement access. Under the gate rule, A is removed despite the nicer deck, because you cannot run the loop without seeing concept-level data.

Reading the result: thresholds and tie-breakers

Treat these thresholds as an editorial decision rule rather than an industry benchmark. A weighted score above 4.0 with both gates passed is a strong signal to proceed. Between 3.0 and 4.0, move forward only through a 90-day pilot with the exit criteria described in the pricing section. Below 3.0, pass. For ties, break on testing workflow first, then on the quality of the Q8 answer, since the agency that volunteers a limitation is the one that will tell you when a concept fails.

When scoring case studies specifically, credit only numbers stated on a public page with a period and a baseline, and ask whether the agency owned both creative and media during the engagement; a result achieved on someone else's media plan says less about the loop.

If you're also weighing broader growth agency options, our seven steps to choose a growth agency covers the procurement side that this scorecard assumes.

Which performance creative agency KPIs matter: CPA, MER, creative hit rate and fatigue

An agency should report two kinds of metrics: scoreboard metrics that show what the business is paying for, and process metrics that show whether the loop is turning. An agency reporting only the first can't explain its own results; one reporting only the second is grading its homework.

Scoreboard metrics: CAC, MER and CPA by concept

CAC and MER are the business scoreboard. MER (total revenue over total ad spend) is the number your CFO trusts because it doesn't depend on platform attribution, and CAC is the cost the growth model is built on. Neither isolates the agency's contribution on its own. CPA by concept gets closer to the creative-level signal by showing what each idea cost to acquire a customer against the account control, provided the surrounding test conditions are reasonably consistent. Insist on concept-level reporting, since ad-level numbers let a dozen variations of one idea look like twelve wins.

Process metrics: hit rate, time-to-kill and refresh cadence

Creative hit rate is the share of concepts in a batch that beat the account's control CPA. There's no sourced benchmark in our research to quote, and any agency that quotes a universal one should be asked what spend, category and test design it assumes, because all three move it. Time-to-kill is the days between a concept reaching its minimum spend and the pause decision; it measures the kill rule from Q2 in practice. Refresh cadence is how often a fatiguing winner receives a briefed variation, and the fatigue report behind it should show frequency and CPA trend per winning concept with the trigger the agency committed to.

Metrics to discount: aggregate ROAS screenshots and engagement rates

Aggregate ROAS is the easiest number to flatter. The Nielsen analysis discussed earlier, which found media's share of sales effect rising over the eleven years to 2017, cuts both ways here: if media quality explains a growing share of outcomes, a blended ROAS lift may belong to the buyer, the bid strategy or the season, and it cannot tell you what the creative did.

The study predates algorithmic delivery, so the exact split today is unknown, which is precisely why creative-level reporting is more useful than blended account metrics when you want to understand what the creative contributed. Engagement rates belong in the same bin unless the agency can show they predict CPA in your account.

Ownership and cadence are our recommendations; AppsFlyer's creative optimization report linked in the questions section describes the creative-level measurement category these metrics sit in.

Attach this reporting checklist to the SOW as a required monthly deliverable, with each definition written out:

• CPA by concept against control, with spend and conversions per concept
• Hit rate per batch with the control definition used
• Kill list and scale list with dates and reasons
• Fatigue report: frequency and CPA trend for every scaled concept
• Time-to-kill distribution for the month
• MER and CAC movement over the review period, with the attribution window stated

Performance creative agency pricing models and how to tie fees to ROI

Pick the fee structure by what it rewards, then add the clause that offsets its incentive problem. No price figures are in our research for this category, so what follows is structure.

Retainer, per-asset and performance-linked fees compared

The table is our synthesis; the rewards and failure modes follow from the incentive logic in the comparison section. Since published pricing varies widely by scope, compare fee structures by what they reward and what the contract requires the agency to deliver.

What each model incentivises the agency to do

A retainer pays the agency to exist, so it needs a learning obligation attached. Per-asset pays for output: a hypothetical contract shipping 40 variations a month with no readout costs more in media waste than a smaller retainer with a mandatory kill list, because nobody is paid to stop the losers. Performance-linked fees are the most aligned when the agency owns the buy, and the most dangerous when it doesn't.

Tie at least part of the review to MER and CAC movement over a defined test period, as discussed in the KPI section, and never to asset count. Judge the fee against the media budget and learning rate in your own account, and optimise the contract around how quickly the agency turns spend into useful creative decisions. If your buy sits with a separate paid media partner, write the readout meeting into both contracts.

Contract clauses that protect you when creative does not perform

• Days 1 to 30: audit the account, agree the control CPA, deliver the first test matrix and batch.
• Days 31 to 60: run two full loops, deliver kill lists, readouts and the first rewritten brief.
• Days 61 to 90: scale winners, deliver the fatigue report and a 90-day MER and CAC readout against baseline.
• Exit criterion: every agreed artifact is delivered on cadence and the account shows enough signal to judge the testing process. If the workflow is sound but no concept has beaten control at scale yet, extend the pilot with a defined next review date. If the artifacts or testing discipline are missing, end the engagement.

How inBeat runs performance creative and media buying as one system

We built inBeat around the loop this article describes, which is why we're comfortable being scored on it.

Creators and UGC as the creative supply

Our creative supply comes from creators, mostly micro, matched by persona and sourced at volume, alongside performance creative produced in house. That matters for the loop because a persona matrix needs a matching creator for every cell, and because when you collaborate with a creator you buy their likability and their read on their audience along with the file. The feed and the comments are the first round of research for the next brief.

Media buying as the test lab

Our paid social team runs the dark-post tests, reads concept-level signal and whitelists what earned it, with the readout feeding the next brief. The Booksy snapshot in the integration section is the published example we reference here, with the figures taken directly from that captured case-study result.

What to bring to a first conversation

Bring your completed scorecard for the agencies you've already heard from, your current CPA by concept if you have it (and the fact that you don't, if you don't), your test cadence and batch size, and the control CPA your finance team trusts. With those four things, the first call is a diagnosis of where your loop leaks, and we can tell you quickly whether the gap is creative volume, test design or the buy.

Then talk to inBeat's paid social team about running creative and media as one loop.

Performance creative agency FAQs

Should I keep my current media buyer and hire a creative-only agency instead of an integrated one?

Keep the buyer if their results are strong and they'll commit to a weekly joint readout with the creative agency, documented in both contracts. The risk in the split is step seven: results that never reach the next brief. If the buyer treats creative as an input they receive rather than a variable they test, the split will cost you more in stalled learning than a switch would, and the integrated model is the safer choice.

How large does my paid social budget need to be before a performance creative agency makes sense?

Large enough that each concept in a batch can reach a verdict within a few weeks. The threshold depends on your CPA and conversion volume: an account with low-cost conversions can test many concepts on modest spend, while a high-ticket account needs more budget per concept to read anything. Below that point, a per-asset creative supplier plus a disciplined in-house buyer usually makes more sense than a full loop.

What should the first 90 days with a performance creative agency produce if it's working?

Two or three full loops with every artifact delivered: test matrices, batch sheets, readouts, dated kill lists, a fatigue report and at least one rewritten brief. On the scoreboard, at least one concept beating control at scaled spend and a documented MER and CAC readout against baseline. If the artifacts are there but the scoreboard hasn't moved yet, extend the pilot; if the artifacts are missing, end it.

How does heavy use of AI in production change what I should evaluate?

It shifts your evaluation from production to judgement. Since most agencies now use AI in creation, as the Forrester figure in the introduction shows, volume and speed tell you little. Evaluate who designs the test matrix, who sets the minimum spend before verdicts, who signs the kill list, and whether the agency can show a brief that changed because of data. Ask also how AI-generated assets are checked for brand and compliance before launch.

Who should own the ad accounts and the creative files when the engagement ends?

You should. Ad accounts, pixels and audiences stay in your business manager with the agency as a partner, so nothing is lost at exit. Creative files, briefs, test matrices and readouts should transfer to you under the contract, with usage rights for creator content spelled out by duration and channel. If an agency insists on owning the account, treat it as a red flag on commercial fit and walk.

Cover photo: Photo: Maksim Goncharenok / Pexels. Art direction: inBeat Agency.