Three vendors respond to the same RFP. All three say they rank sponsored and organic products together. 

One calls it holistic page optimization. One calls it unified decisioning. One calls it a single ranking system. 

Underneath those three phrases sit at least three different architectures, with different answers to what you can change in year three, which is why you are really comparing vendors that are not the same shape.

The argument about whether sponsored products should be ranked by bid alone, in a silo, next to organic results chosen by a far more sophisticated system, is basically over. Where that combined decision runs is not. So stop grading the phrase and start grading the architecture. The five questions below come from language retailers are already using in live RFPs, and none of them can be answered convincingly by a vendor whose system does not work the way the slide says.

‍

TL;DR

• Several vendors now sell some version of ranking sponsored and organic products together, under different names. The idea has converged. The architecture has not.
• The phrase on the slide tells you almost nothing. Where the ranking decision physically runs tells you almost everything, including what you can replace later and what you cannot.
• Five questions separate a real combined ranking system from a repackaged one: where the decision lives, what happens to your existing systems, whether the proof runs on your traffic, whether relevance means better ads or just more of them, and what you can undo.
• A vendor who answers these with a named data flow and a test design is describing something real. A vendor who answers with a product name is asking you to take it on faith.
• Independent A/B testing across multiple retailers shows 80 to 140 percent more ad revenue from existing ad inventory, with no new ad slots added, and roughly double the click-through rate versus legacy ad serving alone. Source: Pentaleap internal A/B testing, 2024 to 2025.

‍

Where does the decision actually live?

Ask this first, because every other answer depends on it. Every vendor in this category has something that decides which products appear in which positions. There are only three places that something can sit: inside the ad server, inside the search and personalization engine, or in an independent layer between the two.

Each of those is a real, defensible choice with different consequences. Inside the ad server, decisioning stays bundled with demand and campaign management, which is commercially tidy and makes adding a competing demand source hard. 

Inside the search engine, the decision runs close to the data and latency improves, but your search provider and your media business are now entangled, so changing one means touching the other, and real-time bidding gets difficult. In an independent layer between the two, each piece stays replaceable, at the honest cost of one more vendor in the stack.

What a good answer sounds like: a diagram, a named sequence of calls, and a clear statement of which system makes the final call on position. What a weak answer sounds like: the name of a proprietary engine, repeated.

The follow-up worth asking in the same breath: at the moment your ranking decision runs, what does it know? If the sponsored decision can only see bid and budget, the page is largely decided before a shopper arrives. If it can see the organic ranking, inventory, margin and shopper context, that is a materially different system regardless of what either one is called.

‍

What happens to our existing search AI and ad server?

Douglas and MediaMarktSaturn both wrote versions of this into their RFPs, and the phrasing is specific in a way that matters: describe your integration approach with our existing search provider, not a plan to replace it.

Ask the vendor to state plainly which of your current systems stay, which get replaced, and which become optional later. 

This is one of the requirements most retail media RFPs leave out entirely, usually because the document was written from the incumbent's feature list. 

Then ask what happens to the relevance work your team has already done. Search and personalization engines get retuned constantly. If adopting a ranking vendor means that tuning now lives partly inside their system, you have quietly made your search provider harder to change.

There is a second version of this question that gets skipped and should not. Vendors describe their AI in a way that makes it sound like a capability they are adding. Sometimes it is, and you are now paying to license and train a second model that ranks products alongside the one you already pay for. Sometimes it is not, and the vendor is using the relevance signals your existing search stack already produces. Both can work. They have very different cost structures and very different failure modes, and the difference almost never shows up on a feature grid.

Every vendor in this category will tell you there is no rip and replace. That phrase describes the pitch, not the contract. 

The useful version of the requirement, and the reason your stack should decide the integration rather than the vendor's roadmap: name which systems must remain independently replaceable after go-live, and get the vendor to say in writing what happens to your data and integrations if you replace any one of them on its own.

‍

Can you show this on our own traffic, not someone else's?

Every vendor arrives with a chart where the line goes up. The question that makes that chart useful is whose traffic it was measured on.

A lift number from another retailer tells you the mechanism works somewhere. It does not tell you what it is worth on your catalogue, your query mix, your ad load, or your margin structure. 

The version worth asking for is a test against your own baseline, run on a slice of your traffic alongside your current setup, with the split and the statistical method agreed before the test starts rather than after the result comes in.

Ask which test the vendor uses and why. 

A Welch's T-test is the standard answer here, and the reason is specific. It accounts for differences in traffic volume and variance between the two groups rather than comparing raw averages, which means it holds up when your own analytics team pulls it apart. 

That last part is the point. The test is not there to convince the vendor's team. It is there to survive the meeting where your data science lead asks how the groups were balanced.

One more thing to insist on: agree what a failure looks like before the test runs. A pilot with no pre-agreed threshold is not a test, it is a demo with a longer runway.

‍

Does more relevance mean more ads, or just better ones?

This is the question your merchandising lead will ask, and it deserves a straight answer, because the honest answer is that it can mean either, and the two are separate decisions.

The first mechanism is re-ranking what you already show. 

Same slots. Same ad load. Better ordering. Because a sponsored product that genuinely fits the query now has to earn its position against the organic results around it rather than simply outbidding other ads. This is where the revenue gap between unified ranking and reserved tiles mostly sits. Ad revenue increases of 80 to 140 percent have come from existing inventory with no additional ad slots added, driven by higher click-through on more relevant placements. 

The second mechanism is coverage. 

Once ranking is relevance-led rather than bid-led, sponsored products can appear anywhere organic products appear instead of only in fixed reserved tiles, which quietly cap what is monetizable. 

That genuinely is more ads. 

It is also optional, sequenced separately, and something you should be able to turn on gradually rather than accept as a condition of the first change. Analysis of Pentaleap's H1 2025 benchmarks data, covered by Kiri Masters in Forbes, found leading retailers reaching very high sponsored coverage without a corresponding hit to the shopper experience. This suggests the ceiling is set by relevance rather than by slot count.

So the question to put to the vendor is not whether ad load goes up. It is: can we take the relevance gain first, hold ad load flat, measure conversion rate alongside ad revenue, and decide about coverage separately. 

If the answer is that the two arrive together as one change, that is worth knowing before merchandising finds out on launch day.

‍

What's reversible if this doesn't work?

Procurement's job in this evaluation is not to find the best system. It is to prevent a five-year mistake. Those are different jobs and the second one is answered by a different set of questions.

Ask what happens at each stage rather than at the end. 

A well-structured rollout has separable steps: fix ranking first, connect additional demand second, extend to other channels third. Each stage should be independently reversible, which means you can stop after step one and keep the benefit, or unwind step two without touching step one.

Then ask the contract questions that nobody enjoys asking in a first meeting. 

What is the exit path. Who owns the performance data if you leave. If the vendor's system sits between your ad server and your product grid, what does your page do on the day you turn it off. A vendor who has done this before will have an answer ready, because someone has already asked. It is also worth asking what actually keeps retailers on a vendor after the pilot ends, which is a different question from what wins the pilot.

The test worth applying across all five questions: can you undo this one piece without touching the other four. If the answer is no, you are not buying a ranking change. You are buying a stack.

‍

Three questions to take into your next vendor call

#1 Which single system makes the final call on product position on our search results page today, and would that change?

#2 If we replaced our search provider next year, what would happen to our sponsored ranking?

#3 Has anyone from our ecommerce or search team been in a vendor meeting for this category yet?

‍

Key Takeaways

• The industry has converged on ranking sponsored and organic together. It has not converged on a name or an architecture, so the label on the slide is not evidence of anything.
• Where the ranking decision runs, inside the ad server, inside the search engine, or in an independent layer, determines what you can replace later. Ask that first and everything else follows from the answer.
• Ask what signals the ranking decision can see when it runs. A system limited to bid and budget will list two or three; a genuinely combined system will list the same relevance signals driving organic results.
• Proof on someone else's traffic is a mechanism demonstration, not a forecast. Require a test on your own baseline, with the split and the statistical method agreed in advance.
• More relevance and more ads are separate decisions. Take the relevance gain first, hold ad load flat, and decide about coverage on its own timeline.
• Reversibility is not a consolation prize. Each stage of adoption should be undoable on its own, and the contract should say so rather than the pitch deck.

‍

Frequently Asked Questions

Where should the ranking decision live in a retail media tech stack? In one of three places, each with a different trade-off. Inside the ad server keeps decisioning, demand and campaign management bundled with one vendor, which is simple commercially and restrictive architecturally. Inside the search and personalization engine reduces latency but ties your ecommerce search provider to your media business, making both harder to change and real-time bidding difficult. An independent layer between the two keeps both systems in place and separately replaceable, at the cost of adding a vendor to the stack.

How do we tell a genuine combined ranking system from a rebranded one? Ask what signals the sponsored ranking decision can access at the moment it runs. A system that ranks sponsored products separately and then merges the results will answer with a short list: bid, budget, keyword match. A system that ranks everything in one decision will answer with the same relevance, margin and conversion signals that drive organic results.

Does this replace our existing search engine or ad server, like Algolia or Bloomreach? It depends entirely on the architecture, which is why the question belongs in the RFP rather than the negotiation. An independent optimization layer is designed to leave both in place and coordinate them. Other approaches require migrating one or the other. If a vendor's answer to "can we keep our search provider" involves a migration plan, that tells you which architecture you are being offered regardless of the language on the slide.

How do we validate a vendor's performance claims on our own site? Run an A/B test against your existing setup, on your own traffic, with the traffic split, success threshold and statistical method agreed before the test starts. Ask specifically which test is used and why. A Welch's T-test is the common answer because it handles unequal traffic and variance between groups, which is what makes the result defensible to your own analytics team rather than only to the vendor's.

Does improving relevance mean showing shoppers more ads? Not by default. The first change is re-ranking the sponsored products you already show, which holds ad load flat and lifts revenue through higher click-through on better-matched placements. Expanding sponsored products beyond fixed tiles into the full grid is a separate, later decision. Ask the vendor to confirm the two can be sequenced independently, and measure conversion rate alongside ad revenue throughout.

What should we be able to reverse if the results do not hold up? Every stage individually. A rollout that fixes ranking first, connects demand second and extends to other channels third should let you stop or unwind any one of those without disturbing the others. Ask for the exit path and data ownership terms in contract language during evaluation, not after selection.

‍