How To Evaluate Demand-Side Platform (DSP) Companies

If you’ve spent any time in digital advertising, you know how difficult it is to make sense of all the different players. There are a lot of smoke and mirrors out there. Many DSP companies claim to be the best—but the best in what and for whom?

Having personally held leadership roles in product management and product marketing in a company that built a DSP from scratch—and later sold it—I feel like I’m in a unique position to help with the evaluation process.

Whether you are a marketer, agency, publisher, or investor, this guide will provide you with an extensive evaluation framework for conducting your own due diligence of DSP companies. The framework will help you perform a comprehensive evaluation, so you can ask the right questions and choose the right DSP based on your needs.

DSPs have several categories of differentiators, each with a number of more granular factors to consider. In this guide, we break down the evaluation into 10 sections.

To skip directly to a specific section of this guide, use the following links:

• Part 1: Inventory Integrations
• Part 2: Technology Infrastructure
• Part 3: Media Buying Methods and Creative Support
• Part 4: Targeting Options
• Part 5: Reporting and Analytics
• Part 6: Brand Protection and Integrity
• Part 7: Product Usability and Workflow
• Part 8: Product Management and Strategy
• Part 9: Service, Support, and Pricing
• Part 10: Case Studies and References

Estimated reading time: 39 minutes.

That said, let’s begin with an important note about the DSP product category.

Preface: Not All DSP Companies Are Technology Companies

Before we jump into the evaluation criteria, you need to know one important point when conducting an evaluation: many companies that claim to be DSPs aren’t actually DSPs.

Newcomers to ad tech may be surprised at how many companies refer to themselves as DSPs. But ad tech veterans are wise to this practice. Some companies call themselves DSPs but have no significant technology of their own. Instead, they use another company’s DSP behind the scenes, whether manually, programmatically via application programming interface (API), or by licensing third-party technology.

In the manual scenario, these so-called DSPs rely on an ad operations team to pull the levers, so to speak, on the underlying DSP. This angle only works, however, if the company offers managed services to its customers, where the advertiser only cares about the outcome, not how it happens.

In the API scenario, a company may offer a self-service option (usually a white-labeled version of another company’s DSP) or a rudimentary dashboard built on top of another company’s technology via API. This is not necessarily a bad thing, especially if it’s embedded within another application and suited to the context. Embedding DSP functionality via API allows for the scalable creation of many small campaigns for specific use cases. However, the underlying DSP technology does not belong to the company you’re hiring.

Lastly, some companies use real DSP technology, but they license it from a technology provider as a service because building a DSP is hard work (we will cover DSP building in a future article). For example, IPONWEB and Beeswax provide custom DSP instances for their customers, who in turn sell these DSPs into market as their own.

For marketers, agencies, and publishers, this may not matter. But investors will want to ensure that they dig deep into the actual builder and operator of the technology underpinning the DSP business.

Why would a company claim to be a DSP company when it is actually a licensor? One reason is that many media buyers prefer to work directly with the technology provider to reduce costs. Another reason is that DSP companies with their own technology are valued at a higher rate. They might even be licensing the DSP technology as a stopgap measure until they build out their own, but more on that later.

Bottom line? Beware of agencies or ad networks disguised as programmatic tech companies. Advertisers risk paying both service fees and tech fees, when only a service fee is necessary, while investors risk overpaying for a business. In both cases, this guide will help you determine whether you are looking at a genuine DSP or an intermediary service company.

With that preamble out of the way, let’s jump into the first evaluation category.

Part 1: Inventory Integrations

A DSP’s primary value proposition is its aggregated audience reach. The quantity and selection of inventory integrations largely determines the size of its reach.

(Click image to enlarge)

Often it’s assumed that all DSPs have access to the same inventory. But this is only true in a general sense. In reality, there are very real differences in the supply each DSP has access to.

The main cause for these differences is channel focus. Some DSPs focus purely on desktop display, while others go deeper on mobile inventory integrations, and still others on native supply, video supply, and so on. Since it costs money to listen to impressions, DSPs will generally focus on one particular channel more than others.

Another area to examine is the available channel environments. How you judge this depends on your target audience, your desired ad format, and in which environments you wish to reach them (e.g., desktop video versus mobile video, or mobile web versus mobile apps).

Audience reach is also affected by the DSP’s global footprint. In other words, in what countries is inventory available? To reach a country’s population requires a certain volume of inventory from that country. As a result, reaching a desired country often requires that the DSP have servers in that country or region. Given the nature of real-time auctions, DSPs must return bids within the limited timeout window or they won’t win the volume of impressions necessary to reach an audience. For example, having a server in Amsterdam is often a prerequisite to supporting programmatic buyers in Europe.

Request a list of countries to which customers have access—and the approximate number of impressions in each—to ensure you can get the reach you desire.

It’s also important to understand how many of their supply integrations are direct with exchanges and how many are through aggregators like BidSwitch. Added intermediaries increase costs, which may put you at a disadvantage against advertisers on other DSPs that have lower costs.

Potential questions to ask during an evaluation:

• Which supply sources are you integrated with?
• Can you break down what those sources are, such as desktop, mobile, video, and native?
• How many of those integrations are direct, and how many are through aggregators like BidSwitch?
• How many daily/monthly desktop impressions do you have access to in total?
• How many daily/monthly mobile impressions do you have access to in total?
• How many daily/monthly video impressions do you have access to in total?
• How many daily/monthly native impressions do you have access to in total?
• For mobile, what is the split between in-app inventory and mobile web for each source?
• Which supply sources allow mobile rich media? Mobile video? Native?
• What is the percentage of inventory that has GPS information attached to it?
• Do you have any exclusive inventory sources? Any non-RTB sources?

Part 2: Technology Infrastructure

There are a few factors to consider with regard to technology infrastructure: sufficient capacity and scale, technical caliber, and the availability of an API. Let’s look at capacity first.

Ensuring Capacity and Scale

The larger the advertiser, the more important it is for the DSP to have the capacity and scale to support its buying requirements. One of the key indicators of capacity is queries per second, or QPS. For example, Forrester’s DSP evaluation criteria, in the past, required vendors to support a minimum of at least 1 million QPS because its clients are Fortune 500 brands that require such large capacity.

The ability to process at a high QPS rate comes at a real cost to DSPs, mainly in the form of servers and other infrastructure expenses. That’s why DSPs need to be strategic about how they scale their capacity to handle a higher QPS rate.

What this comes down to is accurately defining your needs. Judging DSPs purely on raw QPS is ineffective. What ultimately matters is whether the bid requests seen by the DSP are relevant to its advertiser’s campaigns. The Forrester criteria make sense because of its reports’ target market, but it should not be taken as a universal guideline.

For example, MediaMath serves the global Fortune 500 market segment—a diverse and global segment. It’s important for MediaMath, then, to have a high absolute QPS capacity to give it the broadest reach possible. But its infrastructure costs will also be very high, especially with the rise of header bidding. Even so, MediaMath is likely smart about the inventory it chooses to look at. It would be wasteful to spend precious server resources looking at Albanian inventory if it has no advertisers or campaigns targeting Albania.

On the other hand, suppose there was a niche DSP that catered specifically to the Mexican market. Its absolute QPS capacity would be irrelevant, especially if you were comparing it to MediaMath’s QPS capacity. What matters to the niche DSP and its customers is relevant Mexican inventory, which could be higher than competitors’, even though its absolute QPS is lower.

Do you need a global DSP like MediaMath? That depends on whether you’re trying to reach the broadest possible audience. If you’re looking for a more niche audience, then you have more DSP options available.

Technical Caliber

Another way to determine whether the company offers its own DSP is to dig deeper into its technology roots. This is important to understand because you don’t want an intermediary when it comes to knowledge and support. Understanding the technology offering and how it works is not something you want a middleman mediating. The information you receive might ultimately be the same, but there are subtle differences that add up in the long run.

For example, ask whether the DSP was written from scratch in house and if the company owns the code. If so, have they leveraged any open source bidding frameworks like RTBkit or Google’s Open Bidder, or was everything written from the ground up? If not, have they simply built a reporting dashboard on top of a DSP’s API? Or have they licensed a custom bidder from IPONWEB or Beeswax?

If the answer to these last two questions is “yes,” then you are dealing with a service company or intermediary, not the underlying DSP vendor.

Asking where the servers are located and how many it has will also tell you a great deal about the caliber of a DSP. For example, robust DSPs have servers located in multiple data centers. In the United States, this often means having servers on both the East and West Coasts, usually in New Jersey and California, respectively. Amsterdam is a common location for European data centers, while Singapore is a popular hub for the Asia-Pacific region.

Asking how many developers are employed is another way to learn more about the company’s commitment to actually building technology. You can verify this number by researching the company on LinkedIn. More doesn’t necessarily mean better, but a true DSP company will need enough engineers to keep the systems operational and the product moving forward. That could be anywhere from a dozen to a few dozen at minimum, depending on the company’s maturity.

Accessing A Robust API

The availability and robustness of an API is another factor to consider. Oftentimes, advertisers or agencies will want to pull reports directly into their own custom data warehouses. A strong API with good support goes a long way towards accommodating savvier users. Especially those that want to build custom dashboards for their users. The Trade Desk, for example, is known for having an excellent API, which is well documented and developer friendly.

Conversely, service companies posing as DSPs will almost never have their own APIs. If they claim to have one, ask them to provide you with the documentation. Genuine DSPs are never built on the API of another DSP, whereas faux DSPs are either built on the APIs of another company’s DSP, or simply operating a real DSP behind the scenes using people. This is why the API question is important.

In a similar vein, asking the DSP whether it’s possible to white-label their user interface is another great filtering question. Only a real DSP will offer the ability to white-label their interface with another brand. Companies posing as DSPs cannot offer the same ability.

Potential questions to ask during an evaluation:

• Is your DSP homegrown, proprietary code, or is it licensed from IPONWEB, AppNexus, or Beeswax?
• Is your DSP built on top of another DSP’s API, such as The Trade Desk’s?
• If proprietary, do you use cloud hosting like AWS or dedicated data centers?
• If you host in data centers, where they located? How many servers do you have in total?
• Are you leveraging any open source bidding frameworks? Or is the code written from scratch?
• What is the average latency of your bid requests? Does it differ by supply source?
• What is your current average daily QPS?
• What is your maximum QPS limit?
• What percentage of inventory has been throttled? What countries have been opened up?
• Do you have an open API that advertisers can build on top of?
• Do you have any example customers that have built their businesses on your API?
• How many developers do you have? What percentage are they out of all employees?
• Is it possible to white-label the technology (interface) with a different brand?

Part 3: Media Buying Methods and Creative Support

The availability of different media buying methods and creative support is another major category of consideration. “Media buying methods” refers to the options available for procuring inventory. In other words, can inventory be purchased in a real-time fashion (via RTB)? What about guaranteed direct deals that allow advertisers to reserve inventory in advance (i.e., futures-based buying)? Or are the buys “blind,” as is common among ad networks?

Each of these capabilities require different technology components and in some cases, such as direct or blind buys, additional people in ad operations roles.

The answers to these questions will also tell you whether you are looking at a real DSP, a service company (i.e., a faux DSP), or both.

For example, blind buys can indicate that the company manually operates the ads on another company’s DSP. In blind buys, campaigns are executed in a fully managed but opaque fashion. You have no visibility into how the campaign was executed, and any reporting you receive is generally in Excel spreadsheets, which lack integrity.

Pricing Options

Pricing is a component of buying methods. Different advertisers have different campaign objectives, so a DSP that offers a variety of pricing options enables you to choose the one that best aligns with your campaign goals. (Note: This is different from the pricing of the DSP itself, which we will cover in “Platform Pricing.”)

For advertisers with brand awareness objectives, the CPM pricing option aligns well, because it’s best suited to ensure reach and delivery. It also aligns well with publishers, because it’s the least risky in terms of performance; the publisher merely has to deliver the impressions.

If you have more sophisticated direct-response campaigns and perhaps specific cost-per-click (CPC) targets, a CPC buying option is valuable. It transfers some of the risk to the DSPs, because now the pricing depends on ads receiving a certain click-through rate (CTR). Keep in mind, though, that every transaction in the RTB ecosystem is fundamentally transacted on a CPM basis. If a DSP is offering CPC as a pricing model, it is taking a risk to do so.

(Caution: Some DSPs may mitigate their risk by relaxing their fraud-detection standards in order to hit your goals. That’s why extra vigilance should be exercised in these situations.)

It’s important to note that this is different from optimizing towards a CPC goal. For CPC pricing, a DSP must have a sophisticated algorithm to determine whether any given campaign’s CPC bids are high enough, and the ad’s CTR is high enough, to continue receiving impressions.

But let’s say you want the highly desirable cost-per-acquisition (CPA) or cost-per-install (CPI) options. These options are offered by extremely confident DSPs because they essentially transfer all the performance risk to the DSP. It can be a win-win scenario if the CPA/CPI goals are achieved, which is more likely if your acquisition funnel is already well optimized and your success metrics determined. However, this basically turns DSPs into ad networks or performance-based affiliates, which comes with real business risks for them.

Again, this is very different from optimizing towards a CPA or CPI goal, which almost all DSPs can and should do. In contrast, for CPC or CPA pricing models, advertisers only pay if there are clicks or conversions.

Creative Formats

The types of creative formats supported by the DSP are another important differentiating factor. Your format requirements will vary, but static images (or banners) are standard. Other formats you may desire include rich media ad units, interstitials, video, native, and audio. Video support is in high demand and growing quickly, but video ad serving has unique technical requirements, especially on mobile.

In some cases, you may need creative development services, especially with video. Some DSPs offer creative development of static, rich media, dynamic, and video ad units. Others also offer post-click creative development (i.e., mobile landing pages), in cases where the advertiser may not have a mobile-friendly website.

Potential questions to ask during an evaluation:

• What media buying models do you support: CPM, CPC, CPA, or CPI?
• Are they actually pricing models, or do you simply optimize towards them?
• Are you able to establish and support private marketplace deals?
• Do you support direct (guaranteed) buys?
• What creative formats do you support: rich media, native, video, audio, others?
• Do you support mobile rich media?
• If so, which providers do you support: Celtra, DoubleClick Studio?
• Do you offer creative development services?

Part 4: Targeting Options

A fundamental criterion for evaluating DSPs is the sophistication of available targeting. But before asking more advanced targeting questions, make sure the DSP being evaluated has most of the standard targeting capabilities, including the ability to target:

• Specific websites
• Specific mobile apps
• Audiences using first- or third-party data
• Desktop operating system (e.g., Windows, Apple, Linux)
• Mobile operating system (e.g., iOS, Android, Windows Phone)
• Specific OS version (e.g., iOS 6.0+, Android 4.1+)
• Device type (e.g., feature phone, smartphone, tablet, laptop, gaming console)
• Device model (e.g., iPhone 3G vs. iPhone 5S, or Samsung Note 3 vs. Samsung Note 4)
• Connection type (e.g., Wi-Fi, specific carrier, connection speed)
• Location (e.g., country, city, postal/zip code, street, geo-fence (GPS + radius))
• Language (e.g., handset language, browser language)

Factors For One-to-One Targeting

Are you aiming for an extremely broad reach in your campaigns (one side of the spectrum) or specific targeting (one-to-one) that allows for highly personalized messaging? Know that the latter requires much deeper data capabilities.

If specific targeting is important, investigate the availability of data segments for audience targeting. Which data sources are accessible via API integrations:

• Proprietary publisher data
• First-party advertiser data from data management platforms (DMPs)
• Third-party data from DMPs
• Device-ID lists
• Third-party location data

How many of those data sources overlap with the your target audience? Can look-a-like audiences be generated based on first-party (i.e., your) data or customer relationship management (CRM) data? (This doesn’t necessarily need to be provided by your DSP. Many DMPs offer this functionality in a more portable manner, so if you go the DMP route you can remove this as a deciding factor for your DSP.) How much of that data is based on cookies? Device ID? Cross-device ID?

As for audience segmentation, DSPs offer varying levels, from the most basic of audience segments to real-time targeting of dynamic audience segments to personalized and even predictive targeting.

Identity Management

That brings us to the final component of targeting: identity management.

In mobile environments, there are many ways of managing and determining identity and targeting, from basic to highly sophisticated. The first question to ask is how important cross-device identity management is to your campaigns. Then ask what ability the platform has to identify mobile users consistently. Is it based on unique login data, a proprietary algorithm, a third-party algorithm, device-ID lists, or another mechanism? The method used will have an impact on both its accuracy and reach.

If mobile doesn’t matter to you, then don’t worry about cross-device targeting or measurement. However, according to the latest IAB reports, you might want to think twice about caring about the increasing importance of mobile.

Potential questions to ask during an evaluation:

• What targeting parameters are available for each campaign type?
• How granular is the geographic targeting? Is hyperlocal (GPS) targeting supported?
• Which third-party data sources are you integrated with?
• In which countries can this data be used?
• What is the cost associated with using third-party data?
• Are the costs passed through, or do you mark it up?
• What DMP functionality do you have? Can your DMP plug in to other DSPs?
• What are the audience segmentation options available?
• What is the cost associated with using your DMP on a self-serve basis?
• Can you onboard our first-party data for us?
• If so, what kind of data: email addresses, list of mobile IDs, other?
• Can you create look-a-like audiences for us?
• Do you have your own exclusive data segments available in your platform?
• Do you support cross-channel targeting and attribution?
• If so, how does it work? How accurate is it?

Part 5: Reporting and Analytics

Reporting and analytics are of major importance to all advertisers, but especially so for particular types of marketers. For example, a performance ad agency with sophisticated campaigns might need a deeper level of granularity when it comes to its campaigns.

An advertiser’s objectives also impact the kind of reporting needed. For example, brand advertisers who have awareness objectives may be more interested in reach and frequency reporting, which outlines how many unique users have been hit in a given campaign.

It’s also important to remember that there are many different areas of reporting and analytics. They span the planning phase, where volume and reach estimates are pertinent, all the way to the campaign performance phase, where they help with midflight optimization. Reports related to billing and enriched post-campaign insights are other areas as well.

With respect to planning, you may want reach estimates for cookie or device-ID pools and impression forecasting. These estimates show how many people could be reached given specific targeting parameters (e.g., site lists, DMAs, retargeting pools).

Regarding campaign performance, ask about inventory and site-level reporting transparency. Such transparency helps with troubleshooting and optimizing campaigns, especially when being manually optimized (see next section for more details).

When scaling campaigns, you may find it helpful to view audience look-a-like analytics, which show audiences that are similar to existing converters or other insights that suggest ways to improve or scale your performance.

Ultimately, to determine what level of reporting you need, figure out what reporting you need to either verify return on investment (ROI) or please clients.

Once you understand your needs, it’s important to evaluate the reporting interface itself. Does the user interface include the ability to run advanced analytics reports (i.e., custom queries) or to schedule reports?

You’ll want to know about the platform’s run time, or responsiveness, as well. Does it take a few seconds to run a report, or does it take minutes or hours? How about supported file sizes? All of these questions uncover a given DSP’s robustness—or lack thereof.

Data freshness in the reporting interface is another factor to examine. Is there any delay in report accuracy? If so, is it 24 hours? 12 hours? 3 hours? 3 minutes? 15 seconds? Typically, the more hands-on you are, the more you will desire real-time reporting rather than delayed reports.

If you have in-house reporting technology and want to integrate programmatically with other platforms, you’ll also want your DSP to be able to integrate with your in-house technology via API.

Potential questions to ask during an evaluation:

• What levels of campaign reporting are available: campaign, placement, creative, daily, hourly?
• What dimensions of reporting are available: location, device, audience, contextual, domain, frequency?
• Are there any qualitative insights or special data analysis capabilities?
• What is the delay on generated reports? How long do they take to generate?
• What is the delay of stats in the user interface (UI)? How often is it updated?
• Can reports be exported from the system in bulk?
• What brand study or engagement measurement integrations do you have?
• Which third-party measurement partners do you support? How many support mobile?
• Is it possible to white-label reports from your interface?
• Do you have a reporting API?

Part 6: Brand Protection and Integrity

When it comes to protecting your brand from the risks of programmatic advertising, there are a few key areas to consider: ad fraud, contextual brand safety, and data privacy.

Ad Fraud

Dealing with ad fraud is an unfortunate reality of programmatic advertising—at least for now. That’s why it’s important to work with a DSP that takes a proactive role in protecting you from the risks of ad fraud through monitoring and blocking low-quality inventory.

If fraudulent impressions are proliferating through exchanges, the DSP is responsible for combatting the fraud. While many are up to the challenge, it’s a never-ending game of whack-a-mole. There are literally hundreds of thousands of sites that come through the exchanges. Checking them all by hand is virtually impossible.

There are, however, measures DSPs can take. At my previous company, our DSP used a combination of human checks and technical measures to monitor and block fraud on our platform. In addition, inventory on our platform was monitored using third-party fraud-detection technology. This produced rich reporting on questionable traffic that we blocked from future campaigns.

Unfortunately, not all DSPs go to these lengths to fight fraud, which passes the problem along to marketers. And without the support of their DSP partners, marketers will have a tougher time fighting fraud.

Contextual Brand Safety

Contextual brand safety is the practice of ensuring that ads show up only on sites deemed appropriate by brand standards. Naturally, the tolerance for what is appropriate depends largely on the brand itself. There are generally a few ways to ensure brand safety: third-party contextual brand-safety vendors, blacklists and whitelists, and direct deals with select publishers. Let’s focus on the first two.

Most DSPs have direct integrations with third-party contextual vendors, like Proximic (acquired by comScore), Grapeshot, or Peer39 (acquired by Sizmek). These contextual analysis tools scan the contents of every page that your ad has a chance of being displayed on and ensure that the content doesn’t include any subject matter that may be deemed inappropriate for the brand in question.

Mind you, some brands may have a very specific standard of protection that requires custom brand-safety segments. If so, can the DSP work with their providers to enable that?

Most DSPs also support the creation of domain blacklists, which allow you to curate a list of undesirable sites to exclude from all current and future campaigns. The opposite approach is to use whitelists, which allow ads to be shown only on the listed sites. Neither approach is perfect, but it’s nonetheless important to verify that this functionality exists.

Data Privacy

Lastly, it’s important to understand how the DSP uses your data in all its forms. For example, your actual campaign performance data could be extremely valuable to a competitor. Would you feel comfortable with an account manager sharing this data with another customer, even if they didn’t reveal that it was your campaign data?

The same concern applies to audience data. The majority of DSPs allow the collection of first-party audience data—mostly used for retargeting—by letting advertisers place a piece of JavaScript code on their websites. Do they know how this data is used? Is it shared, repackaged, or resold with anyone else on or off the platform?

To be extra vigilant, keep your eyes open for related clauses in the terms of service (ToS) or master service agreement (MSA), especially before accepting the terms or signing the agreement.

Potential questions to ask during an evaluation:

• What processes and tools do you have in place for protecting against invalid impressions and clicks?
• What ad verification companies is your platform integrated with?
• What contextual brand-safety integrations does your platform have built in?
• Does your platform support blacklists for exclusion and whitelists for targeting?
• What processes do you have in place to ensure brand safety for users?
• What processes do you have in place to protect first-party data?
• Do you share advertiser campaign data and performance with other advertisers?
• Do you aggregate campaign or audience data and sell it in any way to anybody?

Part 7: Product Usability and Workflow

A platform’s usability and workflow are an extremely important part of the evaluation process. That’s because the platform’s user friendliness and design affect the daily operations of the actual end users. It’s hard to quantify, so how adaptable the interface is to the workflow and processes of users—especially as it relates to campaign management—is key.

Usability

On one end of the usability spectrum is a product that is intuitive and easy to use for people at all levels of sophistication; on the other end is a complex product with a steep learning curve and that seems to have no rhyme or reason in how it was designed.

When a DSP is intuitive, onboarding and training for new users becomes that much easier. It also makes a difference as you experiment with various buying strategies, tactics, and new features down the road.

Having good usability is also extremely important in helping to minimize mistakes. Programmatic media buying is very powerful, and mistakes or misunderstandings—whether in the campaign setup or optimization process—can be costly.

Unfortunately, the only way to truly evaluate usability is by testing the DSP out for a period and perhaps comparing it with other platforms. Alternatively, you can base your evaluation on other people’s reviews, which we will cover in ”Case Studies and References.”

Workflow

Creating and managing campaigns is a DSP user’s core task. This includes the ability to upload creative assets to campaigns and define targeting parameters in a self-serve manner. Platforms that are best in class have features like batch-upload functionality, which saves a great deal of time.

Other campaign management features include solutions for managing a multitude of pixels (for audience targeting, conversions, etc.) and site lists (blacklists for exclusion, whitelists for inclusion). Having batch-upload and export functionality in these areas makes life easier for end users, especially agencies.

Oftentimes, people who handle daily campaign operations need a user-friendly management interface to keep track of campaigns. One of the most common tasks is ensuring that campaigns are properly pacing based on the flight dates and pre-defined budgets. Anything a platform can do to assist in making that task easier, whether it’s alerts or helpful visualizations, goes a long way in making the user’s job easier.

Examples of such alerts include when a creative asset is denied for quality purposes or when delivery drops off and campaigns risk under-pacing. Features like dashboards, real-time reporting, and a responsive interface all help improve the campaign manager’s daily life.

Also consider the available optimization options. Is manual campaign optimization an option? In other words, can you log in and perform manual campaign optimizations by blocking underperforming placements or increasing bids and ad spend on high-performing placements?

What about automated optimization algorithms? Can the platform automatically manage bid prices based on a multitude of variables (e.g., location, time, frequency, behavior)? Does it also manage the placement-level optimizations that a human operator would make, all to achieve specific campaign goals?

With respect to transparency, ask your DSP how it develops some of its algorithms. Some DSPs claim to have superhuman algorithms that are apparently unfathomable to average minds. Such claims are often used to justify the secrecy behind the methods, which many refer to as “black boxes.” Be skeptical of this type of avoidance. A real DSP should tell you, at least in broad strokes, how its algorithms work and what variables it takes into account when making optimization decisions on your behalf.

With respect to analytical insights, certain software suites, like Google’s DoubleClick stack, also have additional timesaving features, where platforms automatically share data with each other, allowing for richer reporting and insights. In Google’s case, DoubleClick Campaign Manager syncs with DoubleClick Bid Manager.

The answers to all of these questions determine how friendly a DSP is to its users. Programmatic buyers and campaign managers typically have significant workloads. If a DSP does not have adequate usability and laborsaving features, it may not be a realistic option for self-service use.

Potential questions to ask during an evaluation:

• Does the DSP have a self-serve interface, or is it fully managed?
• Do you support custom frequency capping?
• Do you support split testing of creative assets?
• At which campaign levels can you implement budget caps and pacing?
• Can you implement both lifetime and daily budget caps?
• Do you support evenly paced budget delivery?
• Are there any pacing visualization features to help end users?
• Are there any batch-upload or batch-update features?
• Do you have a creative asset management system?
• Is bid optimization done manually or automatically via algorithm?
• Is media optimization done manually or automatically via algorithm?
• How do the bidding and optimization algorithms work?
• What parameters are evaluated for each impression considered for purchase?
• Are creative concepts and sizes part of the optimization equation?
• Are there different kinds of optimization algorithms to choose from?
• How many conversion events do your algorithms need before they start optimizing?
• Can advertisers define their own custom optimization algorithm on the platform?
• Do you have an API for campaign management?

Part 8: Product Management and Strategy

One of the best ways to gauge the quality of a DSP is to ask detailed questions about its technology organization, specifically the product management strategy. By getting a handle on the company’s ultimate vision and its strategy for achieving it, you will gain a better understanding for why it prioritizes some features or integration efforts over others.

A large part of that strategy involves learning about its intended target market. Is it tailoring its platform more for marketers or agencies? Is it targeting enterprise customers, midmarket customers, or smaller customers?

Then there is the question of user sophistication. Is the DSP targeting more sophisticated customers (“power users”) and aiming to give them as much transparency and controls as possible? Or is it building an easier-to-use product for the less-sophisticated market?

Once you understand the target market and sophistication levels, you can dive into roadmap questions. The most important timeframe to understanding is the company’s roadmap for the next 90 days. What features and integrations does it have planned and why? From there, feel free to probe about longer-term product vision—six months to a year out.

Note that in technology and product management, long-term roadmaps should be taken with a grain of salt. Product teams can predict their roadmaps with a higher degree of certainty for about 90 days into the future. Yet even with a project management organization, predictions tend to get fuzzy after that. There are so many moving parts to technology businesses, not to mention the market itself, that long-term plans should be considered visions rather than promises or commitments.

You also want to look for conflicts of interest, in cases where a DSP offers products or solutions for publishers, like an exchange or supply-side platform (SSP) arm. Such conflicting products throw into question a company’s true allegiance. There is a natural tension between advertisers’ goals and publishers’ goals. Generally speaking, advertisers want lower media costs (more bang for their buck), while publishers want to maximize media costs. When push comes to shove, who is the DSP most aligned with?

If you find a concerning lack of clarity about the DSP’s overall vision, roadmap, target market, and alignment, you may discover the unflattering truth that it’s not a real DSP at all but rather an imitator.

On the other hand, if it appears that the DSP is an agile technology company that has a carefully considered roadmap, especially one that matches your own needs and builds no other conflicting products, you are on probably on the right track. Having a true DSP vendor that is open to feedback and input into the product roadmap is indeed a valuable partner.

Potential questions to ask during an evaluation:

• How is technology development integrated into your organization?
• Who is your intended target market: marketers, agencies, or publishers?
• What level of user sophistication are you targeting: power users, novices, others?
• What does your team see as the next key areas for your product development?
• What updates do you have in your product roadmap for the next 3, 6, or 12 months?
• Please explain the rationale for your product roadmap prioritization.
• To what extent do your customers have influence over the product roadmap?
• How do you develop and refine your algorithms?
• Do you have an exchange or SSP arm?
• Is your business model aligned more with media buyers or sellers?
• How are these differing allegiances addressed internally?
• Do you have any other technology products outside of your DSP?

Part 9: Service, Support, and Pricing

Advertisers often overlook the need for services and support. Sometimes advertisers underestimate the amount of service, training, and support they require, especially if they are brand marketers taking programmatic ad operations in house.

Service and Support

For a variety of reasons, advertisers require different levels of support when it comes to using DSPs. That’s why it’s important to understand how much of a service layer is available.

If no service layer is available, then the platform is purely self-service, in which case there ought to be a modicum of customer training and support available to help with the software’s learning curve. Some DSPs charge for this initial training (a.k.a., onboarding or implementation) under a professional services fee, while others don’t.

If managed services are available, are they free, or is there an extra charge? If there’s a fee, what is it? Typical fees for managed services range from 10 percent to 15 percent of the ad spend, in addition to the tech fees for the platform itself.

Outside of campaign management services, it’s helpful to understand the robustness of the support, in case users have questions or need help troubleshooting campaigns. Is support only available during specific business hours or is it 24/7? If it’s during business hours, in which time zone are those hours? Is support available by phone, email, or both? Are there any service-level agreements (SLAs) around the responsiveness of support, interface availability, pixel and ad server uptime, and so on?

There is also the question of training and educational materials. Oftentimes, these are made available through a knowledge base. Users can often find specific resources on topics such as campaign management, tracking tokens, or pulling custom reports. Having such a self-service resource center is a major benefit to actual users.

Platform Pricing

Pricing structure and terms of use are also critical to assess, since they differ by vendor, and factors like fees, minimums, and contractual commitments are hard constraints for many advertisers. For example, one DSP vendor might offer an escape clause for the first 90 days, while another DSP might not offer one at all. Vendors often offer more flexibility to high-value advertisers, like Fortune 1000 brands or holding company agencies, for obvious reasons.

Generally speaking, DSP fees can be anywhere from 20 percent of media spends all the way down to single digits. The largest determining factor is the ad spend commitment. Smaller advertisers that have a monthly media budget only in the four-figure range may have little negotiation leverage with the DSP, so the tech fee is likely to be closer to 20 percent. However, larger advertisers, especially agencies with monthly media budgets in the millions, have all the leverage necessary to secure single-digit–percentage fees from DSP vendors.

Potential questions to ask during an evaluation:

• What is the fee structure for self-service model and for the managed service model?
• Is it a percentage or flat CPM? What is the cost?
• Are there any minimum spend levels, spend commitments, or tiered fee structures?
• What levels of support do you provide? Do you provide access to an account manager?
• Do you assist in setting up campaigns, if needed?
• What training options do you provide for your platform?
• What is the level of technical assistance you provide?
• Is there cost associated with support? If yes, how much?
• How many account managers do you have in total?
• What is your system uptime? Any scheduled downtime occurrences?
• What is your onboarding or implementation process for new customers?
• Are there any media delivery fees or ad serving costs?
• Are there any hidden fees? (e.g., third-party data markup, costs for specific tactics, etc.)
• Explain how one dollar spent through your DSP breaks down in terms of media costs vs. fees.

Part 10: Case Studies and References

In addition to all the information you gather about a company, its product, and its services, you want to gather as much proof as possible about the experiences others have had with it. That way you can verify the DSP’s track record and product claims.

In general, there are three sources for such information:

• The DSP vendor
• Its customers
• Third-party sources

With respect to the DSPs themselves, case studies are the main proof you want. In other words, you want real-life examples of how customers were able to meet their marketing objectives by using the platform. Obviously, the more relevant those case studies are to your business, the better. If you represent a college or university, seeing the success of other learning institutions is much more comforting than seeing the success of an automotive brand.

The most useful source of information comes from actual customer reviews, which can come in different forms. The first would be customer references provided by the DSP. As you might imagine, there are some credibility issues with this approach, because the references are obviously handpicked friendly customers. However, speaking to existing customers and asking them about their specific experiences is still a prudent step.

Vendor-neutral websites like G2 Crowd are another source of customer reviews. (Fun fact: G2 is a military term for intelligence.) These crowd-sourced review platforms offer a tremendous amount of intelligence to anyone interested, and users are verified first. For example, G2 Crowd uses LinkedIn to verify the identity of users before they can leave reviews. This process adds credibility and prevents malicious reviews.

The reviews themselves consist of the type of questions a product management team might ask during customer research:

• What do you like best about the product?
• What do you dislike about the product?
• What would you recommend to others using the product?
• What business problems does the product solve for you?
• What benefits have you realized from the product?

When I worked in product marketing, third-party review sites were a goldmine of competitive intelligence. I think it will prove just as valuable to you whether you are a marketer, agency, publisher, or investor looking to evaluate DSPs (or other technology products).

Lastly, you might want to dig into reports conducted by third-party research firms, like Forrester or Gartner. Although the quality, comprehensiveness, and objectivity of their reports are questionable, that doesn’t make them entirely worthless. It just means that you need to consider a couple of things when weighing the value of these research reports:

• Who is their target customer? Forrester, for example, serves major business-to-consumer brands, meaning it prefers to include vendors that cater to its target customers. This is why you often see many DSPs missing from such reports.
• Are they completely objective? Research firms don’t just cater to brand marketers but to ad tech companies as well. It stands to reason that being a client could conceivably influence the inclusion and ranking of specific companies in the report, despite what integrity policies might claim.

In the end, knowledge is power, and the more data you have the better. Even if some of those data sources may be questionable, it helps to look at the data as a whole and then make your own conclusions.

Potential questions to ask during an evaluation:

• Have you supported other advertisers with similar needs to ours? If so, how?
• Can you provide case studies highlighting success stories on your platform?
• If so, please provide a wide range of objectives (brand lift, direct response, etc.)
• Also, please provide a variety of industries and verticals.
• Can you provide three relevant customer references that we could call?
• Are there any third-party reports about your platform, such as from Forrester or Gartner?

Closing Thoughts

At this point you might be thinking that this evaluation process seems like a lot of work. And you would be right. Conducting proper due diligence of a DSP company can be a long and arduous process.

Sometimes it might be neither realistic nor necessary to go through such a rigorous evaluation. In such cases, you may just want some recommendations for the best DSP for your needs. Oftentimes, people forego the formal evaluation process and go with a recommendation from an industry colleague, which is a fair approach.

On the other hand, if you are a larger organization with a formal procurement process or you are embarking on an earnest evaluation of DSPs, this guide will help you ask the right questions and choose the right DSP based on your needs.

~

 

Last updated: July 10, 2017