This scenario question evaluates whether the candidate can design a practical, disciplined A/B test for a familiar social media context, demonstrating an understanding of variable isolation, audience consistency, budget considerations, and objective decision-making.
Start by stating the goal (improve CTR) and the single variable you would test (e.g., image, headline, CTA button color). Walk the interviewer through your plan: how you split the audience, set the budget and duration, ensure other elements remain identical, and decide on a clear winning metric with a minimum difference threshold. Use hypothetical numbers to show practicality.
Start by naming the single variable you would test, and say plainly that everything else stays identical, down to the caption, the call to action, and even the landing page. Explain that you would split the audience randomly, ideally using the platform's built-in testing tool, so each creative reaches a similar mix of age, location, and device. Set a clear duration, at least several days to cover a full business cycle, and a budget that gives each version enough impressions to be statistically meaningful, not just a few hundred clicks. Then define your winning metric before the test runs, for example a CTR lift of at least fifteen percent, and commit to declaring a winner only if that threshold is met. If the results are close, say you would run a follow-up test rather than forcing a decision. In the Philippine context, acknowledge that ad performance shifts with payday weekends, holidays, and BPO shift patterns, so you would run the test through a full week to capture that variation. Keep your tone practical and concrete, and if the interviewer asks for numbers, offer a simple example like a five-day test with two thousand pesos per creative, then walk through how you would read the results.
A common mistake is testing too many variables at once, like changing both the image and the copy in a single test. The candidate might say, 'Gagawin ko na lang mas maganda yung ad, palitan ko na lahat para sigurado.' (I’ll just make the whole ad better by changing everything at once so I’m sure.) Instead, a strong answer isolates one variable, so you can clearly attribute any difference in CTR to that specific change.
Situation
When I was volunteering to manage social media for a local restaurant group, their Facebook ads had a CTR of only 0.8%, and the owner asked me to find a way to make the ads more engaging.
Task
My goal was to increase CTR by at least 20% within two weeks, without increasing the budget. I decided to run an A/B test on the ad creative because visuals typically have the biggest immediate impact on clicks.
Action
I chose to test a single variable: the main image. Version A used a static photo of the dish, while Version B used a short, looping video showing the dish being plated. I set up the campaign so each version would serve exactly the same target audience (women aged 25–40 in a 10 km radius), with the same copy and call-to-action. I allocated a total daily budget of PHP 600, split evenly between the two ads, and pre-determined that the test would run for 7 days to gather at least 1,000 impressions per ad. The winner would be declared purely on CTR, provided the difference was at least 0.3 percentage points.
Result
After 7 days, the video version achieved a CTR of 1.35% versus the static image’s 0.95%, so I halted the underperforming ad, reallocated the full budget to the video, and sustained a CTR above 1.2% for the remainder of the month.
Testing a single element with a clear metric and sufficient runtime gives you confidence to scale the winner without second-guessing.
Write your own answer, then get instant AI feedback graded against:
Get AI feedback on your answer — free.
3 free AI-graded answers + 1 free mock interview, no card needed.
Sign Up FreeAlready have an account? Log in
Sign in to join the conversation.
No answers shared yet — be the first to show how you'd approach this.