On the benefits of tests: how little things affect the players’ reaction
We’ve been conducting tests every week for many years and we can say that it’s almost impossible to understand what players will like right away if you rely exclusively on your gut feeling. There are some exceptions, but it’s pure luck. Our way is to test theories on a real audience as early as possible.
I’ll give you a few examples of how the most seemingly insignificant changes in the game affect the metrics and allow development studios to accumulate their own expertise for future projects.
UI and visuals
We experimented a lot with Hidden Object games. With Diesel Puppet studio alone, we released five projects, testing different hypotheses, settings, and mechanics (spoiler alert: it turned out well in the end). There’s a separate article about this, but it barely mentions the UI.
I’ll show a small example of how the interface redesign affects the metrics:
As a result:
- R1 increased by more than 3%, R3 increased by 5%+, R7 — almost by 2%.
- Playtime increased by 20-50% in the first week.
- Interstitial views increased by 4-5% in the first week.
- Views of rewarded ads increased by 8% on average.
The background
We tried to use artwork instead of a monochromatic background which is seemingly not supposed to distract users from the gameplay.
- Retention from Day 1 to Day 7 increased by 30% on average.
- Day 1 playtime increased by 76%.
- Interstitial views increased by 40%.
- And players started watching rewarded ads 70-150% more often in the first week.
The scale
In Crowd Sort, we decided to play with the number of figures in the flasks. It’s not an obvious thing, but we conducted dozens of tests and found this solution:
Retention and playtime haven’t increased much, only by a couple of percent, but the number of views for various ads increased by 15-20%.
Final thoughts
In a competitive market, even small things can’t be ignored. You usually don’t think about stuff like that right away, but if you rely on analytics and pay attention to feedback and metrics, success becomes a matter of time, work and experience accumulation. To achieve this, we regularly prepare detailed presentations for developers, but they can also see it themselves any time they want — our partners always have access to our analytics and tests.