In 2018 I joined a newly formed experimental team with the goal of looking at some of our more opportunistic markets and seeing if we could find regional or market-specific experiences that could have a positive impact on our growth metrics for that given region 1.
We initially set our sights on Japan and one of the projects that came out of our more than two months in market was a refresh of the Netflix home page that we codenamed “Fuji” that aimed to make our service more clear and understandable in a market where video streaming services were fairly nascent.
For this project, we built a modular system that allowed us to test different value propositions in different orders throughout the page to see which ones would resonate the strongest with the Japanese market.
Here’s a before (left) and after (right) look at the Netflix home page that we designed:
One of the risks with this new design was that it was a bit more image-heavy than the current home page and we knew from previous testing that page performance can have a big impact on our growth metrics. Therefore, we spent a lot of time with engineering optimizing everything throughout the new home page and tuning our ability to measure things like time to render and time to interaction in order to be confident that the overall concept was what was resulting in any change in our metrics and not the page performance.
We also found that the order of the value propositions on the page resulted in different results across desktop and mobile, so we were able to bifurcate our experiences accordingly to get the most out of each platform.
At the end of the day, we were able to confidently measure an increase in overall sign ups and retention (which translates into more revenue) over time. These results were global and not just isolated to the Japanese market.