Finding games on Steam has always been difficult, but the flood of titles being published on the platform has made discoverability a massive problem for creators and gamers alike. Valve is attempting to improve the situation with a new game recommendation system that relies on AI, rather than user-curated metadata. The AI isn’t given any information about a game at all, rather than its release date, and it isn’t impacted by review scores or tags. Instead, it learns about games solely based on what players do.
Underlying this new recommender is a neural-network model that is trained to recommend games based on a user’s playtime history, along with other salient data. We train the model based on data from many millions of Steam users and many billions of play sessions, giving us robust results that capture the nuances of different play patterns and covers our catalog. The model is parameterized so that we can restrict output to games released within a specified time-window, and can be adjusted to prefer games a higher or lower underlying popularity. These parameters are exposed to the user, allowing you to select whether to see only recent releases in the results, or go all the way back to include games released a decade ago. Similarly, you can choose whether to see mainstream hits, or deep cuts from the catalog. Regardless of the settings of the sliders, the results will always be personalized and relevant to the individual user.
Valve’s ability to create recommendation engines, search tools, and discoverability enhancements that treat players and creators fairly has taken some hits over the past year. During the recent “Grand Prix” summer sale, Valve created a confusing competition around the event. As a result, gamers began removing indie titles from their wish lists en masse in an attempt to game the system and receive expensive AAA titles for free, out of a mistaken belief that deleting low-cost titles increased their chances of receiving expensive free games. Creators panicked. Having a game on your wish list means receiving updates when it goes on sale, and those notifications apparently have a major impact on how many copies a game sells overall. Developers are exceedingly unhappy with how the Steam sale played out this year, and many saw no uptick at all in either wish list pickups or overall sales.
The new game recommendation system Valve is unveiling today isn’t a direct response to the Grand Prix debacle, but it’s an example of how the company is attempting to improve its own recommendation algorithms in ways that will be fair to game creators and help players find titles they want to experience. To date, Valve’s changes to its recommendation system have been controversial; an algorithm bug last year drove far more traffic to already-established titles at the expense of smaller titles. After the Grand Prix problems, some developers have lost faith that Valve is particularly interested in solving the situation.
“Before October 2018 (and for a few months after that while I gave Steam the benefit of the doubt), I told anyone who asked me that Steam was 100% worth it for indie developers,” developer Yitz told Kotaku via Twitter DM:
Now, that trust is gone, and it’s not because I’ve changed or become more cynical… This Steam sale was a disaster, but I’m far more concerned about the overall trend we’ve seen in the Steam algorithm since October last year: pushing unpopular (including ‘mostly negative’ reviewed) triple-A games over titles that Steam has more than enough data to know would be a better match for the consumer.
I decided to take the Steam recommendation algorithm out for a spin, to see what kind of titles it would recommend for me, personally. In my specific case, I’m prepared for the algorithm to be a little less accurate — some of the games I’ve spent the most time “playing,” historically, are titles I’ve used for benchmarking, and it’s possible that will throw off the algorithm. I also have a very bad habit of leaving games alt-tabbed while running in the background, which can also drastically inflate my own playtimes.
That’s the default view when you login. Your top games are listed along the left, while recommended titles are on the right. Here’s the top game listing when you play with the “Niche” versus “Popular” slider. My “Popular” games list is on the left below, while the full “Niche” recommendations are on the right.
Regarding its recommendation algorithm, Valve writes:
One direction is to gather every single piece of information about a game, and then make guesses about what games are similar, and then recommend those “similar” games. But that allows for all sorts of weird distortions— just because you play a lot of Beat Saber, doesn’t mean we should only ever recommend you VR rhythm games. This model takes a different approach. It disregards most of the usual data about a game, like genre or price point. Instead, it looks at what games you play and what games other people play, then makes informed suggestions based on the decisions of other people playing games on Steam. The idea is that if players with broadly similar play habits to you also tend to play another game you haven’t tried yet, then that game is likely to be a good recommendation for you.
The “niche” versus “popular” slider seems as though it could use some fine-tuning. Somehow, Metro 2033 Redux is the most popular title recommended for me under both options. But it’s odd that this game should be recommended to me in the first place, given that I own (but have never bothered to beat) Metro 2033. The same is true of Metro Last Light Redux. It makes sense to recommend Metro 2033 Redux to me if the only thing you’re examining is either hours played (I used it for benchmarking) or “number of games owned in the Metro series.” Examining the number of unlocked achievements in these titles would show that I barely played either.
The other shortcoming I see in this data isn’t exactly Valve or Steam’s fault. I own Far Cry 3 on uPlay and Witcher 3 on GoG. I played games like Dishonored, Borderlands 2, and Wolfenstein: The Old Blood using a friend’s Steam account. The quality of the recommendations is a bit mixed — I loved Dishonored and found Borderlands 2 relatively amusing, but was not a huge fan of Far Cry 3 and have never actually played very much of The Witcher 3. The “niche” titles are games I haven’t played or heard of, so it seems to be fulfilling its goals in that regard, though that also makes it more difficult to interpret if I’d like them.
The goal is for this tool to be a better recommendation engine for games that isn’t susceptible to being gamed or manipulated, or that requires developers to worry about optimizing for underlying and unknown algorithms. The idea is for the AI model to watch what players are playing and recommend these games to people who play similar games to those people. We’ll have to wait for data on how it works, but discoverability has been a major problem for every kind of online store. Solving it, or even improving it, would be a major achievement for any storefront.