As anyone using the Internet can tell, there are recommendation systems everywhere – from suggesting the movies you should watch and products you should buy to suggesting the people you should date. The problem with these systems is that they are based on the assumption that similar people have similar preferences and buy similar things – they predict which item you will like best based solely on your, and the item’s, previous ratings. In other words, they are too simplistic.
In the real world, every person has a unique mixture of interests. Just because a person has liked one science fiction movie, doesn’t mean they will like another, recommended science fiction movie, as these usually have nothing else in common except for the genre.
The good news is that researchers from the Santa Fe Insitute have created a new recommendation system that is better and faster than the existing ones. The new algorithm differs from conventional models in two major ways:
1. It allows people and items to belong to mixtures of multiple overlapping groups
2. It doesn’t assume that ratings are a simple function of similarity – instead, it predicts probability distributions of ratings based on the groups to which the individual or item belongs to.
In other words, the new algorithm is way more flexible, and therefore more realistic.