Implementing a simple web-log based recommender system

I’ve now implemented such a system as an extension to Catalyst, the open source Perl web framework. The system isn’t yet ready for general distribution, but I’d like to share my approach.

First, I’ve gathered ten years of web access logs from WormBase, a generic model organism database where I work as the project manager.

Next, I correlated IP addresses with requests and tried to trace browsing patterns from one object to the next. This isn’t an exact science since we haven’t historically tried to uniquely identify users.

Data is loaded into a simple MySQL schema with object and object2related tables. Expediently simple.

Recommender systems for biological databases

Recommender systems [Wikipedia] seek to provide users with information related to what they are currently browsing. These are now ubiquitous in e-commerce sites such as Amazon, where each page contains a list of items viewed or purchased by other users.

I’ve long felt that a recommender system could revolutionize the browsing and mining of biological data. The idea would be to provide users with a list of related objects based on browsing history of cadres. See this post for some preliminary implementation notes.

I am worried that a recommender system won’t be received with open arms. Given that my current implementation is based on web log analysis it presents serious privacy issues. It’s conceivable that it’s use could inadvertently reveal the identity of uncloned and unpublished loci.