Tuesday, October 05, 2010

[FREE-FOOD] google research seminar, oct 6, 12:30-1:30pm, Y2E2 building 105

Lunch will be served

User Browsing Models: Relevance versus Examination

Sugato Basu, Google Research.

There has been considerable work on user browsing models for search
engine results, both organic and sponsored. The click-through rate
(CTR) of a result is the product of the probability of examination
(will the user look at the result) times the perceived relevance of
the result (probability of a click given examination). Past papers
have assumed that when the CTR of a result varies based on the
pattern of clicks in prior positions, this variation is solely due
to changes in the probability of examination.

We show that, for sponsored search results, a substantial portion
of the change in CTR when conditioned on prior clicks is in fact
due to a change in the relevance of results for that query instance,
not just due to a change in the probability of examination. We then
propose three new user browsing models, which attribute CTR changes
solely to changes in relevance, solely to changes in examination
(with an enhanced model of user behavior), or to both changes in
relevance and examination. The model that attributes all the CTR
change to relevance yields substantially better predictors of CTR
than models that attribute all the change to examination, and does
only slightly worse than the model that attributes CTR change to
both relevance and examination. For predicting relevance, the model
that attributes all the CTR change to relevance again does better
than the model that attributes the change to examination. Surprisingly,
we also find that one model might do better than another in predicting
CTR, but worse in predicting relevance. Thus it is essential to
evaluate user browsing models with respect to accuracy in predicting
relevance, not just CTR.

No comments: