Journal Papers under Review

Longitudinal Impact of Preference Biases on Recommender Systems' Performance, with Jingjing Zhang, and Gedas Adomavicius. [PDF]
Click for Abstract

Research studies have shown that recommender systems' predictions that are observed by users can cause biases in users' post-consumption preference ratings. Because users' preference ratings are typically fed back to the system as training data for future predictions, this process is likely to influence the performance of the system in the long run. We use a simulation approach to study the longitudinal impact of preference biases (and their magnitude) on the dynamics of recommender systems' performance. We look at the influence of preference biases in two conditions: (i) during the normal system use, where biases are typically caused by the system's inherent prediction errors, and (ii) in the presence of external (deliberate) recommendation perturbations. Our simulation results show that preference biases significantly impair the system's prediction performance (i.e., prediction accuracy) as well as users' consumption outcomes (i.e., consumption relevance and diversity) over time. The impact is non-linear to the size of the bias, i.e., large bias causes disproportionately large negative effects. Also, items that are less popular and less distinctive (in terms of their content) are affected more by preference biases. Additionally, intentional recommendation perturbations, even on a small number of items for a short time, substantially amplify the negative impact of preference bias on a system's longitudinal dynamics and cause long-lasting effects on users' consumption. Finally, given the impact of preference bias on the recommender systems' performance, we explore the problem of debiasing user-submitted ratings. We empirically demonstrate that relying solely on historical rating data is unlikely to be effective in debiasing. We also propose and evaluate two debiasing approaches that take into account additional relevant information that can be collected by recommendation platforms. Our findings provide important implications for the design of recommender systems.

Accepted at Information Systems Research

Healthcare across Boundaries: Urban-Rural Differences in the Financial and Healthcare Consequences of Telehealth Adoption, with Xuelin Li, and Gordon Burtch. [PDF]
Click for Abstract

We study the impacts of telehealth adoption on geographic competition among urban and rural healthcare providers. We consider a quasi-natural experiment: states' entry into the Interstate Medical Licensure Compact, wherein the entry events facilitate healthcare providers to adopt telehealth technology. By analyzing a representative sample of providers, we first establish the Compact-entry shock's validity and its positive effect on the supply of medical services. We then report evidence that there are service and payment shifts from rural providers to urban providers, i.e., urban providers are more likely to benefit from the Compact-entry financially. Relying on patients' telehealth reimbursement claim data, we observe two mechanisms contributing to the revenue re-distribution: the substitution and gateway effects of telehealth. Finally, we show that telehealth readiness and service quality moderate the impact of telehealth adoption. These findings speak to both potentially positive and negative consequences for welfare.

Accepted at Information Systems Research. Covered by Slate .

Design and Evaluation of New Product Category Recommendations: Evidence from a Randomized Field Experiment, <with Ravi Bapna, Gedas Adomavicius, and Jonathan Hershaff. [PDF]
Click for Abstract

Recommending new product categories to existing consumers (i.e., categories that they have not yet purchased) can be useful for increasing customer lifetime value as well as for reducing risks from category-specific supply shocks and category-specific competition. In this paper, we design category-introduction-oriented recommendation methods to increase customers' purchases from new product categories. We focus on application settings where the sales are highly concentrated, i.e., where the new category recommendation is particularly challenging. We use granular consumer journey data, employ comprehensive feature engineering and selection, and compare 15 recommendation models designed for new category introduction with robust offline evaluations. Then we estimate the causal economic impact of new category recommendations using a large-scale randomized controlled trial (RCT). We find that the new product category recommendation can increase the purchase probability by up to 35% compared with no recommendation. We also explore two dimensions, namely, (i) increasing the choice in recommended new categories and (ii) providing personalized (as opposed to non-personalized) recommendations. We find that increasing choices further increases the sales in the recommended categories by up to 9% as compared to recommending a single category, and personalized new category recommendation leads to 11% more purchases than recommending the most popular (non-personalized) new category. However, when recommending personalized new categories, more choices do not further increase sales as compared to recommending only one category. Finally, we go beyond standard average treatment effect analysis to discover customer heterogeneity. We find that the most recent visitors (who visit the platform within last a couple of days before the new category recommendation) are most responsive to multiple choices. In contrast, personalizing recommendations is more effective for not-so-recent customers, who visit the platform within three months before the treatment. A conditional average treatment effect treatment policy, which deploys the best treatment for different user segments, shows favorable lift in profit.

Reject and Resubmit at Management Science

Working in Progress

Impact of Data Privacy Regulations on Recommender Systems Performance in the Presence of Stable vs. Dynamic User Preferences., with Liben Chen, Yicheng Song, and Gedas Adomavicius.