Publication
1
Longitudinal Impact of Preference Biases on Recommender Systems' Performance, with Jingjing Zhang, and Gedas Adomavicius. [PDF]
Information Systems Research, Forthcoming.
Information Systems Research, Forthcoming.
- Best Student Paper Award at 2020 INFORMS Workshop on Data Science
Click for Abstract
Research studies have shown that recommender systems' predictions that are observed by users can cause biases in users' post-consumption preference ratings. Because users' preference ratings are typically fed back to the system as training data for future predictions, this process is likely to influence the performance of the system in the long run. We use a simulation approach to study the longitudinal impact of preference biases (and their magnitude) on the dynamics of recommender systems' performance. We look at the influence of preference biases in two conditions: (i) during the normal system use, where biases are typically caused by the system's inherent prediction errors, and (ii) in the presence of external (deliberate) recommendation perturbations. Our simulation results show that preference biases significantly impair the system's prediction performance (i.e., prediction accuracy) as well as users' consumption outcomes (i.e., consumption relevance and diversity) over time. The impact is non-linear to the size of the bias, i.e., large bias causes disproportionately large negative effects. Also, items that are less popular and less distinctive (in terms of their content) are affected more by preference biases. Additionally, intentional recommendation perturbations, even on a small number of items for a short time, substantially amplify the negative impact of preference bias on a system's longitudinal dynamics and cause long-lasting effects on users' consumption. Finally, given the impact of preference bias on the recommender systems' performance, we explore the problem of debiasing user-submitted ratings. We empirically demonstrate that relying solely on historical rating data is unlikely to be effective in debiasing. We also propose and evaluate two debiasing approaches that take into account additional relevant information that can be collected by recommendation platforms. Our findings provide important implications for the design of recommender systems.
2
Healthcare across Boundaries: Urban-Rural Differences in the Financial and Healthcare Consequences of Telehealth Adoption, with Xuelin Li, and Gordon Burtch. [PDF]
Information Systems Research, Volume 35, Issue 3, September 2024.
Information Systems Research, Volume 35, Issue 3, September 2024.
- Covered by Slate .
- Best Paper Award at 2021 ZEW Conference on ICT
Click for Abstract
We study the impacts of telehealth adoption on geographic competition among urban and rural healthcare providers. We consider a quasi-natural experiment: states' entry into the Interstate Medical Licensure Compact, wherein the entry events facilitate healthcare providers to adopt telehealth technology. By analyzing a representative sample of providers, we first establish the Compact-entry shock's validity and its positive effect on the supply of medical services. We then report evidence that there are service and payment shifts from rural providers to urban providers, i.e., urban providers are more likely to benefit from the Compact-entry financially. Relying on patients' telehealth reimbursement claim data, we observe two mechanisms contributing to the revenue re-distribution: the substitution and gateway effects of telehealth. Finally, we show that telehealth readiness and service quality moderate the impact of telehealth adoption. These findings speak to both potentially positive and negative consequences for welfare.Working Paper
3
New Category Recommendation in Concentrated Sales Environments: Evidence from a Randomized Field Experiment, with Ravi Bapna, Gedas Adomavicius, and Jonathan Hershaff. [PDF]
Under Review
Under Review
Click for Abstract
We design category-introduction-oriented recommendation methods for highly concentrated sales environments to increase purchases from new product categories. We show that traditional recommendation techniques such as collaborative filtering do not work well on the task. Instead, we leverage granular consumer journey data coupled with comprehensive feature engineering and a robust machine-learning process to implement a predictive model for this task. We validate the model using extensive offline evaluations and benchmark it against a large number of the baseline approaches. Furthermore, to estimate the causal economic impact, we conduct a large-scale randomized controlled trial (RCT). Importantly, we find that the proposed approach to new category recommendation can increase the purchase probability by up to 35% compared with no recommendation. Our detailed experiment design provides a number of additional insights on how the purchase probability, revenue, and profit are impacted when the new category recommendations are personalized vs. non-personalized (e.g., recommending the most popular category that is new to a given customer) or when one vs. several new categories are recommended. We go beyond inferring average treatment effects and use our rich data to exploreheterogeneous treatment effects. We find differential impacts of new product recommendations on more recent vs. less recent customers. A conditional average treatment effect treatment policy, which deploys the best treatment for different user segments, shows favorable lift in profit.
4
Impact of Data Privacy Regulations on Recommender Systems Performance, with Liben Chen, Yicheng Song, and Gedas Adomavicius.
Major Revision at ISR
Major Revision at ISR
- Best Student Paper Nominee at Workshop on Information Technologies and Systems (WITS) 2021
Click for Abstract
Data privacy regulations empower consumers to control the collection of their personal data. A significant consequence of these regulations is their effect on various data-driven business solutions, especially on personalization technologies and recommender systems. We investigate the potential impacts of diverse real-world data privacy practices (that can be adopted by firms in response to various data privacy regulations) on the recommender systems performance. We also examine how these impacts vary across different personalization contexts and applications. In particular, we distinguish between scenarios where the user population exhibits more stable vs. more dynamic (i.e., changing) preferences, as these scenarios often represent distinctly different recommendation settings. We use a simulation framework, carefully seeded with real-world data, for a comprehensive evaluation of the recommender system performance under numerous scenarios, including: different recommendation algorithms, different data privacy practices, different degrees of users' preference dynamics, different sizes of privacy-conscious sub-population, different degrees of users' reliance on the recommender system, and the use of traditional vs. incremental learning approaches. Extensive computational experiments uncover several robust performance patterns for different data privacy practices and highlight substantial important differences between recommendation settings with stable vs. changing user preferences. The findings of this study have significant implications for the design of privacy-aware recommender systems in the context of contemporary data privacy regulations. The findings can also be informative to policymakers for understanding the practical implications of various data privacy practices and for designing future policies.
5
Can Decision Support Systems Distort Human Capital?, with Xuelin Li. [PDF]
Under Review
Under Review
- Best Paper Award at The Conference on Health IT and Analytics (CHITA) 2024
Click for Abstract
We document that interactions with manipulated decision support systems can distort the development of human capital using the context of opioid prescription. Physicians in our sample adopted electronic health record software from a list of federally certified companies in 2011. Between 2016 and spring 2019, one company secretly embedded a biased decision support system function to promote extended-release opioid sales. Affected physicians not only increased opioid claims relative to the control group during the treatment window but also maintained a higher propensity for prescriptions even after the removal of the biased function. This long-term distortion of human capital relies on the unconsciousness of algorithmic biases and does not occur following other explicit promotions, such as pharmaceutical detailing payments. Using machine-learning algorithms, we quantify that human capital distortion explains 54% of the treatment effects in a physician decision model with dynamic learning. Experience with opioids, along with caution regarding elder patients, mitigates the distortion.In Progress
1
The Impact of Telehealth Expansion on “Doctor-Shopping” and Drug Overdoses
with Byoung-Hyuk Ahn and Gordon Burtch
with Byoung-Hyuk Ahn and Gordon Burtch