Dissertations, Theses, and Capstone Projects

Date of Degree

9-2021

Document Type

Dissertation

Degree Name

Ph.D.

Program

Business

Advisor

Karl R. Lang

Committee Members

Nanda Kumar

Shadi Shuraida

Guillaume Frechette

Subject Categories

Business Analytics | Business Intelligence | Management Information Systems

Keywords

algorithmic decision-making, human-machine collaboration, collaborative decision-making, organizational decision-making, experimental economics

Abstract

This dissertation examines organizational decision-making in the context of big data and artificial intelligence (machine learning) technologies. There are three studies. All three focus on collaborative decision-making in organizations, with study 1 examining it in the context of big data, while study 2 and 3 in the context of artificial intelligence.

Study 1: This study examines the impact of different manners of presenting information on collaborative decision-making performance. Using controlled economic experiments, I assign participants with a resource allocation decision-making task (adapted from the game theoretic public goods provision problem) and examine the collaborative outcomes of groups when exposed to different levels of information aggregation and visualization formats. Interestingly, the results show that in certain cases, the more effective means of presenting information for individuals (i.e., graphs or tables compared to raw data) do not bode as well for groups. This study contributes to the information visualization literature, which has mostly looked at individual decision-making, by examining the collaborative task context and by combining perspectives from experimental game theory, cognitive fit and information processing theories. Methodologically, this study also contributes to the information visualization literature by accounting for the dynamics of collaborative decision-making over time.

Study 2: Organizations increasingly deploy artificial intelligence (AI) systems to automate specific tasks and assist human experts in organizational decision-making. In this study, I focus on complex task settings wherein human decision-makers work with AI systems. Using credit authorization for consumer loans as our specific context, I conduct an economic experiment with a repeated round design to investigate how organizations can create business value from the new human–machine collaborative decision-making paradigm. This study contributes to extant literatures on algorithmic decision-making and automation by moving beyond only examining individual decision-makers’ attributes to examine the intertwined roles of organizational factors and AI’s characteristics. The results show that when firms implement complementary organizational practices in parallel with AI investments, they achieve higher levels of algorithm appreciation, leading to better decisions, made with stronger confidence, in turn increasing organizational profits. I also show that human decision-makers and machines develop increasingly more effective work relationships over time and outperform AI machines in stand-alone settings. Finally, I show evidence that keeping humans in the loop could enable AI-powered firms to achieve the most productive outcomes.

Study 3: Extant research on algorithmic bias has mostly approached the subject from a technical perspective, with few studies investigating the decision bias of human-machine collaborative decision-making, wherein human experts have the final say after working with the algorithms. In this study, I conduct a controlled economic experiment with a repeated-round design. I assign participants with a task that models a complex organizational decision-making process wherein human decision-makers (DMs) work with an AI repeatedly over 10 decision periods to evaluate consumer loan applications. I use loan data from a large-scale, historic dataset and manipulate the AI predictions to create two experimental conditions: (1) Prediction Bias, where DMs work with AI predictions that discriminate against one group of loan applicants and favor another, and (2) No Bias, where DMs work with AI predictions that treat the two loan applicant groups equally. This study contributes to current research on algorithmic bias mitigation and bias in human-machine collaboration by showing that human DMs can over time learn to adapt to a biased algorithm, implicitly detect the bias in the AI, adjust their behavior to significantly improve their performance, and importantly, outperform the biased AI working alone, both in terms of reducing decision bias and increasing organizational profit.

Download

Share

COinS