Research

Working Papers

Measuring Higher-Order Rationality with Belief Control (2024) R&R at Experimental Economics

(with Wei James Chen and Meng-Jhang Fong)

Determining an individual's strategic reasoning ability based solely on choice data is a complex task. This complexity arises because sophisticated players might have non-equilibrium beliefs about others, leading to non-equilibrium actions. In our study, we pair human participants with computer players known to be fully rational. This use of robot players allows us to disentangle limited reasoning capacity from belief formation and social biases. Our results show that, when paired with robots, subjects consistently demonstrate higher levels of rationality and maintain stable rationality levels across different games compared to when paired with humans. This suggests that strategic reasoning might indeed be a consistent trait in individuals. Furthermore, the identified rationality limits could serve as a measure for evaluating an individual's strategic capacity when their beliefs about others are adequately controlled.


Sequential equilibrium is the conventional approach for analyzing multi-stage games of incomplete information. It relies on the requirement of mutual consistency of beliefs. To relax mutual consistency, I theoretically and experimentally explore the dynamic cognitive hierarchy (DCH) solution. One property of DCH is that the solution can vary between two different games sharing the same reduced normal form, i.e., violation of invariance under strategic equivalence. I test this prediction in a laboratory experiment using two strategically equivalent versions of the dirty-faces game. The game parameters are calibrated to maximize the expected difference in behavior between the two versions, as predicted by DCH. The experimental results indicate significant differences in behavior between the two versions, and more importantly, the observed differences align with DCH. This suggests that implementing a dynamic game experiment in reduced normal form (using the "strategy method") could lead to distortions in behavior.


A Note on Cursed Sequential Equilibrium and Sequential Cursed Equilibrium (2023) Working Paper

(with Meng-Jhang Fong and Thomas R. Palfrey)

In this short note, we compare the cursed sequential equilibrium (CSE) by Fong et al. (2023) and the sequential cursed equilibrium (SCE) by Cohen and Li (2023). We identify eight main differences between CSE and SCE with respect to the following features: 

(1) the family of applicable games, 

(2) the number of free parameters,

(3) the belief updating process,

(4) the treatment of public histories, 

(5) effects in games of complete information, 

(6) violations of subgame perfection and sequential rationality,

(7) re-labeling of actions, and 

(8) effects in one-stage simultaneous-move games. 


Cursed Sequential Equilibrium (2023) R&R at American Economic Review

(with Meng-Jhang Fong and Thomas R. Palfrey)

This paper develops a framework to extend the strategic form analysis of cursed equilibrium (CE) developed by Eyster and Rabin (2005) to multi-stage games. The approach uses behavioral strategies rather than normal form mixed strategies, and imposes sequential rationality. We define the cursed sequential equilibrium (CSE) and compare it to sequential equilibrium and standard normal-form CE. We provide a general characterization of CSE and establish its properties. We apply CSE to five applications in economics and political science. These applications illustrate a wide range of differences between CSE and Bayesian Nash equilibrium or CE: in signaling games; games with preplay communication; reputation building; sequential voting; and the dirty faces game where higher order beliefs play a key role. A common theme in several of these applications is showing how and why CSE implies systematically different behavior than Bayesian Nash equilibrium in dynamic games of incomplete information with private values, while CE coincides with Bayesian Nash equilibrium for such games.


Published Papers

Cognitive Hierarchies for Games in Extensive Form (2024) Journal of Economic Theory

(with Thomas R. Palfrey) 

In the cognitive hierarchy (CH) framework, players in a game have heterogeneous levels of strategic sophistication. Each player believes that other players in the game are less sophisticated, and these beliefs correspond to the truncated distribution of a "true" distribution of levels. We develop the dynamic cognitive hierarchy (DCH) solution by extending the CH framework to games in extensive form. Initial beliefs are updated as the history of play provides information about players' levels of sophistication. We establish some general properties of DCH and fully characterize the DCH solution for a wide class of centipede games. DCH predicts a strategy-reduction effect: there will be earlier taking if the centipede game is played as an alternating-move sequential game rather than as a simultaneous move game in its reduced normal form. Experimental evidence reported in García-Pola et al. (2020) supports this prediction. In all three centipede games for which the DCH strategy-reduction effect is predicted, termination occurs earlier when played sequentially rather than simultaneously with reduced strategies. In a fourth centipede game, where this effect is not predicted, it is not observed. 


Using Machine Learning to Understand Bargaining Experiments (2022) Bargaining: Current Research and Future Directions, edited by Kyle B. Hyndman and Emin Karagözoğlu

(with Colin F. Camerer, Hung-Ni Chen, Gideon Nave, Alec Smith and Joseph Tao-yi Wang)

We study dynamic unstructured bargaining with deadlines and one-sided private information about the amount available to share (the "pie size"). "Unstructured" means that players can make or withdraw any offers and demands they want at any time. Such paradigms, while lifelike have been displaced in experimental studies by highly structured bargaining because they are hard to analyze. Machine learning comes to the rescue because the players' wide range of choices in unstructured bargaining can be taken as "features" used to predict behavior. Machine learning approaches can accommodate a large number of features and guard against overfitting using test samples and methods such as penalized LASSO regression. In previous research we found that LASSO could add power to theoretical variables in predicting whether bargaining ended in disagreement. We replicate this work with higher stakes, subject experience, and special attention to gender differences, demonstrating the robustness of this approach.


Conducting Large, Repeated, Multi-game Economic Experiments Using Mobile Platforms (2021) PLoS One

(Co-first Author; with Zhi Li, Si-Yuan Kong, Dongwu Wang and John Duffy)

We demonstrate the possibility of conducting synchronous, repeated, multi-game economic decision-making experiments with hundreds of subjects in-person or remotely with live streaming using entirely mobile platforms. Our experiment provides important proof-of-concept that such experiments are not only possible, but yield recognizable results as well as new insights, blurring the line between laboratory and field experiments. Specifically, our findings from 8 different experimental economics games and tasks replicate existing results from traditional laboratory experiments despite the fact that subjects play those games/task in a specific order and regardless of whether the experiment was conducted in person or remotely. We further leverage our large subject population to study the effect of large (N = 100) versus small (N = 10) group sizes on behavior in three of the scalable games that we study. While our results are largely consistent with existing findings for small groups, increases in group size are shown to matter for the robustness of those findings.


Evidence of General Economic Principles of Bargaining and Trade from 2,000 Classroom Experiments (2020) Nature Human Behaviour

(First Author; with Alexander L. Brown, Taisuke Imai, Joseph Tao-yi Wang, Stephanie W. Wang and Colin F. Camerer)

Standardized classroom experiments provide evidence about how well scientific results reproduce when nearly identical methods are used. We use a sample of around 20,000 observations to test reproducibility of behaviour in trading and ultimatum bargaining. Double-auction results are highly reproducible and are close to equilibrium predictions about prices and quantities from economic theory. Our sample also shows robust correlations between individual surplus and trading order, and autocorrelation of successive price changes, which test different theories of price dynamics. In ultimatum bargaining, the large dataset provides sufficient power to identify that equal-split offers are accepted more often and more quickly than slightly unequal offers. Our results imply a general consistency of results across a variety of different countries and cultures in two of the most commonly used designs in experimental economics.


Other Publications

Reproducibility in Management Science (2024) Management Science

(As a member of Management Science Reproducibility Collaboration; with Miloš Fišar, Ben Greiner, Christoph Huber, Elena Kotak, Ali Ozkes and Management Science Reproducibility Collaboration)

With the help of more than 700 reviewers, we assess the reproducibility of nearly 500 articles published in the journal Management Science before and after the introduction of a new Data and Code Disclosure policy in 2019. When considering only articles for which data accessibility and hardware and software requirements were not an obstacle for reviewers, the results of more than 95% of articles under the new disclosure policy could be fully or largely computationally reproduced. However, for 29% of articles, at least part of the dataset was not accessible for the reviewer. Considering all articles in our sample reduces the share of reproduced articles to 68%. These figures represent a significant increase compared with the period before the introduction of the disclosure policy, where only 12% of articles voluntarily provided replication materials, of which 55% could be (largely) reproduced. Substantial heterogeneity in reproducibility rates across different fields is mainly driven by differences in data set accessibility. Other reasons for unsuccessful reproduction attempts include missing code, unresolvable code errors, weak or missing documentation, and software and hardware requirements and code complexity. Our findings highlight the importance of journal code and data disclosure policies and suggest potential avenues for enhancing their effectiveness.


Artificial Intelligence, the Missing Piece of Online Education? (2018) IEEE Engineering Management Review

(with Andrew Wooders, Joseph Tao-yi Wang and Walter M. Yuan)

Despite the recent explosive growth of online education, it still suffers from suboptimal learning efficacy, as evidenced by low student completion rates. This deficiency can be attributed to the lack of facetime between teachers and students, and amongst students themselves. In this article, we use the teaching and learning of economics as a case study to illustrate the application of artificial intelligence (AI) based robotic players to help engage students in online, asynchronous environments, therefore, potentially improving student learning outcomes. In particular, students could learn about competitive markets by joining a market full of automated trading robots who find every chance to arbitrage. Alternatively, students could learn to play against other humans by playing against robotic players trained to mimic human behavior, such as anticipating spiteful rejections to unfair offers in the Ultimatum Game where a proposer offers a particular way to split the pot that the responder can only accept or reject. By training robotic players with past data, possibly coming from different country and regions, students can experience and learn how players in different cultures might make decisions under the same scenario. AI can thus help online educators bridge the last mile, incorporating the benefit of both online and in-person learning.

Work in Progress

An Experiment on the Representation Effect of Centipede Games

(with Shiang-Hung Hu, Thomas R. Palfrey, Joesph Tao-yi Wang and Yu-Hsiang Wang)

In this laboratory experiment, we aim to test the “representation effect” predicted by the dynamic cognitive hierarchy solution (Lin and Palfrey, 2024). Within the family of centipede games, the dynamic cognitive hierarchy solution predicts that players tend to end the game earlier when played according to the extensive form representation compared to the reduced normal form, while players will behave similarly when the game is played according to the extensive form representation and the non-reduced normal form. To test this prediction at the individual level, we employ a within-subject design where each player will participate in a sequence of centipede games under the non-reduced normal form, reduced normal form, and extensive form representations. Specifically, we consider two linear centipede games, two exponential centipede games, and two constant sum centipede games. The order of the representations is controlled, and the payoff parameters are chosen to maximize the informativeness of the experiment.


An Experiment on Public Goods Game with Communication 

(with Meng-Jhang Fong and Thomas R. Palfrey)

We propose an experiment to study how varying the number of players (N) and the amount of largest possible contribution cost (K) would affect the efficiency of contribution in a threshold public goods game with private information and pre-game communication. Fong et al. (2023) define the cursed sequential equilibrium, which extends the standard normal-form cursed equilibrium (Eyster and Rabin, 2005) to multi-stage games. In a N-person threshold public goods game under unanimity rule, the CSE predicts that pre-game communication will be less effective in coordinating public goods contribution as N and K increase, while the prediction of sequential equilibrium and standard cursed equilibrium do not change in N and K. 


Magical Equilibrium

(with Thomas R. Palfrey)

We propose an extensive-form solution concept where players incorrectly believe their initial moves can affect other players' behavior, even if these actions are known to be unobservable. This biased reasoning is called "magical reasoning." Our solution concept discriminates between "nothing has happened" and "something has happened, but we don't know what," both of which have the same amount of information from the standard game theory perspective. More importantly, magical equilibrium explains the experimental findings that people tend to behave differently when the timing structure of games varies while maintaining the same amount of information.