STR
TIM
Albert Roh
U. of Southern California -Marshall School of Business, United States
Nan Jia
U. of Southern California, United States
Natalie Carlson
The Wharton School, U. of Pennsylvania, United States
Bart Vanneste
UCL School of Management, United Kingdom
Hyo Kang
U. of Southern California, United States
The recent advances in Large Language Models (LLMs) have dramatically transformed the landscape since the launch of OpenAI’s ChatGPT in November 2022. These models, known for their deep understanding of language and reasoning, have rapidly become integral in various domains, mirroring human cognition with remarkable fidelity. Major tech firms like Microsoft, Google, and Meta have embraced this innovation, launching products like Bing AI, Bard, and Llama. Concurrently, these firms have also reduced hiring and conducted layoffs in roles that are increasingly automated by this technology. These dual facets of LLMs - their capabilities and organizational impact - are pivotal for management scholars. Firstly, in their role as tools, LLMs demonstrate exceptional ability in processing nuanced interpretations and managing extensive textual datasets. This versatility makes them invaluable across various research stages, from ideation to copy editing, as noted by Korinek (2023). Their applications include complex tasks such as data annotation (Gilardi et al., 2023; Rathje et al., 2023; Tornberg, 2023) and simulating participant responses in experimental studies (Boussioux et al., 2023). Second, as research subjects, LLMs are reshaping methodologies in firm-level strategic decision-making, underscoring their transformative potential in both strategy formation and refinement. Furthermore, at the market level, a growing body of research is exploring their impact on employment dynamics (Brynjolfsson et al., 2023; Dell’Acqua et al., 2023; Eloundou et al., 2023; Noy and Zhang, 2023) and organizational decision-making processes. Despite the increasing number of studies addressing these aspects, our comprehension of LLMs, both as tools and subjects, remains notably limited. Recognizing the substantial impact of LLMs and the need for more in-depth understanding, this symposium has been organized to explore LLMs both as a tool (Paper 1-3) and a subject (Paper 4). It brings together preeminent researchers to present their latest findings on how LLMs are shaping the future of strategic management. Each paper contributes to a deeper understanding of the role of LLMs in strategic management, showcasing their unique applications and implications. The first and second papers introduce innovative ways for management scholars to utilize LLMs. The first paper examines the use of LLMs in data annotation and text classification within strategic management research, focusing on identifying product sustainability in crowdfunding projects. This study reveals that ChatGPT can match or exceed the efficiency of traditional methods with careful prompt refinement. However, the authors also found that minor prompt variations can significantly alter annotation outcomes. These variations have serious implications for the accuracy and robustness of subsequent data analysis. To combat this, the study introduces Prompt Variance Estimation (PVE), a method ensuring analytical robustness for LLM-generated labels, complete with detailed instructions and coding guidelines. The second paper explores the application of LLMs in managing and analyzing unstructured data, such as congressional hearing transcripts. It outlines three key research tasks that LLMs can perform: text summarization, topic extraction, and extraction of related concepts. Applying these tasks to data from the U.S. House Space, Science, and Technology Committee, the paper studies the interaction between government policies and firm technology strategies. It showcases the capability of LLMs to process extensive textual data, overcoming issues like context window limitations. This paper emphasizes the efficiency of LLMs and their complementarity to conventional NLP methods, with insights shared via an interactive dashboard and digital platform. The third paper focuses on the application of LLMs as a decision-making tool for firms. It investigates how generative AI, particularly LLMs, can aid in assessing the value of strategic alternatives, a vital task for irreversible business decisions. The study compares traditional machine learning methods with the generative capabilities of AI, assessing 60 AI-created business models across various industries. It aims to determine the extent to which AI aligns with human judgment in strategic decision-making, employing correlational and Bayesian analyses. This research highlights the potential of generative AI in scenarios with limited or unique data, offering fresh perspectives on AI’s role in strategic business decisions. Finally, the fourth paper delves into the role of generative AI, as a subject of study. It examines how LLMs transform research outputs, focusing on the varied impacts on researchers of different language skills and experience levels. The paper suggests that non-native English speakers and less experienced researchers might benefit more from AI tools, thereby potentially reducing the communication gap in academia. Utilizing AI detectors like GPTZero, it evaluates AI usage in submissions to the Academy of Management Annual Meeting, analyzing their linguistic quality. The results demonstrate diverse adoption and benefits of LLMs across varying researcher demographics. This research offers a nuanced perspective on generative AI's impact within the strategic management community, highlighting its potential to address or accentuate disparities in academic communication and representation.
Author: Natalie Carlson – The Wharton School, U. of Pennsylvania
Author: Albert Choi Roh – U. of Southern California -Marshall School of Business
Author: Bart Vanneste – UCL School of Management
Author: Hyo Kang – U. of Southern California