Jenny Baker / Thursday, September 21, 2023 / Categories: 612 What Can ChatGPT Tell Us About I-O Psychology? Gordon B. Schmidt, University of Louisiana Monroe, & Benjamin Biermeier-Hanson, Radford University In late November 2022, ChatGPT was released to the public with great public attention (OpenAI, 2022). With it came discussion of how transformative this technology will be. Would it eliminate HR jobs (Thibodeau, 2023)? Would it make cheating rampant in all essay-based online class assignments (Naidoo, 2023; Surovell, 2023)? Would it do our research and write our papers for us (McAfee, 2023)? (That one sounds nice in a way, as long as it doesn’t also take my job). Would it mean AI could replace I-O psychologists in the future (Frescoj, 2023)? (I don’t like this one as much.) With its release some I-O psychologists began to play with ChatGPT to see what it has to say about I-O topics, such as what I-O psychologists do (in iambic pentameter no less; Tadjbakhsh, 2023) and what would be a better name to make the field more accessible to the public (Schmidt, 2023). So we decided to see what ChatGPT knows about our field. We asked ChatGPT several questions and offered our own analysis of the responses. These questions range from almost completely factual to primarily opinions. Through this process, we hope to better show and understand what ChatGPT provides, for better or worse. And perhaps we will learn something (accurate or not). As we think about the past of I-O psychology and TIP, ChatGPT represents something that may well impact our future. Our aim was to see how ChatGPT did with a variety of question types germane to our field while also having some fun! All questions were asked on the same day (July 10, 2023) and in the order listed below to the free version of ChatGPT 3.5. Feel free to ask ChatGPT (or another program) the same or similar questions and compare the answers you get! Theme 1: Basic Definitions Question 1: What Is I-O Psychology? Industrial-organizational (I-O) psychology is a field of psychology that focuses on studying human behavior in the workplace. It applies psychological principles and theories to various work-related issues with the aim of improving organizational performance and the well-being of employees. I-O psychologists analyze individual, group, and organizational dynamics to understand and enhance various aspects of work, such as employee selection and assessment, training and development, performance appraisal, motivation, job satisfaction, work–life balance, leadership, team dynamics, and organizational culture. Some common areas of focus within I-O psychology include: Employee selection: I-O psychologists develop and implement methods for assessing and selecting job candidates. They design tests, interviews, and other assessment tools to identify individuals with the knowledge, skills, and abilities required for specific job roles. Training and development: I-O psychologists analyze training needs, develop training programs, and evaluate their effectiveness. They aim to enhance employees’ knowledge, skills, and abilities to improve job performance and organizational outcomes. Performance appraisal: I-O psychologists develop performance appraisal systems and tools to assess and provide feedback on employee performance. They help organizations establish fair and effective methods for evaluating individual and team performance. Organizational development: I-O psychologists work with organizations to improve their overall functioning and effectiveness. They may assist in areas such as organizational culture, leadership development, change management, and employee engagement. Work–life balance: I-O psychologists examine ways to promote work–life balance and employee well-being. They explore strategies to reduce work-related stress, enhance job satisfaction, and create supportive work environments. Diversity and inclusion: I-O psychologists study diversity and inclusion issues within organizations. They work to promote equal opportunities, reduce bias, and create inclusive workplaces that value diversity. I-O psychologists utilize research methodologies, data analysis, and statistical techniques to gather and interpret data for evidence-based decision making. They may work as consultants, researchers, educators, or practitioners in various settings, including corporations, government agencies, nonprofit organizations, and academic institutions. Response to Question 1: Overall, our first ChatGPT response is fairly good! ChatGPT hits on the basic definition (applying psychological principles to the workplace), lays out a breadth of I-O-related topics, emphasizes our scientifically driven approach, and sums up where we may work. If we had to nitpick (and we do, trust us), we did note that job analysis was notably omitted from the list of topics. Although some of the other topics are what we may consider our “bread and butter,” we were also interested that work–life balance and diversity and inclusion were included with the others. Although somewhat unsurprising, given the importance of these topics and their inclusion in the most recent SIOP Top 10 Workplace Trends, we nonetheless noted their inclusion with the others and take this as a first example of ChatGPT forecasting (rightly, in our opinion) that these will be/are now a “bread and butter” part of our field. Theme 2: Can ChatGPT Give Good Advice on I-O Processes? Question 2: How Do You Design an Effective Performance Appraisal Form for an HR Manager Position? Designing an effective performance appraisal form for an HR manager position involves considering the specific responsibilities and competencies required for success in that role. Here are some key steps to design such a form: Identify key job responsibilities: Start by identifying the primary responsibilities of an HR manager in your organization. This may include areas such as recruitment and selection, employee relations, performance management, training and development, compensation and benefits, HR policy development, and strategic HR planning. Determine performance dimensions: Break down the HR manager role into specific performance dimensions or competencies that align with the identified responsibilities. Examples of performance dimensions may include leadership and management, communication and interpersonal skills, problem solving, decision making, employee engagement, and HR process knowledge. Define performance indicators: For each performance dimension, define specific performance indicators or behaviors that demonstrate effective performance. These indicators should be observable and measurable. For example, under the leadership and management dimension, an indicator could be “Demonstrates the ability to provide clear direction and guidance to the HR team.” Establish rating scales: Develop a rating scale that allows evaluators to assess the level of performance for each indicator. Common rating scales include numerical scales (e.g., 1–5), descriptive scales (e.g., “needs improvement,” “meets expectations,” “exceeds expectations”), or a combination of both. Ensure that the rating scale is clear and easy to understand. Include open-ended questions: In addition to rating scales, incorporate open-ended questions or comment sections where evaluators can provide qualitative feedback or additional comments on the HR manager’s performance. This allows for more detailed and context-specific feedback. Consider performance goals: If applicable, include a section for setting and evaluating performance goals for the HR manager. This helps establish performance expectations and allows for ongoing development and improvement. Review and refine: Once the initial draft of the performance appraisal form is created, review it with HR professionals and relevant stakeholders to gather feedback and ensure its comprehensiveness, clarity, and alignment with organizational goals and values. Make any necessary refinements based on the feedback received. Pilot testing: Before implementing the performance appraisal form organization wide, consider piloting it with a small group of HR managers to assess its effectiveness, identify any potential issues, and make further improvements if needed. Training and communication: Provide training and guidelines to evaluators on how to use the performance appraisal form effectively. Communicate the purpose, process, and expectations of the performance appraisal to HR managers to ensure transparency and understanding. Ongoing evaluation and updates: Regularly evaluate the effectiveness of the performance appraisal form, gather feedback from users, and make updates as necessary to ensure its continued relevance and usefulness in assessing HR manager performance. Remember that an effective performance appraisal form should align with the organization’s performance management philosophy, provide clear and objective criteria for evaluation, and encourage ongoing communication and development between HR managers and their supervisors. Response to Question 2: Whereas our first question was quite general, we aimed to get more specific to see how much practical guidance ChatGPT could give to something more technical than Question 1. Performance appraisal design can be seen as a bread-and-butter kind of task asked of I-O psychologists by organizations. We are somewhat more critical of the utility of this response. On the positive side, this lays out a reasonable template of what process is behind developing a performance appraisal. On the negative side, though, there are several key considerations, details, and nuances that are too critical to be left out in our opinion. Examples include (a) no discussion of competencies versus tasks/behaviors, (b) no mention of best practices (i.e., BARS) to guide development, (c) no mention of who evaluates or provides the data, and (d) no mention of context that may affect the job in question (i.e., big vs. small company, size of department, industry, etc.). In many ways, this felt more like a description of an HR department’s roles rather than an HR manager’s roles. When it comes to more specific, actionable guidance, ChatGPT appears to be better at generally laying out steps in the process rather than providing more specific “how-to” guidance, though more specific follow-up questions could potentially guide a user better. Theme 3: Can ChatGPT Give Us Valuable Perspectives? Question 3: What Are the Biggest Threats to the Field of I-O Psychology? Although the field of I-O psychology has been growing and evolving, there are a few potential threats and challenges that it may face. Some of the significant threats include Lack of awareness and understanding: One of the ongoing challenges for I-O psychology is the limited awareness and understanding of the field among the general public, as well as within organizations. This lack of awareness can hinder the recognition of the value and potential contributions that I-O psychologists can make in improving workplace productivity, well-being, and organizational effectiveness. Technological advancements: Although technology can enhance various aspects of I-O psychology, it also presents challenges. Automation and artificial intelligence may automate certain tasks traditionally performed by I-O psychologists, such as data collection and analysis. This could require I-O psychologists to adapt their skill sets, focus on higher level interpretation and strategy, and embrace technological advancements to remain relevant. Ethical concerns: As technology advances, ethical considerations become increasingly important. I-O psychologists must navigate issues related to data privacy, security, fairness, and potential biases in assessment and selection processes. Ensuring the responsible and ethical use of technology and data-driven insights is critical to maintaining trust and credibility in the field. Globalization and cultural context: With organizations operating in a globalized environment, I-O psychologists face the challenge of understanding and addressing cultural differences and diverse work environments. They must adapt their theories, methodologies, and interventions to suit different cultural contexts, ensuring the relevance and effectiveness of their work across diverse populations.] Resistance to change: Implementing changes in organizational practices and policies based on I-O psychology research and recommendations can face resistance from organizational leaders, employees, and other stakeholders. Overcoming resistance to change and fostering a culture that values evidence-based practices requires effective communication, stakeholder engagement, and strong advocacy for the field. Academic–practice gap: There can be a gap between academic research and its practical application in organizations. Bridging this gap is essential to ensure that the field of I-O psychology continues to address real-world challenges and provide practical solutions. Collaborations between researchers and practitioners, as well as efforts to translate research findings into actionable practices, are crucial to closing this gap. To address these threats, it is important for the field of I-O psychology to proactively communicate its value, educate stakeholders about its contributions, adapt to technological advancements, uphold ethical standards, embrace cultural diversity, and foster collaboration between academia and practice. By staying current, relevant, and responsive to evolving workplace needs, I-O psychology can continue to thrive and make significant contributions in improving organizations and the well-being of employees. Response to Question 3: This response immediately raised the question in our mind: What is a threat compared to normal challenges inherent to working as an I-O psychologist today? Although “resistance to change” might make it harder to do a successful intervention, the fact that change is difficult is part of the reason people need I-O psychologists. If things were easy, companies wouldn’t need us, they would just need emails from CEOs telling employees what to do and when. Perhaps we were hoping for something more dramatic when we posed this question (an asteroid that wipes out only I-O psychologists, for example). We found that these suggestions, although important, are mostly things that we should be, and have, been thinking about for years as a field. To nitpick further, ChatGPT does not explain why or how these are threats. What are the consequences of not addressing these? Is the threat more significant to those in practice versus academics? Taken together, ChatGPT’s ability to provide deeper insight or to forecast potential future issues is limited. We can, however, at least pat ourselves on the back as a field a little that we are, according to this response, fairly on the ball as to what to be concerned about (until the asteroid, of course). Theme 4: I-O-Related Brainstorming Question 4: Give Three Examples of Explanations of I-O Psychology That Would Help a Person of any Age Understand What It Is. Sure! Here are three explanations of I-O psychology that are easy to understand for people of any age: For kids: “I-O psychology is like having a superhero at work! It’s all about understanding how people think and behave in jobs and how to make workplaces better. Just like superheroes help solve problems, I-O psychologists use their superpowers to make sure people are happy, work well together, and feel appreciated in their jobs.” For teens: “Think of I-O psychology as the science of work and people. It’s like having a secret formula to make workplaces awesome! I-O psychologists study how to choose the right people for jobs, how to make sure employees are treated fairly, and how to create a positive work environment where everyone can do their best. They help companies become successful by making sure employees are happy, motivated, and working together as a team.” For adults: “I-O psychology is a field that focuses on understanding and improving the workplace. It’s like the ‘people’ side of business. I-O psychologists help organizations by figuring out the best ways to hire, train, and support employees. They also look at how to make work environments fair, productive, and satisfying. Whether it’s selecting the right person for a job, improving teamwork, or designing effective training programs, I-O psychologists play a crucial role in making work better for everyone involved.” Response to Question 4: Now we are getting back into a territory that ChatGPT seems to do well in, namely explaining concepts at a broad level. The response for children, in particular, highlights ChatGPT’s ability to tailor a response for a particular audience. We would feel very comfortable using the response for kids as an age-appropriate explanation that focuses on emotions and relationships (and superheroes!) rather than more complex issues. The responses for teenagers and adults were also strong, escalating in complexity and remaining relatively age appropriate as well. Do note, however, that our question was “people of any age.” ChatGPT made its own judgment to target it at kids, teens, and adults. If we wanted one message for all ages, or really cared about different age ranges of adults, then this answer would not be helpful. A more detailed prompt would be needed for those situations. Question 5: What Is a Good Slogan for I-O Psychology? “Unlocking Potential, Driving Success: I-O Psychology at Work” “Elevate Performance, Empower People: I-O Psychology in Action” Response to Question 5: We ended up asking this question a second time out of sheer curiosity and found both responses strong given what we asked. Although these slogans do not necessarily capture the complexity or breadth of what we do, they are certainly catchy enough to capture the spirit of it while remaining brief enough to put on a coffee mug. Note they are in exactly the same format. So if you are looking to do multiple slogans, watch out for them ending up sounding similar in cadence and structure. Question 6: In 10 Words, What Is the Most Important Thing I-O Psychology Should Know About Using ChatGPT? Validate and interpret ChatGPT outputs carefully to ensure accurate understanding. Response to Question 6: This response captures our feelings on ChatGPT perfectly after conducting this exercise. We’ve learned that ChatGPT, while providing a broad understanding of topics and providing general guidelines, is (a) not entirely accurate and (b) often needs further questioning and interpretation. To this, at least, we can add that it is self-aware enough to warn us of this. So feel free to go out and test ChatGPT for your own uses. Experiment. But don’t expect it to replace your support staff (or you!) all that soon. References [Frescoj]. (2023, Mar 4). Could artificial intelligence replace IO psychology in the future (2-3 decades) [Online forum post]. Reddit. https://www.reddit.com/r/IOPsychology/comments/11i1l0h/could_artificial_intelligence_replace_io/ McAfee, P. (2023, Mar 4). ResearchRabbit—A potential literature review tool. [Post]. LinkedIn. https://www.linkedin.com/pulse/research-rabbit-possible-literature-review-tool-mcafee-phd-mba/?trackingId=WEuT5A4IRDKe9%2FgmAFPaZw%3D%3D Naidoo, L. J. (2023). Max. classroom capacity: ChatGPT shakes up I-O psych education. The Industrial Organizational Psychologist, 60 (4). https://www.siop.org/Research-Publications/Items-of-Interest/ArtMID/19366/ArticleID/7457/preview/true/Max-Classroom-Capacity-ChatGPT-Shakes-Up-I-O-Psych-Education%E2%80%A6 OpenAI. (2022, Nov 30). Introducing ChatGPT. https://openai.com/blog/chatgpt Schmidt, G. B. (2023, Mar 2). I have signed up for #ChatGPT and am asking it the real questions about #IOPsych. [Post]. LinkedIn. https://www.linkedin.com/feed/update/urn:li:activity:7037091532859953152/ Surovell, E. (2023, Feb 8). ChatGPT has everyone freaking out about cheating. It’s not the first time. Chronicle of Higher Education. https://www.chronicle.com/article/chatgpt-has-everyone-freaking-out-about-cheating-its-not-the-first-time Tadjbakhsh, N. (2023, Mar 1). I asked ChatGPT to explain what an I-O psychologist does in iambic pentameter. [Post]. LinkedIn. https://www.linkedin.com/feed/update/urn:li:activity:7036496162761113600/ Thibodeau, P. (2023, Mar 3). A question at HR Tech: Will ChatGPT take HR jobs? TechTarget. https://www.techtarget.com/searchhrsoftware/news/365532192/A-question-at-HR-Tech-Will-ChatGPT-take-HR-jobs Print 1124 Rate this article: 5.0 Comments are only visible to subscribers.