The Indeed Hiring Lab assessed the ability of GPT-4o, a GenAI model developed by OpenAI, to perform more than 2,800 job skills. Each skill was assessed across three main areas: The ability of GenAI to provide theoretical knowledge related to the skill; The ability of GenAI to solve problems using the skill; and GenAI’s determination of the importance of physical presence in utilizing that skill. These three ratings were considered by the model as part of an ultimate assessment of the likelihood that GenAI could replace a human in performing any of those 2,800+ individual skills. You can find a downloadable PDF version of the report here.
Key Points
- Out of more than 2,800 work skills assessed by Indeed, none were deemed “very likely” to be replaced by GenAI. Out of five possible outcomes (very unlikely, unlikely, possible, likely, very likely), the majority (68.7%) of skills assessed were “very unlikely” or “unlikely” to be replaced by GenAI.
- Roughly a quarter of skills (28.5%) could possibly be replaced by GenAI in the future if businesses change some practices and the tools improve.
- Roughly a quarter of skills (28.5%) could possibly be replaced by GenAI in the future if businesses change some practices and the tools improve.
- Overall, GenAI is generally strong at providing theoretical knowledge of skills but less so at using skills to solve problems. As long as a skill requires significant hands-on execution (for example, “aviation” or “cooking” skills), the usefulness of GenAI will remain limited.
- Ultimately, the impact of GenAI on a given occupation will depend on the skills in that occupation that can or cannot be performed well by the technology, both today and in the future.
- In jobs with a high share of skills that require hands-on execution, including nursing, GenAI could help with some repetitive tasks (like documentation) and allow workers to refocus on the core skills necessary in these roles.
- In more stereotypical “office jobs,” including software development, GenAI may potentially be able to offer significant knowledge and solve modest problems, emphasizing the importance of continued upskilling and ongoing learning for human workers.
Despite rapid advances in generative artificial intelligence (GenAI) technology and fears of an impending mass displacement of human workers, human work skills won’t be easily replaced in the workforce any time soon. Today’s generation of GenAI tools are not “very likely” — literally — to replace a competent human worker in mastering and performing even a single one of thousands of common work skills identified by Indeed.
Every occupation requires a worker to utilize a mix of skills across three key areas: Obtaining, retaining, and providing theoretical knowledge related to those skills; using those skills to solve problems; and executing those skills, either in-person/manually or digitally. A successful human worker can seamlessly balance and adapt their skills across these dimensions in order to do their job. But a digital “worker” powered by GenAI, while potentially strong across one or even two of these measures, will struggle to achieve the same seamless balance across all three.
There is significant potential for these models to continue learning and one day replace humans for certain skills, but meaningful changes in digitalization and working norms will need to happen first. The tools can be an immense help for certain time-consuming tasks, such as summarizing dense texts or quickly generating highly polished images or audio. In the future, combined with advances in robotics, it’s possible the tools will become more adept at mimicking and mastering how humans manually apply their skills. But as it stands today, GenAI is best suited to applying its skills to help with relatively straightforward work tasks that require only modest problem solving and, most importantly, no hands-on execution.
An involved approach
Starting with Indeed’s universe of hundreds of millions of job postings, we identified more than 2,800 common work skills, from “account management” to “wound care.” With this detailed skills taxonomy, Hiring Lab then developed a sophisticated prompt framework to evaluate these skills using GPT-4o, a large language model developed by OpenAI. The model was asked to assess its own ability to perform a skill across the three dimensions noted above. Each of these three assessments was scored on a 5-point scale, with a rating of 1 indicating very limited or no ability to apply a skill across that dimension, and a 5 indicating a strong ability. Finally, ratings across those three dimensions were considered by the model as part of a final assessment asking it to rate the likelihood that GenAI could replace a human across any of those 2,800+ individual skills, on a similar 5-point scale (from 1 — “very unlikely,” to 5 — “very likely”).
This analysis is the culmination of an intense, months-long, human/AI collaboration that resulted in a highly specific, 1,000+-word prompt to ensure our digital colleague followed our human instructions to the letter. Engineering the prompt took many attempts to get right — the more complex the ask is of GenAI, the more complex and detailed the prompt needs to be. All findings were validated by human researchers, and if the results did not meet expectations, the prompt was adjusted accordingly. Once the prompt was finalized, the analysis was run 15 times and consolidated into a final rating to increase the robustness of the GenAI-derived responses. Ironically, this involved approach helped prove the headline findings from this research: Even as GenAI evolves and learns to complete demanding tasks, humans that oversee, guide, and correct GenAI-derived output will not easily be replaced.
Applied knowledge
Overall, GenAI rated its ability to provide theoretical knowledge somewhat highly, but its ability to solve problems slightly less so. For 79.7% of skills evaluated, GenAI scored itself a 4 (“good”) in its ability to provide theoretical knowledge. It said it could solve “moderate” problems (a 3 on the 5-point scale) using 70.7% of skills. Critically, physical execution was deemed “essential” (a 1 on the 5-point scale) or “highly necessary” (a 2 on the 5-point scale) for a small majority (54%) of skills analyzed. Taken together, there were no skills for which GenAI rated itself “very likely” to be able to replace a human worker. More than a quarter of skills (28.5%) could “potentially” be replaced by GenAI as it continues to improve and if certain changes to workplaces and/or working norms occur going forward. But overall, more than two-thirds of workplace skills analyzed (68.7%) were determined to be either “very unlikely” (a 1 on the 5-point scale) or “unlikely” (a 2 on the 5-point scale) to be replaced by GenAI.
Theoretically powerful
Our research shows that the ability to provide theoretical knowledge related to skills is currently GenAI’s greatest strength. Theoretical knowledge was the only dimension (aside from physical necessity) in which any of the 2,800+ skills analyzed received the strongest possible rating of 5. The models are “trained” on vast amounts of data over the course of several years, including data from academia, private databases, and other proprietary training information. Even in cases where the models have not been trained on the most recent data, they can often access search engines and look things up. These products can summarize, re-format, and display much of this knowledge in the blink of an eye, and as reference tools, they are likely to only keep improving.
But they are hardly perfect reference tools. While a basic web search today will simply regurgitate what has already been written elsewhere, GenAI is more likely to make new inferences and create new conclusions — sometimes erroneously. The large language models (LLMs) that underpin these tools have a strong theoretical backing in language and writing, but are also prone to leaps and lapses in logic and may make certain things up.
Many GenAI products are also designed to create images or audio that can be impressively realistic and sometimes weird. The data used to train audio/visual-generating tools has often been problematic, leading to images or audio that can reinforce negative racist and/or sexist stereotypes. GenAI also has no real sense of ethics of its own, and cannot know if its responses to a given prompt are morally “right” or “wrong.” Human supervision of the prompts that ask for these text, audio, and/or visual outputs — and careful review of the outputs themselves to ensure accuracy, realism, and/or appropriateness — is essential.
Hands-on problems
Beyond providing general knowledge, it is the relative ability of the models to use a skill to solve problems and the importance of physical execution in applying that skill that have the biggest impacts in determining the likelihood of GenAI replacing a human at that given skill. An ability to solve problems is a hallmark of human ingenuity, and the greater a GenAI model’s ability to use a given skill to solve problems, the greater its likelihood of replacing a human. For more than 7 in 10 skills assessed (70.7%), GenAI’s ability to solve problems was rated a 3 (“moderate”). For more than a quarter (27.7%) of skills rated a 3 for problem-solving, the likelihood of GenAI replacing a human was also rated a 3. And for the 2.8% of skills at which GenAI rated its problem-solving ability a 4 (“good”), it also assigned itself a human replacement rating of 4 (“likely”).
Let’s remember, these tools do not yet have physical “bodies” that would allow them to perform manual tasks, though that may change in the future alongside advances in robotics. But until then, the necessity of manual, hands-on execution of a given skill plays an outsized role in helping determine its ultimate likelihood of GenAI replacement. The necessity of physical execution was rated as only “slightly necessary” or “not necessary” for all 101 skills that received a “good” rating (a 4 on the 5-point scale) for problem-solving. This shows that even if the GenAI model’s ability to solve problems using a certain skill is relatively strong, as long as that skill requires hands-on execution GenAI will likely remain limited in its ability to replace a person. Of all skills analyzed, GenAI said it was “very unlikely” or “unlikely” to replace a human for almost a third (30.1%) of those skills where physical presence was determined to be “essential.”
Impact on jobs
The distribution of skills at which GenAI is potentially better or worse is not equal across the thousands of occupations performed by millions of workers every day. Many tech roles, for example, require a greater proportion of digital skills that require limited hands-on execution and that are more likely to be able to be done at a high level by a GenAI model. Care roles, including nursing or child care, have a greater mix of skills for which GenAI may be able to provide some knowledge, but which are also more likely to require physical execution.
This analysis found that it is “possible” or “likely” that GenAI could replace a human at more than 71% of skills commonly found in job postings for software development occupations, for example. At the other end of the spectrum, only about 32.9% of skills found in the typical job posting for a nurse were rated as possible or likely to be replaced by GenAI.
Generally, the lower the share of skills in a given job that require physical execution, the higher the share of skills that could possibly or likely be replaced by GenAI.
A strong ability to solve problems also correlates with a higher likelihood of eventually replacing a human. But currently, GenAI isn’t particularly strong at solving problems using skills found in many common jobs. Almost 78% of skills commonly found in a typical accounting occupation, for example, were rated as “possible” or “likely” to be replaced by GenAI, and GenAI said its problem-solving skills were “good” for 30.5% of skills common to accounting job postings. But accounting occupations are an outlier — it is the only occupation analyzed in which GenAI said it was at least “good” at problem solving for at least 30% of common skills. For most other occupations analyzed, the model’s problem-solving abilities were more limited and the share of skills likely to be replaced by GenAI was also lower. If GenAI models improve their problem-solving abilities for more skills within more jobs, it’s likely that the share of skills that may eventually be replaced in those jobs will also rise.
Common skills
Consider a nurse and a software developer. Both jobs typically require similar soft skills, including communication skills and leadership skills. But the skills specific to each job represent different kinds of skills at which GenAI has different abilities. GenAI is adept at learning and applying various coding languages like Python or JavaScript that are commonly found in software developer job postings, and in troubleshooting human-written code when needed. But while it may be able to help a nurse draw up an appropriate care plan for a patient, GenAI cannot actually administer physical care of any kind, including administering medication or conducting acute care.
On average, some 40.6% of software developer job postings require the skill “Java,” a coding language. GenAI gives itself comparatively high ratings for theoretical knowledge (4 — “good”), problem-solving (3 — “moderate”), and replacement potential (3 — “possible”) for Java. GenAI itself justifies its overall rating for the skill by saying, “The model can assist in solving coding problems, debugging, and generating code snippets, but complex problem-solving and integration require human expertise.”
Within nursing, the opposite is true. One of the most common skills found in nursing job postings is (unsurprisingly) “nursing,” found in 44.8% of nurse job postings, on average. For this specific skill, while GenAI gave itself a strong rating on theoretical knowledge (4 — “good”), it gave itself a comparatively low rating for solving problems (2 — “basic”). It deemed physical execution of the skill “essential,” with the simple explanation that, “Nursing requires physical presence for patient care, administering medication, etc.” Therefore, the replacement potential for the “nursing” skill was rated as “unlikely” (2) because, “The model can assist with information and advice but cannot replace the physical aspects of nursing.”
In this example, nursing requires more skills at which GenAI is comparatively worse at problem-solving and which require some hands-on execution, so nursing jobs ultimately have a lower share of skills that are likely or possible to be replaced by GenAI. Physical presence was determined to be “highly necessary” or “essential” for a majority (58%) of skills commonly found in the typical nursing job posting. Physical presence was deemed highly necessary or essential for just 13% of skills commonly found in job postings for software developers.
Unrealized potential
Accounting is a good example of an occupation with a high share of skills that are neither especially likely or especially unlikely to be replaced by GenAI. A majority (58%) of skills commonly found in accounting job postings have only the possibility (a rating of 3 on the 5-point scale) of being replaced by GenAI, and the ultimate impact of GenAI on the accounting field will be determined by a host of as-yet-unknown factors.
A lot of accounting work doesn’t require a physical presence — crunching numbers and determining profit/loss margins have long been strengths of digital software. But certain accounting businesses may require more hands-on work than others. For example, a small mom-and-pop accounting business may deal regularly with paper statements handed over by clients, or engage in regular face-to-face meetings with clients to discuss strategies, etc. In these cases, while GenAI may be adept at crunching the actual numbers, it cannot accept a batch of new documents or physically scan them once in possession of the business, nor can it laugh at a client’s joke.
The ability to solve problems at a high level also greatly influences the overall likelihood of replacement for the skills common to accountant jobs. More than two-thirds of skills (68%) fall into the “moderate” problem-solving bucket for accountants (a rating of 3), largely because GenAI begins to falter as problems become more complex. For example, “organizational skills” (found in 18.8% of accounting job postings, on average) were assigned a “moderate” problem-solving ability because, “The model can offer practical solutions for organizing tasks and managing time, but may struggle with highly complex or dynamic scenarios.” For the “accounting” skill (found in 74.8% of accounting job postings), it gave itself the same problem-solving rating because, “The model can solve moderate accounting problems and provide practical advice for common scenarios, but it may struggle with highly complex issues.” It is possible that as the tools evolve and learn more, their ability to solve those more complex problems will also improve. But it’s not definite. And so the ultimate likelihood of replacement for many of these skills remains only possible, and not (yet?) likely.
Looking ahead, heavy investments in digitization and electronic record-keeping — the kinds of investments that are likely to enable greater assistance from a GenAI helper — will be required to realize GenAI’s full potential for a number of businesses and industries, including accounting. Every enterprise will need to decide for itself if these kinds of investments are worthwhile for them. Understanding the tools’ strengths and weaknesses, as outlined in this analysis, can help business leaders determine which levers to pull and dials to turn to maximize their business and strike the right balance between human and digital help.
Conclusion
The biggest obstacle to widespread GenAI adoption may be the exacting nature of the tools themselves. Extracting as much useful information as possible from GenAI models can require a great deal of specificity and care when interacting with the tools. An answer generated in response to a loose prompt may change, by a little or a lot, when asked the same way a second time. More advanced queries, or asks for help with more advanced tasks, will require more-advanced prompt-writing skills. It is unrealistic to expect a meaningful share of workers to quickly learn the kinds of advanced prompt-engineering skills required to extract the most value from today’s GenAI models. Companies and academic institutions will need to invest heavily in developing and deploying large-scale training programs that teach basic and advanced GenAI-specific skills. GenAI will find its way into daily workflows as more software and job-specific apps directly incorporate GenAI models (examples include Microsoft’s Copilot, Salesforce’s Einstein, or Github’s Copilot products). Both approaches will take time.
The global workforce continues to age, and going forward the global labor market will need to keep growing productivity with fewer workers. GenAI tools that help maximize employee productivity are a necessary part of the solution to this challenge. They can potentially help fill skill gaps in certain industries and relieve some labor market tightness in areas where demand for certain types of jobs exceeds the ready supply of human workers. But this analysis shows that GenAI cannot be applied evenly across tight markets. If a market is short on skilled tech workers, GenAI is likely to be a bigger help than if a market is short on skilled healthcare workers. And the importance of physical necessity continues to loom large.
In jobs with a high share of skills that require hands-on execution, GenAI could help workers to refocus on the core skills necessary in these roles. In the healthcare field, it could be used to help document medical work, allowing a nurse or doctor to focus more on performing actual patient care or learning more about it. In more hands-off, stereotypical “office jobs,” GenAI may potentially be able to obtain the same level of knowledge and be able to solve modest problems at a level similar to an early-career professional. This fact alone emphasizes the importance of continued human upskilling and career-long continuous learning in pursuit of expert knowledge, ensuring that the tools will reinforce a competent human, not replace them.
The takeaway is the same for all workers: While GenAI is highly unlikely to take a person’s job, a person who knows how to use GenAI productively almost certainly will. Now is the time to explore these tools, learn how they function, evaluate their strengths and weaknesses, and assess how they could impact all types of work.
Methodology
This analysis utilized Indeed’s skill database, which currently contains more than 2,800 skills common in US job postings, and a definition of those skills. Using generative artificial intelligence itself, we evaluated how GenAI could support those skills in a job setting.
The prompt developed for this evaluation was revisited over several months. We examined GenAI’s abilities across three dimensions: Its ability to offer theoretical knowledge about the skill, its ability to apply that skill in problem-solving, and the significance of physical presence in effectively carrying out the skill. Each of these dimensions was rated on a 5-point scale, where a score of 1 represented minimal or no capability in that dimension, while a 5 signified high proficiency. These scores were then considered by the model in an ultimate determination of the overall likelihood that GenAI could replace human involvement across any of the 2,800+ individual skills, using a similar 5-point rating system.
We used the OpenAI API to send this data to GPT version gpt-4o-2024-05-13 and applied a temperature of 0. To increase the robustness of the results, we ran the full analysis 15 times. The final GenAI rating is a consolidation of the results of these 15 runs using the mode for each rating. In case of a tie, the lower value was taken, a conservative approach. All output was reviewed by the human authors and resulted in prompt revisions until all results were determined to be logical, robust, and repeatable.
We then joined these evaluations with job postings published on Indeed in the US between August 01, 2023, and July 31, 2024. We calculated the impact of GenAI across a cross-section of 16 occupations (and the skills mentioned in postings for these occupations) representing a wide range of the labor market, to calculate the impact of GenAI on certain common jobs.
