DesignOps & ChatGPT — a faster and unbiased approach to job family creation

Introduction

The task of creating a job family within the realm of DesignOps presents unique challenges so I’ve been experimenting with ChatGPT to see how I could tackle two significant aspects of common tasks: Efficiency and Objectivity.

• Efficiency: Could ChatGPT help me analyze and summerize 150 job descritions to create a job family?
• Objectivity: Could ChatGPT assist me in avoiding any unconscious bias during the process?

A job family is traditionally curated by managers, often reflecting their own views or biases, consciously or not. This can potentially skew the definition and scope of certain roles based on unconscious stereotypes or preconceived notions.

In light of this, I thought that using ChatGPT when creating a Product Design job family could be transformative. Not only did I view completing the task faster, but I also imagined doing so with an increased level of objectivity. I think I achieved both.

ChatGPT can help mitigate biases — the data it uses is just data, offering an unbiased and objective perspective. (For the contrarians — Yes — ChatGPT, LLMs, and AI writ large can be biased … if trained improperly.) The quality of impartiality can prove invaluable in tasks such as crafting a job family, helping us ensure fairness and consistency across roles and levels.

Caveats before I begin

#1 You are foolish to rely entirely upon ChatGPT, but you are fool to ignore it.

#2 As per the foolish, I reviewed the output from each step and compared it against training data and further compared it against similar types of content from other sources.

#3 Absolutely ZERO intellectual property was used for this project. I trained ChatGPT on public content and then formatted the output to align with conventional job description formats.

Here we go!

Training ChatGPT to Create a Product Design Job Family

Step 1: Gathering Data

I scraped 20 job descriptions from LinkedIn, Indeed, and Glassdoor for each level within the Product Design job family: associate, mid-level, senior, lead, principal, manager, senior manager, and director. This provided a rich data set representing the job market’s current state. 20 seemed to provide practical significance for my experiment. I tried 40 for a single job description but it didn’t make much of a difference.

Step 2: Data Categorization

The information from each job description was organized into six data points: Job number, Salary range, Location, Summary, Responsibilities, and Qualifications. This structured format made it easier for ChatGPT to digest the information and gave us a clearer analysis.

Step 3: Feeding the Data to ChatGPT

Next, ChatGPT was given the 20 job descriptions per job level to read, comprehend, and retain information. It analyzed these descriptions to understand patterns, requirements, and expectations for each role.

PROMPT: I would like you to thoroughly read, comprehend, and retain information from 20 “Senior Product Designer” job descriptions. The job descriptions have 6 data points, in the following order: Job number, Salary range, Location, Summary, Responsibilities, and Qualifications. I will be providing the job descriptions to you one at a time and after each one, simply acknowledge with ‘Got it’.

A time consuming part of this training process was dealing with the ChatGPT character count and its proclivity to begin analysis before all of the job decriptiosn were uploaded. It took a couple hours to hone the training prompts. Pure frustration. Here is one of the workaround:

PROMPT: Here is another job description. Remember, I need you to thoroughly read, comprehend, and retain information. I will be providing them one at a time, and after each one, simply acknowledge with ‘Got it’.

Data Analysis with ChatGPT

Following the data ingestion and comprehension, I performed a few types of analysis using ChatGPT:

Initial Analysis

ChatGPT was used to analyze salary ranges, geographic locations, frequency of keywords, required experience, and education from the gathered data. This gave me an understanding of market trends and expectations, I would not use this to formally dictate comp.

PROMPT: Count the number of job descriptions that mention “remote” or “hybrid” or similar terms. Count the number of job descriptions that mention a specific location. List locations and count. I do not need to see how you made the calculations, create a section called Location and provide me with the final results. Count number of job descriptions that have a salary range. Calculate the average salary for job descriptions with salary ranges and ignore salary ranges that are unavailable. Identify the minimum and maximum salary figures from job descriptions with salary range. Create a list of 5 most common software mentioned in the job descriptions and calculate the frequency of each.

Creating a Composite Job Description

Based on the initial analysis, ChatGPT synthesized the job descriptions for each level into a single composite description that encapsulated the most common and essential requirements and responsibilities.

PROMPT: First create a composite job summary based on themes within each job summary. Next review the responsibilities for each of the job descriptions and create a composite list of 15 responsibilities, listed in descending order based on frequency. Next review the qualifications for each of the job descriptions and create a composite list of 15 Qualifications, listed in descending order based on frequency.

Creating the Job Family with ChatGPT

Once the analysis was done, I used ChatGPT to create a job family for product design roles:

1. Training ChatGPT on Each Level

ChatGPT was trained on each composite job description, allowing it to understand the unique requirements and duties of each level.

PROMPT: I would like you to thoroughly read, comprehend, and retain information from 3 leadership job descriptions: Product Design Manager, Senior Product Design Manager, and Director of Product Design. Each job description has the same 6 data points, in the following order: Job number, Salary range, Location, Summary, Responsibilities, and Qualifications. I will provide them one at a time. I do not need immediate analysis, just say “Got it”.

2. Creating a Linear Progression

Next, ChatGPT compared the composite job descriptions and rewrote them to create a linear progression from Product Design Manager to Director of of product Design. This clearly illustrated the growth path and expectations at each career stage.

PROMPT: Generate a table grid for all of the job descriptions. The job titles are the table columns and job details are the data points on the Y axis. The output should reflect a linear progression in responsibilities and skills across these roles.

3. Promotion Pathways

Finally, ChatGPT used its understanding of each job level to create a list of ways a person could be promoted from one level to the other. This clarified the advancement process and identified key areas for growth and development.

PROMPT: List 5 key behaviors and outcomes an employee can demonstrate to be considered for promotion from one level to the next.

The outstanding questions

So I created a job family and I created tangible pathways to promotion. Here are a few outstanding questions:

…can it be relied upon?

…What human factor would be lost if we handed it all over to ChatGPT and the like?

…Could there be a misconception that we are being “judged” by an AI?

…Could this even be acomplished in your organization? (Firewalls, etc.)

…Does your organization view ChatGPT as helpful augmentation, an unknown risk, or a clear and present danger?

Conclusion

This exploration into DesignOps & AI has revealed the innovative potential of ChatGPT to reshape the process of creating job families. ChatGPT’s innovation lies in its ability to bring both efficiency and objectivity to the task, providing a quicker, more impartial process, and sidestepping the unconscious biases and stereotypes that even well-intentioned managers might unknowingly incorporate.

The traditional creation of a job family, encompassing various roles and levels within a field, can be influenced by these human biases, leading to potential skewed definitions and role scopes. Through my experimentation with ChatGPT, I’ve found a way to combat this issue. It’s not just about increasing efficiency in DesignOps with ChatGPT; it’s about introducing a level of objectivity previously unattainable.