Advanced Clay Uses
Prospecting With Clay
Back to Library
Find and Scrape Company Career Pages + Generate Messages that Reference Information Found

Find and Scrape Company Career Pages + Generate Messages that Reference Information Found

Scrape career pages and use OpenAI to generate messages referencing the site information.

Matthew Quan
3 min.

How to Create AI Messages by Scraping Career Pages

If you're a recruiter or a business strategist looking to make tailored, effective approaches, this step-by-step guide will show you how to use Clay and GPT to scrape career pages and send out personalized outbound to companies using the information found.

Step 1: Identify Target Companies
Select the companies you're interested in and pull up their official websites. Look specifically for the 'career' pages as these often provide insights into the company's values, culture, perks, and benefits.

Step 2: Use Get SiteMap URL Integration
With the domain of your target company, run the Get Site Map URL integration in Clay. This tool helps you map the company domain and look for specific keywords related to job hiring such as 'careers' and 'jobs'. This helps you find the right webpage containing the valuable information you need.

Step 3: Scrape Website
After finding the careers page, use the 'Scrape Website' integration to extract the information contained in the page. Map the website link from the Git Site Map into the Scrape Website integration. The result is a comprehensive body of text containing the company's values and hiring information.

Step 4: Summarize the information with ChatGPT
This extracted information can be large and difficult to digest. To make it easier, use our “Complete Prompt” integration to summarize it instead. This will make it easier to understand when crafting your AI messaging later.

Step 5: Input and Output Messages
Create an input message to the system providing context. For example, "You're my assistant. Help me research the values of different companies. Look at the info from their careers page and highlight the ideal character fit."
The second message should be as the user, where you will instruct the tool to complete a prompt in under eight words and include a prefix like "It looks like you're hiring for people who...". Make sure to quote the input text to make it clear.

Step 6: Analyze Outputs
Review the outputs given by the Conversion Chat GBT. These outputs will be the summarized information about the company's values and ideal employee characteristics. This information can then be used to tailor your approach whether you're reaching out as a recruiter or if you're promoting a tech or business solution.

Using Clay and GPT for researching company values can yield personalized insights that can give you an edge in your outreach efforts. Whether it's recruiting or sales, showing that you understand a company's core values and the traits they're looking for in their employees can go a long way in building fruitful relationships. Happy researching!

Tools Used
More like this
Explore More