claybook

Scrape public databases with Clay's Chrome scraper

Automate data extraction from public directories and build ready-to-go lists without ever leaving Clay.

What we used

Clay (Free): to scrape paginated databases online and enrich results

free resources
need help
Ask a question in the slack community
Schedule a meeting with Clay expert
Sign up for free to Clay

Before we begin

Say goodbye to copy-pasting listings by hand. Now Clay automates it for you.

This Claybook shows you how to scrape data from public databases like Psychology Today, automatically generate profile URLs for each record using Clay’s integration with Google Search, and enrich those records with emails, phone numbers, or other data. 

Whether you’re building prospect lists or researching a niche segment, you can handle the entire workflow without leaving Clay.

How it works:

  • Copy the template to your Clay workspace
  • Identify and add the URL of the public database you want to scrape
  • Use a Clay formula to generate paginated URLs
  • Scrape each profile into a Clay table
  • Use Google Search to find personal or professional profile URLs
  • Further enrich results with work emails, phone numbers, and more

Steps

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Step 1:

Copy this template

First, copy this template in Clay. 

If you’re already a user, you’ll be directed to your workspace. If not, you’ll be prompted to create a free Clay account.

The interactive demos on the right will walk you through each step of this Claybook.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Step 2:

Find the URL of the database you want to scrape

Next, we need to identify the URLs of the public database you want Clay to scrape into your table. In this example, we’re targeting New York–based therapists listed in Psychology Today’s online directory.

First, locate the URL for the New York therapist listings on Psychology Today. Then, use a Clay formula to generate URLs for each page of that directory. Clay’s scraper will use these URLs to automatically pull in the data.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Step 3:

Find user profile via Google Search

Next, after the table automatically writes each individual therapist listing to a new table, using Clay’s native integration with Google Search, find the user profile URL of every therapist. Or in your case, the direct profile URL of every result in the database.

From here, you can further enrich results with work emails, phone numbers, and more.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Copy the template

Start scraping listings from public databases, directly in Clay. 

Simply copy this template, follow the steps above, and let Clay work its magic.

Copy this template

Need help?

If you have any issue, you can ask for help in Slack or schedule time with a Clay expert.

Learn with our resources

Check out Clay University, read our GTM blog, or try out our top templates to transform your growth ideas into outreach in minutes.

Clay branded illustration of a wall made out of 3D, colourful, clay bricks and a whimsical person opening a door in the wall to access a new world of opportunities.

Clay University

Master Clay in a few core lessons. Uplevel your data enrichment, craft better outbound messages, and book more meetings.

Learn with Clay

Go to market blog

Our best content on growing your GTM motion, from outbound sales automation to AI lead generation.

Explore our blog
Clay branded illustration made out of 3D clay objects showing  whimsical operators connecting cables to 3D Clay table

Integrations

Browse through all the integrations you can connect to Clay in seconds and discover what you can do!

Explore integrations
Clay branded illustration of a Clay table with colourful, 3D connectors made out of Clay around it

Templates

Power your outbound campaigns with our best templates.

Visit clay templates
Clay branded illustration of a messy tangle of 3D, colourful strings made out of clay being untangled into one, colourful, twisted string

Clay Experts

Hire a Clay Enterprise expert or find a Clay partner to help with your next project

Explore Clay experts

Clay's Chrome extension

Clay’s Chrome Extension makes it wicked easy to find leads almost anywhere on the internet

Explore Chrome Extension