Clay has partnered with The Kiln to setup a series of large-scale data test across mobile phone, work email, personal email, email verification, and more. Below, we explain the approach to these tests.
Sample Data Set
Our methodology focused on creating a rigorous, real-world assessment of email enrichment providers across European markets. Using Clay’s find companies & people capabilities, we compiled a diverse dataset of approximately 5,000 VP and C-suite contacts distributed across 15 European countries. The countries of focus for this test were the United Kingdom, Germany, France, Italy, Spain, Switzerland, Netherlands, Denmark, Sweden, Norway, Finland, Belgium, Austria, Luxembourg, and Poland.
Enrichment Inputs
We used first name, last name and company domain to run all of the work email enrichment providers inside of Clay.
Verification Methodology
There are a lot of ways to verify if an email is valid or not, for example:
- Compare it to a known source of truth (eg: your own customer database)
- Run email campaigns and monitor for replies & bounces
- Use a consensus-based approach with multiple trusted verification tools that take different technical approaches to verifying emails.
All of these methods have certain benefits and drawbacks. Our priority was to give customers a high-confidence indicator of which email providers to use, anywhere in the world.
In order to run a statistically significant data test in every region of the world, option 1 wasn’t an option, as it was not feasible to get a perfectly split source of truth with enough data in every region.
For option 2, running outreach campaigns can lead to a large degree of false positives, especially when reaching out to catch-all domains, as not all of them will send back a hard-bounce. So you could think an email is valid, but it actually isn’t.
So to get a representative sample size, and provide a high degree of confidence, we used a combination of all 3.
We first conducted a separate benchmark of 10 email verification services using a sample of 9,500 emails from actual email campaigns that included both bounced emails and confirmed replies. Based on these results, we selected the top five performing email verifiers that demonstrated effectiveness across both catch-all and non-catch-all domains. The providers selected were Findymail, Icypeas, Kitt AI, Listmint, and Catch-all Verifier. See those tests here:
We then implemented a consensus-based verification approach to evaluate each email returned by the enrichment providers. Emails were classified according to strict criteria:
- Invalid: Personal emails, emails not related to the person's current job, or emails flagged as invalid by four or more verifiers
- Valid: Emails confirmed as valid by two or more verifiers
- Ultra Valid: Emails confirmed as valid by four or more verifiers
To calculate the final confidence score for each provider, we applied a weighted formula: (Ultra Valid % × 70%) + (Valid % × 30%), prioritizing the highest quality email results while still accounting for generally valid addresses.
Clay has partnered with The Kiln to setup a series of large-scale data test across mobile phone, work email, personal email, email verification, and more. Below, we explain the approach to these tests.
Sample Data Set
Our methodology focused on creating a rigorous, real-world assessment of email enrichment providers across European markets. Using Clay’s find companies & people capabilities, we compiled a diverse dataset of approximately 5,000 VP and C-suite contacts distributed across 15 European countries. The countries of focus for this test were the United Kingdom, Germany, France, Italy, Spain, Switzerland, Netherlands, Denmark, Sweden, Norway, Finland, Belgium, Austria, Luxembourg, and Poland.
Enrichment Inputs
We used first name, last name and company domain to run all of the work email enrichment providers inside of Clay.
Verification Methodology
There are a lot of ways to verify if an email is valid or not, for example:
- Compare it to a known source of truth (eg: your own customer database)
- Run email campaigns and monitor for replies & bounces
- Use a consensus-based approach with multiple trusted verification tools that take different technical approaches to verifying emails.
All of these methods have certain benefits and drawbacks. Our priority was to give customers a high-confidence indicator of which email providers to use, anywhere in the world.
In order to run a statistically significant data test in every region of the world, option 1 wasn’t an option, as it was not feasible to get a perfectly split source of truth with enough data in every region.
For option 2, running outreach campaigns can lead to a large degree of false positives, especially when reaching out to catch-all domains, as not all of them will send back a hard-bounce. So you could think an email is valid, but it actually isn’t.
So to get a representative sample size, and provide a high degree of confidence, we used a combination of all 3.
We first conducted a separate benchmark of 10 email verification services using a sample of 9,500 emails from actual email campaigns that included both bounced emails and confirmed replies. Based on these results, we selected the top five performing email verifiers that demonstrated effectiveness across both catch-all and non-catch-all domains. The providers selected were Findymail, Icypeas, Kitt AI, Listmint, and Catch-all Verifier. See those tests here:
We then implemented a consensus-based verification approach to evaluate each email returned by the enrichment providers. Emails were classified according to strict criteria:
- Invalid: Personal emails, emails not related to the person's current job, or emails flagged as invalid by four or more verifiers
- Valid: Emails confirmed as valid by two or more verifiers
- Ultra Valid: Emails confirmed as valid by four or more verifiers
To calculate the final confidence score for each provider, we applied a weighted formula: (Ultra Valid % × 70%) + (Valid % × 30%), prioritizing the highest quality email results while still accounting for generally valid addresses.
.png)
.avif)
.avif)
.avif)
