LinkedIn is a great source of professional data that can be invaluable for research, recruiting, sales prospecting, and more. However, LinkedIn’s terms of service prohibit scraping or bulk data collection from their site. So how can you legally collect LinkedIn data? The short answer is: with care and caution.
There are a few ways to legally scrape LinkedIn data without running afoul of their terms of service:
- Use the LinkedIn API (with limits)
- Manually browse and collect small amounts of data
- Use a compliant data provider
The key things to keep in mind are respecting opt-outs, not overloading LinkedIn with requests, and generally avoiding behavior that looks suspicious or abusive. With the right approach, it is possible to gather useful professional insights from LinkedIn without breaking the rules.
What does LinkedIn’s terms of service say about scraping?
LinkedIn’s User Agreement states:
“You agree that you will not:
…
– Copy, use, disclose or distribute any information obtained from the Services, whether directly or through third parties (such as search engines), without the consent of LinkedIn”
It also prohibits:
“…automated scraping the Services without LinkedIn’s prior consent…”
So in short – scraping or bulk downloading of data from LinkedIn is prohibited without LinkedIn’s explicit permission. This terms of service is similar to most major websites.
Approaches to legally scrape LinkedIn data
Here are some recommended approaches to gather LinkedIn data legally:
Use the LinkedIn API
LinkedIn offers an API that allows programmatic access to some profile data. Using the API requires registration as a LinkedIn developer and abiding by LinkedIn’s API terms of service. Key things to note:
– There are usage limits enforced to prevent abuse.
– You can only access public profile data that any user could view manually.
– Scraping beyond the intended API uses is prohibited.
The API provides a compliant way to get basic profile information from LinkedIn. But it has significant limitations for more advanced scraping projects.
Manually collect small amounts of data
The surest way to comply with LinkedIn’s terms is to manually browse profiles and record data one profile at a time. As long as you access information the same way an individual user would while browsing, this approach is perfectly legal. The downside is that manual collection is slow and labor intensive.
Here are some tips to stay compliant with manual collection:
– Access profiles organically through search not through automated sequential access.
– Add profiles to maintain a browsing history to look natural.
– Respect opt-outs and don’t contact people without consent.
– Collect only data needed for your purposes rather than indiscriminately downloading profiles.
Manual collection may be feasible for small scrapers or research studies. But it does not scale well to thousands or millions of profiles.
Use a compliant data provider
Several data vendors have access to LinkedIn data through compliant means such as the API. They can provide LinkedIn data legally for research, marketing, and recruiting uses. Examples include:
– Apollo.io
– Lusha
– RocketReach
– Hunter.io
– Clearbit Connect
The advantage of providers is that they handle the work of collecting, cleaning, and organizing LinkedIn data at scale. And they guarantee compliance with terms of service and data laws. But their data offerings can get expensive for larger projects.
In any case, make sure to vet potential data partners carefully. Confirm they are accessing data legally before purchasing.
Scraping methods to avoid
On the flip side, here are some common scraping approaches that violate LinkedIn’s terms and should be avoided:
Automated scraping bots
Operating scraping bots that rapidly crawl and download LinkedIn violates their prohibition on “automated scraping”. Even if well-intentioned, scrapers and bots put strain on their systems. And it becomes difficult to discern acceptable use from abuse. Avoid unleashing any bots or scrapers on LinkedIn directly.
Browser extensions
Browser extensions like Clickagy or Data Miner that auto-scroll and collect data from LinkedIn pages are prohibited by their terms. These tools indiscriminately copy data from the page and don’t obey access limits.
Copying data from Google
You may find chunks of LinkedIn profile data copied into Google search results. But just because that data appears in Google does not make it fair game to copy. LinkedIn prohibits downloading data from third-party sites like Google without permission.
Purchased databases
There are offers out there for bulk LinkedIn data files or account credentials. But these almost certainly violate LinkedIn’s terms in the data collection process. Purchasing such databases is risky and unethical.
Fake accounts
Creating fake LinkedIn accounts to connect with more users and export their data is strictly prohibited. LinkedIn monitors and bans fake accounts aggressively. Only use real accounts from actual users.
LinkedIn profile data you can collect
To recap, here is the profile data that can be collected legally with the proper methods:
– Public profile fields like name, headline, location
– Connections/network
– Profile photo
– Company name and industry
– Skills
– Education history
– Links to website, portfolio, etc.
Essentially most visible profile data is fair game. However, you need to collect it through compliant means with appropriate limits.
Some specific profile fields are off limits:
– Email addresses and contact info
– Detailed experience section and job descriptions
– Posts, articles, activity data
– Any data from private profiles
Avoid collecting private or contact data without direct consent. Focus just on the visible professional profile data needed for your purposes.
Compliant uses of LinkedIn data
Here are some examples of compliant uses of scraped LinkedIn data:
Academic research
Many professors and graduate students want to analyze LinkedIn data as part of academic studies. This can legally be done through manual collection or use of research data providers. But make sure your school is aware of data usage and you anonymize any published datasets.
Marketing research
Market researchers often want to analyze characteristics of professionals in an industry. Centralized professional data can offer valuable samples for B2B research. Just be careful to anonymize data and not misuse contact information.
Recruitment
LinkedIn is designed for recruiting, so using profile data to find job candidates is reasonable. Just focus on public profiles, don’t scrape on a large automated scale, and respect profile settings.
Sales prospecting
In a similar vein as recruiting, sales teams can legally use LinkedIn profiles for lead generation to identify potential prospects. But targeted outreach should be personalized and consent-based.
Reporting/analysis
You can generate aggregated analysis on professional demographics, skills, education, and other attributes using LinkedIn data. This can provide interesting industry insights without harming privacy.
Ethics of scraping LinkedIn profiles
Beyond strict legal compliance, using web data ethically should consider principles like:
– User privacy – Avoid collecting private details without consent
– Transparency – Don’t deceive LinkedIn about your methods
– Minimal data – Only collect what you need, not “as much as possible”
– Security – Store data securely and protect user identities
– Opt-out settings – Respect user preferences and don’t contact non-consenting people
– Rate limits – Control collection volume to not overload systems
– Attribution – Give LinkedIn credit as the data source if publishing results
Scraping carries risks of turning people into data commodities if not handled carefully. Consider more than just the law.
Scraping legal compliance checklist
To wrap up, here is a checklist of key principles for legally scraping LinkedIn:
- Limit data collection to visible public profiles
- Obtain data through manual browsing not automated scraping
- Use the API if possible with proper developer access
- Check opt-out and contact preferences
- Work only with trusted, compliant data partners
- Minimize scraping volume to necessary levels
- Anonymize and protect collected data
- Transparently explain data sourcing and ethics if publishing analyses
Following these responsible practices will allow you to extract value from LinkedIn legally and ethically. Just maintain restraint in your collection approach.
Conclusion
LinkedIn is a goldmine for professional insights but requires careful handling when scraping. Their restrictive terms of service aim to prevent abuse but still allow responsible data collection through proper channels. Manual browsing, using the API, or working with vendors enables legal access without harming the service. Just avoid automated mass scraping or bulk downloads, and treat profile data ethically. With some prudence and transparency, LinkedIn can be mined legally for research, marketing, recruiting, and other purposes to benefit both users and collectors.