YouTube scraping: how to collect video data to study a topic
YouTube scraping done well: frame the search, prefer the official API, collect video and channel data with links, and compare views fairly over time.
Free app. Runs on your Mac with the Claude or Codex plan you already have.
What YouTube scraping is
YouTube scraping is the work of turning a set of videos into a table: title, channel, views, length, publish date, description and link for each one. Marketers use it to see which videos lead a topic and which creators an audience trusts. Product teams use it to study how people review a category. Content teams use it to find the questions no video answers well yet.
The method below is how a video researcher does it by hand. Each step says what to do and why, then how 10folds carries it out.
What it does for the business
- Find the creators your buyers watchThe channels that lead a topic, by views and by how often they appear, are the voices your customers already trust and a shortlist for sponsorship or partnership. Smaller channels with loyal, engaged viewers are worth as much attention as the largest names, and they are often easier to reach and work with.
- Learn how people judge your categoryReview and comparison videos show which features buyers notice, which problems they complain about and which competitors they compare you with.
- Plan videos that fill a gapWhen the leading videos on a question are old, short or off-target, a better video has room to rank. A table of what exists shows those gaps before you spend a day filming.
- Track a topic over timeRunning the same searches every quarter shows which channels are rising and which subjects are gaining attention.
YouTube scraping, step by step
Frame the topic and the searches
Your audience types questions. Search with their words.
What to do
- Write the question you want answered, for example which channels lead home espresso machine reviews.
- List three to five phrasings a viewer would type, including comparisons and how-to questions.
- Choose the region and language, since results differ by country.
- Decide how many results per search to keep, and whether Shorts count.
Why
YouTube's search results change with the phrasing, the viewer's country and the viewer's history. Several phrasings give a fuller picture than one, and the videos that appear under most of them are the ones that truly own the topic. A fixed number of results per search keeps runs comparable, and separating Shorts from long videos avoids mixing two formats that are watched in different ways. Writing the question first also tells you when to stop: once the leading channels repeat across searches, more results add little.
Choose the official route first
When a platform offers a front door, use it.
What to do
- Check whether the YouTube Data API covers your need. It returns search results, video details and channel statistics with an API key.
- Plan around its daily quota, a fixed number of units per project per day, where search requests cost more than detail lookups.
- If you collect from public pages instead, read YouTube's terms of service and robots.txt first.
- Keep a gentle rate, and never log in or rotate identities to get past limits.
Why
The YouTube Data API is the route YouTube permits for this data, and it returns clean, structured fields that do not break when the page layout changes. Its quota limits how much you can collect in a day, which is usually enough for research if you plan the searches in advance and fetch details only for the videos you keep. YouTube's terms restrict automated access to its pages, so any YouTube scraper that reads the pages directly carries more risk and breaks more often.
Collect video and channel fields with links
A view count is a reading on a meter. Write down when you read it.
What to do
- Record video ID, title, channel name and channel ID, publish date, length, view count and description.
- Add the channel's subscriber count where it is shown, to put views in context.
- Note the format of each video: a Short, a live recording or a regular upload.
- Keep the video link and the date you collected it on every row.
- Leave out comments and commenter names unless your question needs them, since they are personal data under GDPR.
Why
The video ID and channel ID are stable identifiers, so they survive title changes and let anyone reopen the source. View counts grow every day, which makes the collection date part of the data. Descriptions often hold the most useful text: links to products, chapters and sponsors. Length matters too, because a twenty-minute review and a one-minute clip compete for different viewers and should be compared separately.
Clean the list and check relevance
A search result is a candidate, not a finding.
What to do
- Remove duplicates by video ID, since the same video appears under several searches.
- Read the titles and drop videos that only share a word with the topic.
- Keep counts as shown, and leave missing values empty instead of writing zero.
- Open a sample of rows and compare them with the live pages.
- Note which searches or pages failed, so the gaps are visible.
Why
Search results include loosely related videos, old uploads and channels from other countries. A quick relevance pass keeps the analysis on the topic. Hidden counts and failed pages must stay visible as gaps, or the table will rank a popular video at the bottom. Rounded figures such as 1.2M views are approximate, so keep the displayed text beside any number you convert and avoid conclusions that depend on small differences.
Compare fairly and find the gaps
Old videos win on totals. Ask what wins per day.
What to do
- Divide views by days since publishing to compare videos of different ages.
- Compare a video's views with its channel's subscribers to spot videos that outperformed.
- Count how often each channel appears across your searches.
- Watch the top videos and note what they cover and what they leave out.
- Hand over the table with the next decision: creators to contact, or videos to make.
Why
A video uploaded five years ago has had five years to collect views, so raw totals favor age. Views per day and views relative to subscribers show what is working now and which small channels punch above their size. The gaps, the questions no leading video answers well, are where a new video has the best chance. A table cannot judge quality, so the watching step is where most of the insight comes from: note the opening, the structure, the claims made and what viewers ask about in the description links and pinned notes.
Common mistakes
- Ranking videos by total viewsOlder videos always win. Use views per day.
- Searching one phrasingViewers ask the same thing in many ways, and each returns different videos.
- Ignoring the API quotaPlan searches so a run does not stop halfway through and leave a partial table that looks complete.
- Collecting commenters' details for no reasonComments are personal data; take only what the question needs.
- Dropping the video IDTitles change. The ID is how you find the video again.
What a good result looks like
- A one-line scope: searches, region, language, results per search and date.
- One row per video, with its ID, link, channel and collection date.
- Views, length, publish date and description as shown.
- Views per day added, so videos of different ages compare fairly.
- A short list of leading channels and of questions left unanswered.
And then what? Using the result
- Shortlist the channels that appear most often and outperform their size.
- Watch the top five videos and write down what a better video would add.
- Brief one video on the clearest gap.
- Repeat the same searches next quarter and compare.
When to run it
- Choosing creators for sponsorshipChannels that lead your topic, and small ones that outperform, make the shortlist.
- Planning a video seriesThe leading videos show what already exists and what is missing.
- Studying how buyers review a categoryReview and comparison videos show what people notice and complain about.
- Before launching a productVideos about the products you will compete with show which claims reviewers test and which features they praise or criticize, so your launch material can answer them.
Questions people ask
Is it legal to scrape YouTube?
YouTube's terms of service restrict automated access to its pages, and the YouTube Data API is the route it allows. Public video data is lower risk than personal data such as comments, but rules differ by country. Use the API where you can and get advice for commercial use at scale.
What data can you scrape from YouTube?
For each video: title, channel, publish date, length, view count, like count when shown and description. For each channel: name, subscriber count when shown and video count. Watch time and audience details are private to the channel owner.
How do you scrape YouTube search results?
The cleanest way is the YouTube Data API's search method, then a details request for each video's statistics. Reading the results page directly also works but breaks when the page changes and is restricted by the terms of service.
Should you use the YouTube Data API or scrape?
Use the API when its quota covers your need: it is allowed, stable and structured. Page collection is a fallback for fields or volumes the API does not provide, with more legal and technical risk.
How do you find YouTube creators in a niche?
Search the questions your buyers ask, keep the channels that appear across several searches, and compare their recent views with their subscriber counts. Then watch a few videos from each, since numbers cannot show whether an audience trusts a creator.
Does it download videos?
No. It saves a table of videos and their links.
More jobs in Scraping
Scrape YouTube videos, on your own Mac.Free app. Runs on your Mac with the Claude or Codex plan you already have. Your files stay on your computer.
Download for Mac