[Home] [Download] [Tools] [Toolbar] [Research] [Documentation] [Terms]

TubeKit is a toolkit for creating YouTube crawlers. It allows one to build one's own crawler that can crawl YouTube based on a set of seed queries and collect up to 16 different attributes. TubeKit assists in all the phases of this process starting database creation to finally giving access to the collected data with browsing and searching interfaces. In addition to creating crawlers, TubeKit also provides several tools to collect a variety of data from YouTube, including video details and user profiles.

The toolkit is implemented primarily using PHP and available to download from here.


Creating a YouTube crawler with TubeKit:
  1. Provide basic information (project name, directory to store the crawler, etc.).
  2. Set up the database.
  3. Select up to 16 different attributes to collect for a YouTube video.
  4. Set up various schedules for crawling.
  5. Access your crawler and enter seed queries.
     
Collecting data from YouTube without crawling:

TubeKit includes several tools, which lets one (1) extract YouTube video links from any webpage, (2) extract YouTube video data (author, keywords, genre, number of views, ratings, comments, etc.), (3) collect text comments for YouTube videos, and (4) extract a YouTube users' profile data.

We have also developed a Firefox toolbar for TubeKit, which lets one accomplish YouTube data collection listed above with just a click.

New: Beta 4 released, which now includes a Firefox toolbar, several enhancements, and bug fixes. The tools are also improved, and a new tool for collecting comments added. Update: current version is Beta 4.6, which includes a few bug fixes.
New: Need help getting started? Watch the video tutorial here.
New: Want to crawl YouTube to collect data and contextual information without downloading TubeKit or installing anything on your side? Try the new release of ContextMiner, which also supports sources such as blogs, Flickr, and Twitter.
Creative Commons License
TubeKit by Chirag Shah is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License. TubeKit was initially supported by NSF grant #IIS 0455970, and now run without any funding. So please consider making a small donation through PayPal button below if you found TubeKit helpful.
| Send comments | Creative Commons License Chirag Shah | Last update: February 8, 2012 |