 |
| [Home] [Download] [Tools] [Research] [Documentation] [Forums] [Terms] |
| Tools |
| In addition to TubeKit, which incorporates a suit of tools to perform query-based YouTube crawling, we have developed a few small tools that lets one grab various forms of information off YouTube without running queries. |
| To use a tool, click on its name, which should display the code in your browser. Save that file. Substitute extension '.txt' with '.php' and you are good to go! Some of the tools require database support and/or additional tools. Database creation and configuration scripts are also provided below. |
| Tool | Usage | Notes |
|---|---|---|
| Extract YouTube video URLs | > php extractYTVideoURLs.php yturls.txt mylist.txt This PHP script takes a set of YouTube URLs (or URLs to almost any webpage), and extracts the embedded URLs that point to YouTube videos. You can use this generated list (here, in 'mylist.txt') to harvest various attributes about those video using the 'Harvest videos' or 'Download YouTube vidoes' tools. | You don't need any database support for this. Just put one URL per line in 'yturls.txt', provide the name of the output file (here, 'mylist.txt'), and run it. |
| Download YouTube videos | > php downloadYTVideos.php mylist.txt Copy and paste this code in a file save it as 'youtube-dl' in the current directory. Then put the URLs of YouTube videos in a text file and pass the name of that file as an argument on the command line. You can write these URLs manually, or use the output of the 'Extract YouTube videos URLs' tool. | You don't need any database support for this. You do need Python 2.4 or higher for running 'youtube-dl' that this PHP script uses to download videos. |
| Harvest videos | > php harvestYTVideos.php mylist.txt This PHP script lets you collect a bunch of attributes of a YouTube video. All you need to do is put the URLs of those videos in a text file and pass the name of that file as an argument on the command line. You can write these URLs manually, or use the output of the 'Extract YouTube videos URLs' tool. | The results of this harvesting go into a MySQL database. You need to get this database and appropriate table ready before you run this script. See below for database configuration. You also need a database connection file, MagpieRSS, and parseRSS.php file (see below). Make sure you open this script and set $magpieRSSLocation. |
| Harvest profiles | > php harvestYTProfiles.php This PHP script reads username handles from a table in which the data is collected by 'Harvest videos' tool, and collects a set of attributes from that user's profile. | You need to create a table in your MySQL database where the data collected with this script can be stored. See below for database configuration. You also need a database connection file (see below). |
| Parsers | Download MagpieRSS parser. Copy and paste this file and store as 'parseRSS.php'. | Store these files in the same directory from where you are running other scripts. |
| Database configuration | This text file explains how to prepare your MySQL database that you can use for the tools listed here. | This file has instructions only and you don't need to download this file. |
| Database connection | Save this file as 'connect.php'. | Store this file in the same directory from where you are running other scripts dependent on database connection. |