EG Information
Training Missions
Knowledge Bank
|
|
Enigma Group's Code Bank
RSS cache scriptBy: Ausome1 - Date Submitted: 2008-09-25 18:59:59 <?php /** * RSS cache script * * Run this script via cron. I run mine every 15 minutes. This will cache rss feeds on your local * machine in /tmp naming the files rss1.cache, rss2.cache and so on. Have your rss reader access * your rss feeds that are locally stored. This will keep you from getting banned to that rss feed * if you're hitting it on every page load and save you and the rss supplier lots of bandwidth. * * This script also uses parallel processing. It will go out and grab the rss feeds all at once. * Lets say you want to gather data from 5 sites and each site takes 5 seconds to scrape. Gathering * each site sequentially would take 25 seconds (5 x 5) for the script to complete. Running the * gathering in parallel allows the script to finish in 5 seconds. The length of the longest scrape. * * @author Ausome1 */ // URLs of content you wish to cache. $urls = array('http://example1.com/rss.xml', 'http://example2.com/rss.xml', 'http://example3.com/rss.xml', 'http://example4.com/rss.xml', 'http://example5.com/rss.xml' ); // No need to alter anything past here. // Create cURL instances, set URL, and other appropriate options. for ($i = 1; $i <= $count; ++$i) { $handle = 'ch' . $i; } // Create the multiple cURL handle. // Add handles for ($i = 1; $i <= $count; ++$i) { $handle = 'ch' . $i; } $running = NULL; { // Retrieve content and write data to cache file. for ($i = 1; $i <= $count; ++$i) { $handle = 'ch' . $i; $data = 'data' . $i; $response_code = 'response_code' . $i; $$response_code = curl_getinfo($$handle, CURLINFO_HTTP_CODE ); if ($$data && $$response_code != '404') { file_put_contents('/tmp/rss' . $i . '.cache', $$data); } } // Close curl session. ?>
|
| Return to
php category list |
|
|
Who's Online
487 Guests, 100 Users TheRetech, ckryptix, Diznablo, Nicid1, Ios, viper0i0, rabbidmind, Nasrudin, CollapsingWalls, mehtaparag, bitstrike, jnony, C, Nusquam-Redono-Sapientia, bazcrown, saifulfaizan, The End, Ultraminor, psychomarine, st3alth, themastersinner, pgmrlink, login, lionaneesh, ishkur88, mahraja, Mac, chekifr, gandalf88, Vap0r, t0ast, tantrum6226, BnE, Distorted, Psiber_Syn, Ausome1, invas10n, oldgoat, freedaysbecumei, BinaryShinigami, Rex_Mundi, Red_beard, Strobeflux, s0m3nak3dguy, Descent, teehee, machupicchu, Genetix, Anandarl, NotMyOwn, thegamerdude, Godzila, popo12341234, RedEvolution, velocity_b, myne17, teto111, aVoid, Central-Gsm, 1101, JackalReborn, InjectioN, h4lted, c0re, DisPater, markt4death, splatta, Jackowacko, saint556, Pyron2312, Azerion, howsens, white.hat.gone.bad, vazzilly, pwunkz, Inverted, QuarterCask, Infernoe11, deskata, cr4ck3rj4ck, Blizer, jasonmax, j0sh, gwenwavor, N4g4c3N, spizeyboy, Network X, Uino59, Jae Cee, ianFDK, saykov, medhaavee, zofy, demonkiller410, Stumble, SaMTHG, kishore, Raze, helasraizam, Venom1019 |
| |
|
|
|
|
|