EG Information

Main Index
EG Manual
Disclaimer
Legal Information
Hall of Fame
Hall of Shame
Member Rankings
Members List
Meet the Staff

Training Missions

Read Me First New
Basic Skills
Realistic Scenarios
Cryptography
Software Cracking
Linux ELF Binary Cracking
Logical Thinking
Programming
Patching
Steganography
Deface This Wall
/dev/null
/dev/urandom

Knowledge Bank

Discussion Forums
Enigma Chat New
RSS Feeds RSS
Articles / Tutorials
Videos
Online EG MP3 Player Radio
Enigma Zine
Downloads
Tools New

Code Resources

Submit Code
Ajax
ASM
Bash
C
CPP
Csharp
Delphi
Haskell
Java
Javascript
Jython
Lisp
mIRC
MySQL
Perl
PHP
Python
QBASIC
VisualBasic

Hakipedia: An open collaborative for all your information security needs.

The Urinal

Click Here To Vote For EG!

Has Enigma Group Helped You? Then Help Us By Advertising For Us. Place One Of The Following Images On Your Site.

enigma group

enigma group

enigma group

enigma group

Enigma Group's Code Bank


RSS cache script

By: Ausome1  -  Date Submitted: 2008-09-25 18:59:59

  1. <?php
  2. /**
  3.  * RSS cache script
  4.  *
  5.  * Run this script via cron. I run mine every 15 minutes. This will cache rss feeds on your local
  6.  * machine in /tmp naming the files rss1.cache, rss2.cache and so on. Have your rss reader access
  7.  * your rss feeds that are locally stored. This will keep you from getting banned to that rss feed
  8.  * if you're hitting it on every page load and save you and the rss supplier lots of bandwidth.
  9.  *
  10.  * This script also uses parallel processing. It will go out and grab the rss feeds all at once.
  11.  * Lets say you want to gather data from 5 sites and each site takes 5 seconds to scrape. Gathering
  12.  * each site sequentially would take 25 seconds (5 x 5) for the script to complete. Running the
  13.  * gathering in parallel allows the script to finish in 5 seconds. The length of the longest scrape.
  14.  *
  15.  * @author Ausome1
  16.  */
  17.  
  18. // URLs of content you wish to cache.
  19. $urls = array('http://example1.com/rss.xml',
  20. 'http://example2.com/rss.xml',
  21. 'http://example3.com/rss.xml',
  22. 'http://example4.com/rss.xml',
  23. 'http://example5.com/rss.xml'
  24. );
  25.  
  26. // No need to alter anything past here.
  27. $count = count($urls);
  28.  
  29. // Create cURL instances, set URL, and other appropriate options.
  30. for ($i = 1; $i <= $count; ++$i)
  31. {
  32. $handle = 'ch' . $i;
  33. $$handle = curl_init();
  34. curl_setopt($$handle, CURLOPT_URL, $urls[$i - 1]);
  35. curl_setopt($$handle, CURLOPT_HEADER, 0);
  36. curl_setopt($$handle, CURLOPT_RETURNTRANSFER, 1);
  37. }
  38.  
  39. // Create the multiple cURL handle.
  40.  
  41. // Add handles
  42. for ($i = 1; $i <= $count; ++$i)
  43. {
  44. $handle = 'ch' . $i;
  45. curl_multi_add_handle($mh, $$handle);
  46. }
  47.  
  48. $running = NULL;
  49.  
  50. {
  51. curl_multi_exec($mh, $running);
  52. } while ($running > 0);
  53.  
  54. // Retrieve content and write data to cache file.
  55. for ($i = 1; $i <= $count; ++$i)
  56. {
  57. $handle = 'ch' . $i;
  58. $data = 'data' . $i;
  59. $response_code = 'response_code' . $i;
  60. $$data = curl_multi_getcontent($$handle);
  61. $$response_code = curl_getinfo($$handle, CURLINFO_HTTP_CODE);
  62. if ($$data && $$response_code != '404')
  63. {
  64. file_put_contents('/tmp/rss' . $i . '.cache', $$data);
  65. }
  66. curl_multi_remove_handle($mh, $$handle);
  67. }
  68.  
  69. // Close curl session.
  70. ?>
Return to php category list

Who's Online

487 Guests, 100 Users
TheRetech, ckryptix, Diznablo, Nicid1, Ios, viper0i0, rabbidmind, Nasrudin, CollapsingWalls, mehtaparag, bitstrike, jnony, C, Nusquam-Redono-Sapientia, bazcrown, saifulfaizan, The End, Ultraminor, psychomarine, st3alth, themastersinner, pgmrlink, login, lionaneesh, ishkur88, mahraja, Mac, chekifr, gandalf88, Vap0r, t0ast, tantrum6226, BnE, Distorted, Psiber_Syn, Ausome1, invas10n, oldgoat, freedaysbecumei, BinaryShinigami, Rex_Mundi, Red_beard, Strobeflux, s0m3nak3dguy, Descent, teehee, machupicchu, Genetix, Anandarl, NotMyOwn, thegamerdude, Godzila, popo12341234, RedEvolution, velocity_b, myne17, teto111, aVoid, Central-Gsm, 1101, JackalReborn, InjectioN, h4lted, c0re, DisPater, markt4death, splatta, Jackowacko, saint556, Pyron2312, Azerion, howsens, white.hat.gone.bad, vazzilly, pwunkz, Inverted, QuarterCask, Infernoe11, deskata, cr4ck3rj4ck, Blizer, jasonmax, j0sh, gwenwavor, N4g4c3N, spizeyboy, Network X, Uino59, Jae Cee, ianFDK, saykov, medhaavee, zofy, demonkiller410, Stumble, SaMTHG, kishore, Raze, helasraizam, Venom1019