Python Crawler

This IMO is one of the dirtiest way for me up till now to accomplish a requirement. But it does the job I want. :D

_ DISCLAIMER: I’m a learner. There must be better, smarter and easier way to accomplish the same task._

rrs@learner:~/My_Documents/My Books $ cat
/home/rrs/devel/eclipse/PythonFun/web_pattern_fetcher.py  

#!/usr/bin/env python  
  
"""  
This tiny little script does the job of crawling into Apache generated
directory listings  
and download scanning a specific pattern.  
I'm using it to download anything that apache shows as TXT or IMG.  
I'm sure others will be able to extend it more.  
"""  
  
import urllib, urllib2, string  
  
url = "http://www.wuppy.net.ru/Fun/"  
req = urllib2.Request(url)  
handle = urllib2.urlopen(req)  
  
x = 1  
data = ''  
  
while x:  
    data = ''  
    line = handle.readline()  
    if "[TXT]" in line or "[IMG]" in line:  
        word_list = line.split(' ')  
        word = word_list[4:5]  
        req_word = str(word)  
        # Break and take out the relevant data uri  
        begin_num = req_word.find(">")  
        end_num = req_word.find("</A" )  
        req_word = list(req_word)  
        while begin_num < end_num - 1:  
            final_word = string.lstrip( string.rstrip(str(req_word[begin_num+1:begin_num+2]), "']"), "['")  
            data += final_word  
            begin_num += 1  
            #data.append(req_word[begin_num+1:begin_num+2])  
        real_url = url + data  
        urllib.urlretrieve(real_url, data)  
    if line == '':  
        x = 0
Categories: Programming 

Pythonic Addiction

#!/usr/bin/env python  
  
def files(root):  
    for path, folders, files in os.walk(root):  
        for file in files:  
            yield path, file  
  
  
def find_match(repository): # aka walk_tree_copy()  
    for path, file in files(repository):  
        if file.endswith ('html') or file.endswith ('htm') or file.endswith ('HTML') or file.endswith ('HTM'):  
        #if file.endswith ('html.gz') or file.endswith ('htm.gz') or file.endswith ('HTML.gz') or file.endswith ('HTM.gz'):  
            try:  
                os.environ['__TEMP__VAL'] = file  
                os.chdir(path) # We need to chdir so that gzip can see the file in the cwd  
                os.system('gzip $__TEMP__VAL')  
                sys.stdout.write("%s/%s has been gzipped\n" % (path, file))  
            except IOError:  
                sys.stdout.write("Aieeeee.... I got some error with %s!\n\n" % (file))  
            continue  
            #return True  
    return False  
  
  
def main():  
    REPOSITORY = raw_input("Please enter a path to look for the files to zip.\nHit Return Key if you want the default path i.e. \"/home/rrs/My_Documents/My Books\"")  
  
    if REPOSITORY == '':  
        REPOSITORY = "/home/rrs/My_Documents/My Books/"  
  
    find_match(REPOSITORY)  
  
if __name__ == '__main__':  
    import os, sys, shutil  
    main() 
Categories: Fun Programming 

iprint - Andrew "Tridge" Tridgell's iprint in Python

Following code does the same thing that Tridge’s iprint.c does.

#!/usr/bin/env python  

import sys

"iprint - Andrew \"Tridge\" Tridgell's iprint in Python"

def toBinary(dec):  
"""  
This function does the coversion from integer to binary  
"""  
    bin = ''  
    while dec:  
        bin = (dec % 2 and '1' or '0') + bin  
        dec = long(dec/2)  
    return bin or '0'
 x = 1  
text = ''  
while len(sys.argv)  > 1 and x < len(sys.argv):  
    text += sys.argv[x]  
    x += 1

for x in text:  
   print ord(x), "\t", hex(ord(x)), "\t", oct(ord(x)), "\t", toBinary(ord(x)), "\t", x  
   print "\n"
Categories: Programming 

Python Fun

Just think what all this small piece of code can do

while x <= 100:
… temp = urllib2.urlopen(address+prefix+str(x)+suffix)
… data = open(prefix+str(x)+suffix, ‘wb’ )
… data.write(temp.read())
… data.close()
… temp.close()
… x +=1

Categories: Fun Programming 

Who says GUI Programming in shell is difficult ?

Today, I wanted a quick utility to help me enter passwords for encfs during mount.
I was looking for something plain, simple and userfriendly.
At the end I ended up to use the shell for it.

Yes, using dialog/Xdialog you can be a GUI programmer in minutes.

If the text is scrambled, you can download the script here.

Cheers!

#!/bin/bash

  1. encfsmount - A simple frontend for encfs
  2. (C) Ritesh Raj Sarraf July 2005

  1. Available only under the GPL

PATH="/bin:/sbin:/usr/bin:/usr/sbin:/home/rrs/bin"  
export PATH

XDIALOG_HIGH_DIALOG_COMPAT=1  
export XDIALOG_HIGH_DIALOG_COMPAT

  1. && exec sudo "$0" "$@"

ASKPASS=`which ssh-askpass`;  
CRYPT="/mnt/windoze/.linuxrc";  
TARGET="/var/tmp/.linuxrc";  
FUSERMOUNT=`which fusermount`;  
ENCFS=`which encfs`;  
ARGS="-u";

#set -vx

DIALOG=`which dialog`;  
[ -n "$DISPLAY" ] && [ -x /usr/bin/Xdialog ] && DIALOG="Xdialog"

mount()  
{ # This is one good way to use dialog to accept passwords # But for some
reason encfs isn't able to read the password # supplied by dialog # Hence, I'm
using ssh-askpass to get the password #$ENCFS $CRYPT $TMP --extpass=`$DIALOG
--title "Password" --password --inputbox "Enter Password" 18 45 2`;

# This uses ssh-askpass $ENCFS $CRYPT $TARGET --extpass=$ASKPASS;

}

check_mount()  
{ # Since encfs isn't returning good exit status # I'm using this hack to
figure out if the mount # passed or failed. grep -i encfs /etc/mtab;
retval=$?;

case $retval in 0) $DIALOG --wrap --title "Information"\ --msgbox
"Successfully Mounted!" 10 41;; 1) $DIALOG --wrap --title "Information"\
--msgbox "Mount Failed!" 10 41;;  
esac  
}

prelim_mount_check()  
{ grep -i encfs /etc/mtab; retval=$?;

case $retval in 0) $DIALOG --wrap --title "Caution"\ --yesno "ALREADY
MOUNTED!\nDo you want to umount ?" 0 0

case $? in 0) umount; esac;; 1) #$DIALOG --wrap --title "Information"\
#--yesno "Do you want to mount crypt ?"\ #case $? in # 0) mount; check_mount;;
#esac;; esac }

umount()  
{ $FUSERMOUNT -u $TARGET;  
}

ask_passwd()  
{ $DIALOG --wrap --title "Password"\ --help "Contact Author Ritesh Raj
Sarraf\nEmail: rrs@researchut.com"\ --inputbox "Please enter your password" 18
45 2 > /tmp/passwd.$$;

retval=$?;  
$INPUT=`cat /tmp/passwd.$$`; #rm -f /tmp/passwd.$$;

case $retval in 0) mount;; #echo "Password is '$input'";; 1) echo
"Cancelled";; 255) echo "Box Closed";;  
esac  
}

main()  
{ $DIALOG --wrap --title "Question"\ --help "Contact Author Ritesh Raj
Sarraf\nEmail: rrs@researchut.com"\ --yesno "Do you want to mount your
encrypted filesystem ?" 0 0

case $? in 0) prelim_mount_check;; 255) $DIALOG --wrap --title "Aborted"\
--infobox "Cancelled"  
esac  
}

main
Categories: Programming  Tags: shell