Skip to main content
Python mechanize For Browsing
How to install Mechanize
we can install mechanize in two ways
Using pip :
pip install mechanize
Or download the mechanize distribution,open it and run it:
python setup.py install
Browsing with Mechanize
Here is an example on how to browse webpage in python program
import mechanize br = mechanize.Browser() br.open("http://www.example.com/")
Follow second link with element text matching regular expression
response1 = br.follow_link(text_regex=r"cheese\s*shop",nr=1)assert br.viewing_html() print br.title() print response1.geturl() print response1.info() # headers print response1.read() # body
To get the response code from a website, you can the response.code
from mechanize import Browser browser = Browser() response = browser.open('http://www.google.com') print response.code
Get all forms from a website
import mechanize br = mechanize.Browser()br.open("http://www.google.com/") for f in br.forms(): print f
Ignore robots.txt
br.set_handle_robots( False )
Google demands a user-agent that isn't a robot
br.addheaders = [('User-agent', 'Firefox')]
Retrieve the Google home page, saving the response
br.open( "http://google.com" )
Select the search box and search for 'foo'
br.select_form( 'f' ) br.form[ 'q' ] = 'foo'
Get the search results
br.submit()
Find the link to foofighters.com; why did we run a search?
resp = None
for link in br.links(): siteMatch = re.compile( 'www.foofighters.com' ).search( link.url ) if siteMatch: resp = br.follow_link( link ) break
Print the site
content = resp.get_data() print content
Comments
Post a Comment