Aic @ sW d Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z y d d l m Z Wn! e k r d d l m Z n Xy d d l Z Wn e k re Z n Xe Z d d l m Z m Z m Z m Z m Z m Z m Z m Z m Z m Z m! Z! m" Z" m# Z# m$ Z$ d d l m% Z% m& Z& m' Z' m( Z( e j) d Z* d a, d e j- d d e d d Z. d Z/ d e0 f d YZ1 d e1 e f d YZ2 e j3 d Z4 d Z5 d dD d YZ6 d dE d YZ7 d Z8 d dF d YZ9 d e9 f d YZ: d e9 f d YZ; d e9 f d YZ< d Z= d e9 f d YZ> d dG d YZ? d! e? f d" YZ@ d# dH d$ YZA d% eA e9 f d& YZB d' eA e9 f d( YZC d) ZD d* dI d+ YZE d, e9 eE f d- YZF d. e9 eE f d/ YZG d0 e9 f d1 YZH d2 eH f d3 YZI eJ e d4 rd5 eH f d6 YZK n d7 e9 f d8 YZL d9 e9 f d: YZM d; ZN d< ZO d= ZP d> e9 f d? YZQ d@ e9 f dA YZR dB eR f dC YZS d S(J s! An extensible library for opening URLs using a variety of protocols The simplest way to use this module is to call the urlopen function, which accepts a string containing a URL or a Request object (described below). It opens the URL and returns the results as file-like object; the returned object has some extra methods described below. The OpenerDirector manages a collection of Handler objects that do all the actual work. Each Handler implements a particular protocol or option. The OpenerDirector is a composite object that invokes the Handlers needed to open the requested URL. For example, the HTTPHandler performs HTTP GET and POST requests and deals with non-error returns. The HTTPRedirectHandler automatically deals with HTTP 301, 302, 303 and 307 redirect errors, and the HTTPDigestAuthHandler deals with digest authentication. urlopen(url, data=None) -- Basic usage is the same as original urllib. pass the url and optionally data to post to an HTTP URL, and get a file-like object back. One difference is that you can also pass a Request instance instead of URL. Raises a URLError (subclass of IOError); for HTTP errors, raises an HTTPError, which can also be treated as a valid response. build_opener -- Function that creates a new OpenerDirector instance. Will install the default handlers. Accepts one or more Handlers as arguments, either instances or Handler classes that it will instantiate. If one of the argument is a subclass of the default handler, the argument will be installed instead of the default. install_opener -- Installs a new opener as the default opener. objects of interest: OpenerDirector -- Sets up the User Agent as the Python-urllib client and manages the Handler classes, while dealing with requests and responses. Request -- An object that encapsulates the state of a request. The state can be as simple as the URL. It can also include extra HTTP headers, e.g. a User-Agent. BaseHandler -- exceptions: URLError -- A subclass of IOError, individual protocols have their own specific subclass. HTTPError -- Also a valid HTTP response, so you can treat an HTTP error as an exceptional event or valid response. internals: BaseHandler and parent _call_chain conventions Example usage: import urllib2 # set up authentication info authinfo = urllib2.HTTPBasicAuthHandler() authinfo.add_password(realm='PDQ Application', uri='https://mahler:8092/site-updates.py', user='klem', passwd='geheim$parole') proxy_support = urllib2.ProxyHandler({"http" : "http://ahad-haam:3128"}) # build a new opener that adds authentication and caching FTP handlers opener = urllib2.build_opener(proxy_support, authinfo, urllib2.CacheFTPHandler) # install it urllib2.install_opener(opener) f = urllib2.urlopen('http://www.python.org/') iN( t StringIO( t unwrapt unquotet splittypet splithostt quotet addinfourlt splitportt splittagt toBytest splitattrt ftpwrappert splitusert splitpasswdt splitvalue( t localhostt url2pathnamet getproxiest proxy_bypassi c C s | s | s | r | d k r- t d n t sB t d n t j d t j d | d | } t d | d t } t | } nF | r t d | } t | } n" t d k r t a } n t } | j | | | S( NsD You can't pass both context and any of cafile, capath, and cadefaults SSL support not availablet cert_reqst cafilet capatht contextt check_hostname( t Nonet ValueErrort _have_sslt sslt _create_stdlib_contextt CERT_REQUIREDt HTTPSHandlert Truet build_openert _openert open( t urlt datat timeoutR R t cadefaultR t https_handlert opener( ( s /usr/lib64/python2.7/urllib2.pyt urlopen s$ c C s | a d S( N( R! ( R( ( ( s /usr/lib64/python2.7/urllib2.pyt install_opener s t URLErrorc B s e Z d Z d Z RS( c C s | f | _ | | _ d S( N( t argst reason( t selfR- ( ( s /usr/lib64/python2.7/urllib2.pyt __init__ s c C s d | j S( Ns <urlopen error %s>( R- ( R. ( ( s /usr/lib64/python2.7/urllib2.pyt __str__ s ( t __name__t __module__R/ R0 ( ( ( s /usr/lib64/python2.7/urllib2.pyR+ s t HTTPErrorc B sA e Z d Z e j Z d Z d Z e d Z d Z RS( sB Raised when HTTP error occurs, but also acts like non-error returnc C sV | | _ | | _ | | _ | | _ | | _ | d k rR | j | | | | n d S( N( t codet msgt hdrst fpt filenameR t _HTTPError__super_init( R. R# R4 R5 R6 R7 ( ( s /usr/lib64/python2.7/urllib2.pyR/ s c C s d | j | j f S( Ns HTTP Error %s: %s( R4 R5 ( R. ( ( s /usr/lib64/python2.7/urllib2.pyR0 s c C s | j S( N( R5 ( R. ( ( s /usr/lib64/python2.7/urllib2.pyR- s c C s | j S( N( R6 ( R. ( ( s /usr/lib64/python2.7/urllib2.pyt info s ( R1 R2 t __doc__R R/ R9 R0 t propertyR- R: ( ( ( s /usr/lib64/python2.7/urllib2.pyR3 s s :\d+$c C s_ | j } t j | d } | d k r@ | j d d } n t j d | d } | j S( s Return request-host, as defined by RFC 2965. Variation from RFC: returned value is lowercased, for convenient comparison. i t t Host( t get_full_urlt urlparset get_headert _cut_port_ret subt lower( t requestR# t host( ( s /usr/lib64/python2.7/urllib2.pyt request_host s t Requestc B s e Z d i d e d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d d Z d Z RS( c C s t | | _ t | j \ | _ | _ d | _ d | _ d | _ d | _ | | _ i | _ x* | j D] \ } } | j | | qm Wi | _ | d k r t | } n | | _ | | _ d S( N( R t _Request__originalR t _Request__fragmentR t typeRF t portt _tunnel_hostR$ t headerst itemst add_headert unredirected_hdrsRG t origin_req_hostt unverifiable( R. R# R$ RN RR RS t keyt value( ( s /usr/lib64/python2.7/urllib2.pyR/ s c C s^ | d d k rQ | d } t t d | rQ t | d | t | | Sn t | d S( Ni t _Request__r_t get_( t hasattrRH t getattrt AttributeError( R. t attrt name( ( s /usr/lib64/python2.7/urllib2.pyt __getattr__ s c C s | j r d Sd Sd S( Nt POSTt GET( t has_data( R. ( ( s /usr/lib64/python2.7/urllib2.pyt get_method s c C s | | _ d S( N( R$ ( R. R$ ( ( s /usr/lib64/python2.7/urllib2.pyt add_data s c C s | j d k S( N( R$ R ( R. ( ( s /usr/lib64/python2.7/urllib2.pyR` s c C s | j S( N( R$ ( R. ( ( s /usr/lib64/python2.7/urllib2.pyt get_data s c C s( | j r d | j | j f S| j Sd S( Ns %s#%s( RJ RI ( R. ( ( s /usr/lib64/python2.7/urllib2.pyR? s c C sV | j d k rO t | j \ | _ | _ | j d k rO t d | j qO n | j S( Ns unknown url type: %s( RK R R RI t _Request__r_typeR ( R. ( ( s /usr/lib64/python2.7/urllib2.pyt get_type s c C sR | j d k rK t | j \ | _ | _ | j rK t | j | _ qK n | j S( N( RF R R Rd t _Request__r_hostR ( R. ( ( s /usr/lib64/python2.7/urllib2.pyt get_host s c C s | j S( N( Rf ( R. ( ( s /usr/lib64/python2.7/urllib2.pyt get_selector' s c C sJ | j d k r( | j r( | j | _ n | | _ | j | _ | | _ d S( Nt https( RK RM RF RI Rf ( R. RF RK ( ( s /usr/lib64/python2.7/urllib2.pyt set_proxy* s c C s | j | j k S( N( Rf RI ( R. ( ( s /usr/lib64/python2.7/urllib2.pyt has_proxy3 s c C s | j S( N( RR ( R. ( ( s /usr/lib64/python2.7/urllib2.pyt get_origin_req_host6 s c C s | j S( N( RS ( R. ( ( s /usr/lib64/python2.7/urllib2.pyt is_unverifiable9 s c C s | | j | j <d S( N( RN t capitalize( R. RT t val( ( s /usr/lib64/python2.7/urllib2.pyRP < s c C s | | j | j <d S( N( RQ Rn ( R. RT Ro ( ( s /usr/lib64/python2.7/urllib2.pyt add_unredirected_header@ s c C s | | j k p | | j k S( N( RN RQ ( R. t header_name( ( s /usr/lib64/python2.7/urllib2.pyt has_headerD s c C s" | j j | | j j | | S( N( RN t getRQ ( R. Rq t default( ( s /usr/lib64/python2.7/urllib2.pyRA H s c C s) | j j } | j | j | j S( N( RQ t copyt updateRN RO ( R. R6 ( ( s /usr/lib64/python2.7/urllib2.pyt header_itemsM s N( R1 R2 R t FalseR/ R] Ra Rb R` Rc R? Re Rg Rh Rj Rk Rl Rm RP Rp Rr RA Rw ( ( ( s /usr/lib64/python2.7/urllib2.pyRH s( t OpenerDirectorc B sS e Z d Z d Z d Z d Z d e j d Z d d Z d Z RS( c C sM d t } d | f g | _ g | _ i | _ i | _ i | _ i | _ d S( Ns Python-urllib/%ss User-agent( t __version__t addheaderst handlerst handle_opent handle_errort process_responset process_request( R. t client_version( ( s /usr/lib64/python2.7/urllib2.pyR/ S s c C s t | d s( t d t | n t } xet | D]W} | d k rS q; n | j d } | | } | | d } | j d r | j d | d } | | d } y t | } Wn t k r n X| j j | i } | | j | <n] | d k r| } | j } n? | d k r3| } | j } n! | d k r; | } | j } n q; | j | g } | rt j | | n | j | t } q; W| rt j | j | | j | n d S( Nt add_parents% expected BaseHandler instance, got %rt redirect_requestt do_opent proxy_opent _i t errorR" t responseRE ( s redirect_requests do_opens proxy_open( RX t TypeErrorRK Rx t dirt findt startswitht intR R~ Rs R} R R t setdefaultt bisectt insortt appendR R| R ( R. t handlert addedt metht it protocolt conditiont jt kindt lookupR| ( ( s /usr/lib64/python2.7/urllib2.pyt add_handler^ sJ c C s d S( N( ( R. ( ( s /usr/lib64/python2.7/urllib2.pyt close s c G sR | j | d } x9 | D]1 } t | | } | | } | d k r | Sq Wd S( N( ( Rs RY R ( R. t chainR t meth_nameR, R| R t funct result( ( s /usr/lib64/python2.7/urllib2.pyt _call_chain s c C s t | t r! t | | } n" | } | d k rC | j | n | | _ | j } | d } x8 | j j | g D]! } t | | } | | } qx W| j | | } | d } x; | j j | g D]$ } t | | } | | | } q W| S( Nt _requestt _response( t isinstancet basestringRH R Rb R% Re R Rs RY t _openR ( R. t fullurlR$ R% t reqR R t processorR R ( ( s /usr/lib64/python2.7/urllib2.pyR" s" c C ss | j | j d d | } | r% | S| j } | j | j | | d | } | rZ | S| j | j d d | S( NRt t default_openR t unknownt unknown_open( R R} Re ( R. R R$ R R ( ( s /usr/lib64/python2.7/urllib2.pyR s c G s | d k r< | j d } | d } d | } d } | } n | j } | d } d } | | | f | } | j | } | r | S| r | d d f | } | j | Sd S( Nt httpRi i s http_error_%si t _errori Rt t http_error_default( R s https( R~ R ( R. t protoR, t dictR t http_errt orig_argsR ( ( s /usr/lib64/python2.7/urllib2.pyR s N( R1 R2 R/ R R R R t sockett _GLOBAL_DEFAULT_TIMEOUTR" R R ( ( ( s /usr/lib64/python2.7/urllib2.pyRy R s / c sR d d l f d } t } t t t t t t t t g } t t d ra | j t n t } xl | D]d } x[ | D]S } | | r t | | r | j | q q~ t | | r~ | j | q~ q~ Wqq Wx | D] } | j | q Wx | D] } | j | q Wx3 | D]+ } | | r=| } n | j | q W| S( s+ Create an opener object from a list of handlers. The opener will use several default handlers, including support for HTTP, FTP and when applicable, HTTPS. If any of the handlers passed as arguments are subclasses of the default handlers, the default handlers will not be used. iNc s t | j t f S( N( R t ClassTypeRK ( t obj( t types( s /usr/lib64/python2.7/urllib2.pyt isclass s t HTTPS( R Ry t ProxyHandlert UnknownHandlert HTTPHandlert HTTPDefaultErrorHandlert HTTPRedirectHandlert FTPHandlert FileHandlert HTTPErrorProcessorRX t httplibR R t sett issubclasst addR t removeR ( R| R R( t default_classest skipt klasst checkt h( ( R s /usr/lib64/python2.7/urllib2.pyR s2 t BaseHandlerc B s) e Z d Z d Z d Z d Z RS( i c C s | | _ d S( N( t parent( R. R ( ( s /usr/lib64/python2.7/urllib2.pyR s c C s d S( N( ( R. ( ( s /usr/lib64/python2.7/urllib2.pyR s c C s# t | d s t S| j | j k S( Nt handler_order( RX R R ( R. t other( ( s /usr/lib64/python2.7/urllib2.pyt __lt__ s ( R1 R2 R R R R ( ( ( s /usr/lib64/python2.7/urllib2.pyR s R c B s# e Z d Z d Z d Z e Z RS( s Process HTTP error responses.i c C sd | j | j | j } } } d | k o7 d k n s` | j j d | | | | | } n | S( Ni i, R ( R4 R5 R: R R ( R. RE R R4 R5 R6 ( ( s /usr/lib64/python2.7/urllib2.pyt http_response s ( R1 R2 R; R R t https_response( ( ( s /usr/lib64/python2.7/urllib2.pyR s R c B s e Z d Z RS( c C s" t | j | | | | d S( N( R3 R? ( R. R R7 R4 R5 R6 ( ( s /usr/lib64/python2.7/urllib2.pyR - s ( R1 R2 R ( ( ( s /usr/lib64/python2.7/urllib2.pyR , s R c B s: e Z d Z d Z d Z d Z e Z Z Z d Z RS( i i c C s | j } | d k r$ | d k s< | d k r | d k r | j d d } t d | j j D } t | d | d | j d t St | j | | | | d S( s Return a Request or None in response to a redirect. This is called by the http_error_30x methods when a redirection response is received. If a redirection should take place, return a new Request to allow http_error_30x to perform the redirect. Otherwise, raise HTTPError if no-one else should try to handle this url. Return None if you can't but another Handler might. i- i. i/ i3 R_ t HEADR^ t s %20c s s3 | ]) \ } } | j d k r | | f Vq d S( s content-lengths content-typeN( s content-lengths content-type( RD ( t .0t kt v( ( s /usr/lib64/python2.7/urllib2.pys <genexpr>L s RN RR RS N( i- i. i/ i3 ( s GETR ( i- i. i/ ( Ra t replaceR RN RO RH Rl R R3 R? ( R. R R7 R4 R5 RN t newurlt mt newheaders( ( s /usr/lib64/python2.7/urllib2.pyR 8 s c C s d | k r" | j d d } n& d | k rD | j d d } n d St j | } | j sy t | } d | d <n t j | } t j | j | } | j } | j d p | j d p | j d s t | | | d | | | n | j | | | | | | } | d k r*d St | d r| j } | _ | j | d | j k syt | | j k rt | j | | j | | | qn i } | _ | _ | j | d d | | <| j | j | j j | d | j S( Nt locationi t urit /i s http://s https://s ftp://s) - Redirection to url '%s' is not allowedt redirect_dicti R% ( t getheadersR@ t patht listt urlunparset urljoinR? RD R R3 R R RX R Rs t max_repeatst lent max_redirectionst inf_msgt readR R R" R% ( R. R R7 R4 R5 RN R t urlpartst newurl_lowert newt visited( ( s /usr/lib64/python2.7/urllib2.pyt http_error_302Z sB so The HTTP server returned a redirect error that would lead to an infinite loop. The last 30x error message was: ( R1 R2 R R R R t http_error_301t http_error_303t http_error_307R ( ( ( s /usr/lib64/python2.7/urllib2.pyR 0 s " 8c C s t | \ } } | j d s0 d } | } nV | j d sR t d | n | j d d } | d k ry d } n | d | !} t | \ } } | d k r t | \ } } n d } } | | | | f S( s3 Return (scheme, user, password, host/port) given a URL or an authority. If a URL is supplied, it must have an authority (host:port) component. According to RFC 3986, having an authority component means the URL must have two slashes after the scheme: >>> _parse_proxy('file:/ftp.example.com/') Traceback (most recent call last): ValueError: proxy URL with no authority: 'file:/ftp.example.com/' The first three items of the returned tuple may be None. Examples of authority parsing: >>> _parse_proxy('proxy.example.com') (None, None, None, 'proxy.example.com') >>> _parse_proxy('proxy.example.com:3128') (None, None, None, 'proxy.example.com:3128') The authority component may optionally include userinfo (assumed to be username:password): >>> _parse_proxy('joe:password@proxy.example.com') (None, 'joe', 'password', 'proxy.example.com') >>> _parse_proxy('joe:password@proxy.example.com:3128') (None, 'joe', 'password', 'proxy.example.com:3128') Same examples, but with URLs instead: >>> _parse_proxy('http://proxy.example.com/') ('http', None, None, 'proxy.example.com') >>> _parse_proxy('http://proxy.example.com:3128/') ('http', None, None, 'proxy.example.com:3128') >>> _parse_proxy('http://joe:password@proxy.example.com/') ('http', 'joe', 'password', 'proxy.example.com') >>> _parse_proxy('http://joe:password@proxy.example.com:3128') ('http', 'joe', 'password', 'proxy.example.com:3128') Everything after the authority is ignored: >>> _parse_proxy('ftp://joe:password@proxy.example.com/rubbish:3128') ('ftp', 'joe', 'password', 'proxy.example.com') Test for no trailing '/' case: >>> _parse_proxy('http://joe:password@proxy.example.com') ('http', 'joe', 'password', 'proxy.example.com') R s //s proxy URL with no authority: %ri iN( R R R R R R R ( t proxyt schemet r_schemet authorityt endt userinfot hostportt usert password( ( s /usr/lib64/python2.7/urllib2.pyt _parse_proxy s 2 R c B s# e Z d Z d d Z d Z RS( id c C s | d k r t } n t | d s3 t d | | _ x= | j D]/ \ } } t | d | | | | j d qI Wd S( Nt has_keys proxies must be a mappings %s_openc S s | | | | S( N( ( t rR RK R ( ( s /usr/lib64/python2.7/urllib2.pyt <lambda> s ( R R RX t AssertionErrort proxiesRO t setattrR ( R. R RK R# ( ( s /usr/lib64/python2.7/urllib2.pyR/ s c C s | j } t | \ } } } } | d k r9 | } n | j | j r_ t | j r_ d S| r | r d t | t | f } t j | j } | j d d | n t | } | j | | | | k s | d k r d S| j j | d | j Sd S( Ns %s:%ss Proxy-authorizations Basic Ri R% ( Re R R Rg RF R R t base64t b64encodet stripRP Rj R R" R% ( R. R R RK t orig_typet proxy_typeR R R t user_passt creds( ( s /usr/lib64/python2.7/urllib2.pyR s N( R1 R2 R R R/ R ( ( ( s /usr/lib64/python2.7/urllib2.pyR s t HTTPPasswordMgrc B s8 e Z d Z d Z d Z e d Z d Z RS( c C s i | _ d S( N( t passwd( R. ( ( s /usr/lib64/python2.7/urllib2.pyR/ s c C s t | t r | g } n | | j k r: i | j | <n xV t t f D]H } t g | D] } | j | | ^ qW } | | f | j | | <qG Wd S( N( R R R R Rx t tuplet reduce_uri( R. t realmR R R t default_portt ut reduced_uri( ( s /usr/lib64/python2.7/urllib2.pyt add_password s (c C s | j j | i } xj t t f D]\ } | j | | } xA | j D]3 \ } } x$ | D] } | j | | rZ | SqZ WqG Wq" Wd S( N( NN( R Rs R Rx R t iteritemst is_suburiR ( R. R t authurit domainsR t reduced_authurit urist authinfoR ( ( s /usr/lib64/python2.7/urllib2.pyt find_user_password s c C s t j | } | d r@ | d } | d } | d p: d } n d } | } d } t | \ } } | r | d k r | d k r i d d 6d d 6j | } | d k r d | | f } q n | | f S( s@ Accept authority or URI and extract only the authority and path.i i i R iP R i Ri s %s:%dN( R@ t urlsplitR R Rs ( R. R R t partsR R R RF RL t dport( ( s /usr/lib64/python2.7/urllib2.pyR ( s c C si | | k r t S| d | d k r( t St j | d | d f } t | t | d k re t St S( sc Check if test is below base in a URI tree Both args must be URIs in reduced form. i i ( R Rx t posixpatht commonprefixR ( R. t baset testt common( ( s /usr/lib64/python2.7/urllib2.pyR ? s ( R1 R2 R/ R R R R R ( ( ( s /usr/lib64/python2.7/urllib2.pyR s t HTTPPasswordMgrWithDefaultRealmc B s e Z d Z RS( c C sD t j | | | \ } } | d k r1 | | f St j | d | S( N( R R R ( R. R R R R ( ( s /usr/lib64/python2.7/urllib2.pyR P s ( R1 R2 R ( ( ( s /usr/lib64/python2.7/urllib2.pyR# N s t AbstractBasicAuthHandlerc B sD e Z e j d e j Z d d Z d Z d Z d Z RS( s1 (?:.*,)*[ ]*([^ ]+)[ ]+realm=(["']?)([^"']*)\2c C s= | d k r t } n | | _ | j j | _ d | _ d S( Ni ( R R R R t retried( R. t password_mgr( ( s /usr/lib64/python2.7/urllib2.pyR/ f s c C s d | _ d S( Ni ( R% ( R. ( ( s /usr/lib64/python2.7/urllib2.pyt reset_retry_countm s c C s | j | d } | j d k rB t | j d d | d n | j d 7_ | r t j j | } | r | j \ } } } | d k r t j d t d n | j d k r | j | | | } | r | j d k r d | _ n | Sq n d S( Ni i s basic auth failedi t "t 's Basic Auth Realm was unquotedi t basici ( R( R) ( Rs R R% R3 R? R$ t rxt searcht groupst warningst warnt UserWarningRD t retry_http_basic_authR4 ( R. t authreqRF R RN t moR R R R ( ( s /usr/lib64/python2.7/urllib2.pyt http_error_auth_reqedp s" c C s | j j | | \ } } | d k r d | | f } d t j | j } | j j | j d | k rr d S| j | j | | j j | d | j Sd Sd S( Ns %s:%ss Basic %sR% ( R R R R R R RN Rs t auth_headerRp R R" R% ( R. RF R R R t pwt rawt auth( ( s /usr/lib64/python2.7/urllib2.pyR1 s N( R1 R2 t ret compilet IR+ R R/ R' R4 R1 ( ( ( s /usr/lib64/python2.7/urllib2.pyR$ X s t HTTPBasicAuthHandlerc B s e Z d Z d Z RS( t Authorizationc C s2 | j } | j d | | | } | j | S( Ns www-authenticate( R? R4 R' ( R. R R7 R4 R5 RN R# R ( ( s /usr/lib64/python2.7/urllib2.pyt http_error_401 s ( R1 R2 R5 R> ( ( ( s /usr/lib64/python2.7/urllib2.pyR< s t ProxyBasicAuthHandlerc B s e Z d Z d Z RS( s Proxy-authorizationc C s2 | j } | j d | | | } | j | S( Ns proxy-authenticate( Rg R4 R' ( R. R R7 R4 R5 RN R R ( ( s /usr/lib64/python2.7/urllib2.pyt http_error_407 s ( R1 R2 R5 R@ ( ( ( s /usr/lib64/python2.7/urllib2.pyR? s c C s} t j j d r; t d } | j | } | j | Sg t | D] } t t j d d ^ qH } d j | Sd S( s Return n random bytes.s /dev/urandomi i R= N( t osR t existsR" R R t ranget chrt randomt randranget join( t nt ft sR t L( ( s /usr/lib64/python2.7/urllib2.pyt randombytes s 1t AbstractDigestAuthHandlerc B sS e Z d d Z d Z d Z d Z d Z d Z d Z d Z RS( c C sO | d k r t } n | | _ | j j | _ d | _ d | _ d | _ d S( Ni ( R R R R R% t nonce_countt last_nonce( R. R ( ( s /usr/lib64/python2.7/urllib2.pyR/ s c C s d | _ d S( Ni ( R% ( R. ( ( s /usr/lib64/python2.7/urllib2.pyR' s c C s | j | d } | j d k rB t | j d d | d n | j d 7_ | r | j d } | j d k r | j | | Sn d S( Ni i s digest auth failedi i t digest( Rs R R% R3 R? t splitRD t retry_http_digest_auth( R. R5 RF R RN R2 R ( ( s /usr/lib64/python2.7/urllib2.pyR4 s c C s | j d d \ } } t t | } | j | | } | r d | } | j j | j d | k rn d S| j | j | | j j | d | j } | Sd S( NR i s Digest %sR% ( RQ t parse_keqv_listt parse_http_listt get_authorizationRN Rs R5 R Rp R R" R% ( R. R R8 t tokent challenget chalt auth_valt resp( ( s /usr/lib64/python2.7/urllib2.pyRR s c C s< t j d | j | t j t d f j } | d S( Ns %s:%s:%s:%si i ( t hashlibt sha1RN t timet ctimeRL t hexdigest( R. t noncet dig( ( s /usr/lib64/python2.7/urllib2.pyt get_cnonce s c C sd yK | d } | d } | j d } | j d d } | j d d } Wn t k r_ d SX| j | \ } } | d k r d S| j j | | j \ } } | d k r d S| j r | j | j | } n d } d | | | f } d | j | j f } | d k r| | j k r?| j d 7_ n d | _ | | _ d | j } | j | } d | | | | | | f } | | | | } nD | d k r| | | d | | | f } n t d | d | | | | j | f } | r | d | 7} n | r5| d | 7} n | d | 7} | r`| d | | f 7} n | S( NR R` t qopt algorithmt MD5t opaques %s:%s:%ss %s:%sR8 i s %08xs %s:%s:%s:%s:%ss qop '%s' is not supported.s> username="%s", realm="%s", nonce="%s", uri="%s", response="%s"s , opaque="%s"s , digest="%s"s , algorithm="%s"s , qop=auth, nc=%s, cnonce="%s"( Rs R t KeyErrort get_algorithm_implsR R R? R` t get_entity_digestRc Ra Rh RO RN Rb R+ ( R. R RX R R` Rc Rd Rf t Ht KDR R6 t entdigt A1t A2t ncvaluet cnoncet noncebitt respdigR ( ( s /usr/lib64/python2.7/urllib2.pyRU sV ! ( c sU | j } | d k r$ d n | d k r< d n f d } | f S( NRe c S s t j | j S( N( R[ t md5R_ ( t x( ( s /usr/lib64/python2.7/urllib2.pyR A s t SHAc S s t j | j S( N( R[ R\ R_ ( Rt ( ( s /usr/lib64/python2.7/urllib2.pyR C s c s d | | f S( Ns %s:%s( ( RJ t d( Rj ( s /usr/lib64/python2.7/urllib2.pyR E s ( t upper( R. Rd Rk ( ( Rj s /usr/lib64/python2.7/urllib2.pyRh < s c C s d S( N( R ( R. R$ RX ( ( s /usr/lib64/python2.7/urllib2.pyRi H s N( R1 R2 R R/ R' R4 RR Rb RU Rh Ri ( ( ( s /usr/lib64/python2.7/urllib2.pyRM s = t HTTPDigestAuthHandlerc B s# e Z d Z d Z d Z d Z RS( s An authentication protocol defined by RFC 2069 Digest authentication improves on basic authentication because it does not transmit passwords in the clear. R= i c C s? t j | j d } | j d | | | } | j | S( Ni s www-authenticate( R@ R? R4 R' ( R. R R7 R4 R5 RN RF t retry( ( s /usr/lib64/python2.7/urllib2.pyR> W s ( R1 R2 R; R5 R R> ( ( ( s /usr/lib64/python2.7/urllib2.pyRx M s t ProxyDigestAuthHandlerc B s e Z d Z d Z d Z RS( s Proxy-Authorizationi c C s2 | j } | j d | | | } | j | S( Ns proxy-authenticate( Rg R4 R' ( R. R R7 R4 R5 RN RF Ry ( ( s /usr/lib64/python2.7/urllib2.pyR@ d s ( R1 R2 R5 R R@ ( ( ( s /usr/lib64/python2.7/urllib2.pyRz _ s t AbstractHTTPHandlerc B s/ e Z d d Z d Z d Z d Z RS( i c C s | | _ d S( N( t _debuglevel( R. t debuglevel( ( s /usr/lib64/python2.7/urllib2.pyR/ m s c C s | | _ d S( N( R| ( R. t level( ( s /usr/lib64/python2.7/urllib2.pyt set_http_debuglevelp s c C s: | j } | s! t d n | j r | j } | j d s[ | j d d n | j d s | j d d t | q n | } | j r t | j \ } } t | \ } } n | j d s | j d | n xH | j j D]: \ } } | j } | j | s | j | | q q W| S( Ns no host givens Content-types! application/x-www-form-urlencodeds Content-lengths %dR> ( Rg R+ R` Rc Rr Rp R Rk R Rh R R R{ Rn ( R. RE RF R$ t sel_hostR t selt sel_pathR\ RU ( ( s /usr/lib64/python2.7/urllib2.pyt do_request_s s. c s | j } | s! t d n | | d | j | } | j | j t | j j t f d | j j D d d <t d j D | j r i } d } | k r | | | < | =n | j | j d | n y) | j | j | j | j Wn, t j k rS} | j t | n7 Xy | j d t } Wn t k r| j } n X| j | _ t j | d t } t | | j | j } | j | _ | j | _ | S( s Return an addinfourl object for the request, using http_class. http_class must implement the HTTPConnection API from httplib. The addinfourl return value is a file-like object. It also has methods and attributes including: - info(): return a mimetools.Message object for the headers - geturl(): return the original request URL - code: HTTP status code s no host givenR% c 3 s- | ]# \ } } | k r | | f Vq d S( N( ( R R R ( RN ( s /usr/lib64/python2.7/urllib2.pys <genexpr> s R t Connectionc s s' | ] \ } } | j | f Vq d S( N( t title( R R\ Ro ( ( s /usr/lib64/python2.7/urllib2.pys <genexpr> s s Proxy-AuthorizationRN t buffering( Rg R+ R% t set_debuglevelR| R RQ Rv RN RO RM t set_tunnelRE Ra Rh R$ R R R t getresponseR R R t recvt _fileobjectR R5 R? t statusR4 R- ( R. t http_classR t http_conn_argsRF R t tunnel_headerst proxy_auth_hdrt errR R7 RZ ( ( RN s /usr/lib64/python2.7/urllib2.pyR s@ , ) ( R1 R2 R/ R R R ( ( ( s /usr/lib64/python2.7/urllib2.pyR{ k s R c B s e Z d Z e j Z RS( c C s | j t j | S( N( R R t HTTPConnection( R. R ( ( s /usr/lib64/python2.7/urllib2.pyt http_open s ( R1 R2 R R{ R t http_request( ( ( s /usr/lib64/python2.7/urllib2.pyR s R R c B s, e Z d d d d Z d Z e j Z RS( i c C s& t j | | | | _ | | _ d S( N( R{ R/ t _contextt _check_hostname( R. R} R R ( ( s /usr/lib64/python2.7/urllib2.pyR/ s c C s% | j t j | d | j d | j S( NR R ( R R t HTTPSConnectionR R ( R. R ( ( s /usr/lib64/python2.7/urllib2.pyt https_open s N( R1 R2 R R/ R R{ R t https_request( ( ( s /usr/lib64/python2.7/urllib2.pyR s t HTTPCookieProcessorc B s2 e Z d d Z d Z d Z e Z e Z RS( c C s4 d d l } | d k r' | j } n | | _ d S( Ni( t cookielibR t CookieJart cookiejar( R. R R ( ( s /usr/lib64/python2.7/urllib2.pyR/ s c C s | j j | | S( N( R t add_cookie_header( R. RE ( ( s /usr/lib64/python2.7/urllib2.pyR s c C s | j j | | | S( N( R t extract_cookies( R. RE R ( ( s /usr/lib64/python2.7/urllib2.pyR s N( R1 R2 R R/ R R R R ( ( ( s /usr/lib64/python2.7/urllib2.pyR s R c B s e Z d Z RS( c C s | j } t d | d S( Ns unknown url type: %s( Re R+ ( R. R RK ( ( s /usr/lib64/python2.7/urllib2.pyR s ( R1 R2 R ( ( ( s /usr/lib64/python2.7/urllib2.pyR s c C sm i } x` | D]X } | j d d \ } } | d d k r[ | d d k r[ | d d !} n | | | <q W| S( s> Parse list of key=value strings where keys are not duplicated.t =i i R( i( RQ ( t lt parsedt eltR R ( ( s /usr/lib64/python2.7/urllib2.pyRS s c C s g } d } t } } x | D] } | r? | | 7} t } q n | r | d k r] t } q n | d k rr t } n | | 7} q n | d k r | j | d } q n | d k r t } n | | 7} q W| r | j | n g | D] } | j ^ q S( sp Parse lists as described by RFC 2068 Section 2. In particular, parse comma-separated lists where the elements of the list may include quoted-strings. A quoted-string could contain a comma. A non-quoted string could have quotes in the middle. Neither commas nor quotes count if they are escaped. Only double-quotes count, not single-quotes. R= s \R( t ,( Rx R R R ( RJ t rest partt escapeR t cur( ( s /usr/lib64/python2.7/urllib2.pyRT s4 c C s- y t j | SWn t j k r( d SXd S( N( R t gethostbynamet gaierrorR ( RF ( ( s /usr/lib64/python2.7/urllib2.pyt _safe_gethostbyname: s R c B s) e Z d Z d Z d Z d Z RS( c C sq | j } | d d k r` | d d !d k r` | j r` | j d k r` d | _ | j j | S| j | Sd S( Ni s //i R R t ftp( Rh RF RK R R" t open_local_file( R. R R# ( ( s /usr/lib64/python2.7/urllib2.pyt file_openB s , c C s| t j d k ru y7 t t j d d t j t j d t _ Wqu t j k rq t j d f t _ qu Xn t j S( NR i ( R t namesR R R t gethostbyname_ext gethostnameR R ( R. ( ( s /usr/lib64/python2.7/urllib2.pyt get_namesM s $ c C s[ d d l } d d l } | j } | j } t | } y t j | } | j } | j j | j d t } | j | d } t j t d | p d | | f } | r t | \ } } n | s | r(t | | j k r(| rd | | } n d | } t t | d | | SWn t k rJ} t | n Xt d d S( Nit usegmti s6 Content-type: %s Content-length: %d Last-modified: %s s text/plains file://t rbs file not on local host( t email.utilst mimetypesRg Rh R RA t statt st_sizet utilst formatdatet st_mtimeR t guess_typet mimetoolst MessageR R R R R R" t OSErrorR+ ( R. R t emailR RF R8 t localfilet statst sizet modifiedt mtypeRN RL t origurlR5 ( ( s /usr/lib64/python2.7/urllib2.pyR X s0 N( R1 R2 R R R R R ( ( ( s /usr/lib64/python2.7/urllib2.pyR @ s R c B s e Z d Z d Z RS( c C s d d l } d d l } | j } | s9 t d n t | \ } } | d k rc | j } n t | } t | \ } } | r t | \ } } n d } t | } | p d } | p d } y t j | } Wn" t j k r } t | n Xt | j \ } } | j d } t t | } | d | d } } | rg| d rg| d } n y/| j | | | | | | j } | rd pd } xM | D]E } t | \ } } | j d k r| d k r| j } qqW| j | | \ } } d } | j | j d } | r;| d | 7} n | d k rd| d k rd| d | 7} n t | } t j | } t | | | j SWn0 | j k r} t d | t j d n Xd S( Nis ftp error: no host givenR= R i i R; t DRK t at AR Rv s Content-type: %s s Content-length: %d s ftp error: %si ( R R R R; Rv R (! t ftplibR Rg R+ R R t FTP_PORTR R R R R R R R Rh RQ t mapt connect_ftpR% R RD Rw t retrfileR R? R R R R t all_errorst syst exc_info( R. R R R RF RL R R R5 R t attrst dirst filet fwRK R[ RU R7 t retrlenRN R t sf( ( s /usr/lib64/python2.7/urllib2.pyt ftp_openu s\ ! c C s% t | | | | | | d t } | S( Nt persistent( R Rx ( R. R R RF RL R R% R ( ( s /usr/lib64/python2.7/urllib2.pyR s ( R1 R2 R R ( ( ( s /usr/lib64/python2.7/urllib2.pyR t s 5t CacheFTPHandlerc B s> e Z d Z d Z d Z d Z d Z d Z RS( c C s1 i | _ i | _ d | _ d | _ d | _ d S( Ni i< i ( t cacheR% t soonestt delayt max_conns( R. ( ( s /usr/lib64/python2.7/urllib2.pyR/ s c C s | | _ d S( N( R ( R. t t( ( s /usr/lib64/python2.7/urllib2.pyt setTimeout s c C s | | _ d S( N( R ( R. R ( ( s /usr/lib64/python2.7/urllib2.pyt setMaxConns s c C s | | | d j | | f } | | j k rJ t j | j | j | <n< t | | | | | | | j | <t j | j | j | <| j | j | S( NR ( RG R R] R R% R t check_cache( R. R R RF RL R R% RT ( ( s /usr/lib64/python2.7/urllib2.pyR s " c C s t j } | j | k rr xT | j j D]@ \ } } | | k r+ | j | j | j | =| j | =q+ q+ Wn t | j j | _ t | j | j k rxD | j j D]3 \ } } | | j k r | j | =| j | =Pq q Wt | j j | _ n d S( N( R] R R% RO R R t mint valuesR R ( R. R R R ( ( s /usr/lib64/python2.7/urllib2.pyR s c C sB x! | j j D] } | j q W| j j | j j d S( N( R R R t clearR% ( R. t conn( ( s /usr/lib64/python2.7/urllib2.pyt clear_cache s ( R1 R2 R/ R R R R R ( ( ( s /usr/lib64/python2.7/urllib2.pyR s ( ( ( ( ( ( (T R; R R[ R R RA R RE R9 R R R] R@ R R. t cStringIOR t ImportErrorR Rx R R t urllibR R R R R R R R R R R R R R R R R R t versionRz R R! R R) R* t IOErrorR+ R3 R: RB RG RH Ry R R R R R R R R R# R$ R< R? RL RM Rx Rz R{ R RX R R R RS RT R R R R ( ( ( s /usr/lib64/python2.7/urllib2.pyt <module>L s ^" r 'i H-@ ? n + 4<