wget#

wget(url, filename=None, save=None, headers=None, params=None, data=None, prefix='http', convert=True, die=False, response='text', verbose=False)#

Download a single URL.

Alias to urllib.request.urlopen(url).read(). See also sc.download() for downloading multiple URLs. Note: sc.urlopen()/sc.wget() are aliases.

Parameters:

  • url (str) – the URL to open, either as GET or POST

  • filename (str) – if supplied, save to file instead of returning output

  • save (bool) – if supplied instead of filename, then use the default filename

  • headers (dict) – a dictionary of headers to pass

  • params (dict) – a dictionary of parameters to pass to the GET request

  • data (dict)

  • prefix (str) – the string to ensure the URL starts with (else, add it)

  • convert (bool) – whether to convert from bytes to string

  • die (bool) – whether to raise an exception if converting to text failed

  • response (str) – what to return: ‘text’ (default), ‘json’ (dictionary version of the data), ‘status’ (the HTTP status), or ‘full’ (the full response object)

  • verbose (bool) – whether to print progress

Examples:

html = sc.urlopen('wikipedia.org') # Retrieve into variable html
sc.urlopen('http://wikipedia.org', filename='wikipedia.html') # Save to file wikipedia.html
sc.urlopen('https://wikipedia.org', save=True, headers={'User-Agent':'Custom agent'}) # Save to the default filename (here, wikipedia.org), with headers
sc.urlopen('wikipedia.org', response='status') # Only return the HTTP status of the site
New in version 2.0.0: renamed from wget to urlopen; new arguments
New in version 2.0.1: creates folders by default if they do not exist
New in version 2.0.4: “prefix” argument, e.g. prepend “http://” if not present
New in version 3.1.4: renamed “return_response” to “response”; additional options
On this page