Utilities for Downloading Streaming Responses¶
- requests_toolbelt.downloadutils.stream.stream_response_to_file(response, path=None, chunksize=512)¶
Stream a response body to the specified file.
Either use the path provided or use the name provided in the Content-Disposition header.
Warning
If you pass this function an open file-like object as the path parameter, the function will not close that file for you.
Warning
This function will not automatically close the response object passed in as the response parameter.
If a path parameter is a directory, this function will parse the Content-Disposition header on the response to determine the name of the file as reported by the server, and return a file path in the specified directory. If no path parameter is supplied, this function will default to the process’ current working directory.
import requests from requests_toolbelt import exceptions from requests_toolbelt.downloadutils import stream r = requests.get(url, stream=True) try: filename = stream.stream_response_to_file(r) except exceptions.StreamingError as e: # The toolbelt could not find the filename in the # Content-Disposition print(e.message)
You can also specify the filename as a string. This will be passed to the built-in open() and we will read the content into the file.
import requests from requests_toolbelt.downloadutils import stream r = requests.get(url, stream=True) filename = stream.stream_response_to_file(r, path='myfile')
If the calculated download file path already exists, this function will raise a StreamingError.
Instead, if you want to manage the file object yourself, you need to provide either a io.BytesIO object or a file opened with the ‘b’ flag. See the two examples below for more details.
import requests from requests_toolbelt.downloadutils import stream with open('myfile', 'wb') as fd: r = requests.get(url, stream=True) filename = stream.stream_response_to_file(r, path=fd) print('{0} saved to {1}'.format(url, filename))
import io import requests from requests_toolbelt.downloadutils import stream b = io.BytesIO() r = requests.get(url, stream=True) filename = stream.stream_response_to_file(r, path=b) assert filename is None
Parameters: - response (requests.models.Response) – A Response object from requests
- path (str, or object with a write()) – (optional), Either a string with the path to the location to save the response content, or a file-like object expecting bytes.
- chunksize (int) – (optional), Size of chunk to attempt to stream (default 512B).
Returns: The name of the file, if one can be determined, else None
Return type: str
Raises:
- requests_toolbelt.downloadutils.tee.tee(response, fileobject, chunksize=65536, decode_content=None)¶
Stream the response both to the generator and a file.
This will stream the response body while writing the bytes to fileobject.
Example usage:
resp = requests.get(url, stream=True) with open('save_file', 'wb') as save_file: for chunk in tee(resp, save_file): # do stuff with chunk
import io resp = requests.get(url, stream=True) fileobject = io.BytesIO() for chunk in tee(resp, fileobject): # do stuff with chunk
Parameters: Raises: TypeError if the fileobject wasn’t opened with the right mode or isn’t a BytesIO object.
- requests_toolbelt.downloadutils.tee.tee_to_bytearray(response, bytearr, chunksize=65536, decode_content=None)¶
Stream the response both to the generator and a bytearray.
This will stream the response provided to the function, add them to the provided bytearray and yield them to the user.
Note
This uses the bytearray.extend() by default instead of passing the bytearray into the readinto method.
Example usage:
b = bytearray() resp = requests.get(url, stream=True) for chunk in tee_to_bytearray(resp, b): # do stuff with chunk
Parameters:
- requests_toolbelt.downloadutils.tee.tee_to_file(response, filename, chunksize=65536, decode_content=None)¶
Stream the response both to the generator and a file.
This will open a file named filename and stream the response body while writing the bytes to the opened file object.
Example usage:
resp = requests.get(url, stream=True) for chunk in tee_to_file(resp, 'save_file'): # do stuff with chunk
Parameters: