Advanced Python Programming

Writing files asynchronously

Most of the time, we would like to collect data by making requests to multiple websites, and simply printing out the response HTML code is inappropriate (for many reasons); instead, we'd like to write the returned HTML code to output files. In essence, this process is asynchronous downloading, which is also implemented in the underlying architecture of popular download managers. To do this, we will use the aiofiles module, in combination with aiohttp and asyncio.

Navigate to the Chapter18/example5.py file. First, we will look at the download_html() coroutine, as follows:

# Chapter18/example5.pyasync def download_html(session, url):    async with session.get(url, ssl=False) as res: filename = f'output/{os.path.basename(url)}.html' ...