Advanced Python Programming

Starting from managing files

As an experienced Python user, you have probably seen the with statement being used to open and read external files inside Python programs. Looking at this problem at a lower level, the operation of opening an external file in Python will consume a resource—in this case, a file descriptor—and your operating system will set a limit on this resource. This means that there is an upper limit on how many files a single process running on your system can open simultaneously.

Let's consider a quick example to illustrate this point further. Let's take a look at the Chapter11/example1.py file, as shown in the following code:

# Chapter11/example1.pyn_files = 10files = []for i in range(n_files): files.append(open('output1/sample%i.txt' ...