Steve deeply believed that to drive excellence in an organization, you need not just talent but small teams, like that cap of a hundred people for the Mac team: When there was an essential need for another person, a current team member had to be moved to some other part of Apple.
As for his story of setting that limit because he couldn’t remember the names of more than a hundred people on a team—nobody was fooled. Though the actual number was arbitrary, his real motivation was that he could not keep closely in touch with the work being done if there were too many people for him to supervise personally.
When the iPhone was being developed, the iPhone team was quartered in a separate building, both to keep the team cohesive and to protect their work. But at the same time, a leader needs to recognize that the small teams have to be able to interact on lots of issues with people from the rest of the organization.