Evaluating third-party firms requires a somewhat different approach, since you're looking to form a relationship with a whole company instead of an individual. That said, if the firm is comprised of a couple of people, I'd interview them as I would an employee, with less emphasis on the interpersonal stuff.
The number one thing to remember with a third-party firm—for example, an offshore contracting house or a web development firm—is to look for success on projects similar to yours. I find out as much as I can about the relevant projects and then I ask to talk to references at those companies. An inability to supply a reference is a red flag, and you need to check that. One other thing I check when examining the references is whether they've completed the project with the firm in question. If they're still actively engaged with the client I'm talking to as a reference, that client reference may hesitate to say anything bad about the firm even if they're not delivering because they won't want to risk making a bad situation worse to help out a stranger.
Table 5.2 shows a hit list of questions for references. These assume that you have an understanding of what the project was and how long it took.
Would you use them again?
Relationships with third-party firms for custom development are difficult to manage and often fail. That's just a fact, and you can double that failure rate for offshore firms. So a few negative ...