So this is an interesting question and I decided to do a little research. It seems that non-resident employees (those who are residents of one country but work in another) are subject to double taxation (being taxed by both their country of residence and country of work) unless they obtain special dispensation by filing a lot of paperwork.
New Zealand
UK
The United States IRS has special tax codes specifically for U.S. residents who are employed by an international organization or foreign government.
Source
The UK also specifies particular tax requirements for employees in offshore areas, including the specific stipulation that "Most offshore areas are now designated as part of a particular country. Both employers and employees will almost certainly have to meet the taxation requirements of that country."
Source
More Info
Clearly the details of the taxation are going to be unique to each specific tax code of the two countries involved. Also note that these links apply to employees, not to independent contractors or suppliers (as Andrew mentioned). For contracting, you're actually purchasing a product/service so the taxation likely falls under trade agreements rather than employment tax law. Outsourcing to/from another country may be liable to import/export taxes. This blog post has some useful advice in it regarding international independent contractors, though hard numbers are again going to be impossible to lay down without research into the specific countries involved.
Your best bet is to talk to an accountant who is already familiar with these tax situations. Otherwise, spend a lot of time reading the tax code for each country where you plan to outsource.