I'm a software consultant for a software company, which basically means I spend a lot of time setting up, installing, configuring and troubleshooting software on client environments.
I bill a lot of time for troubleshooting problems in their environments, at the same rate as we charge for development. The decision to bill or not is usually dependent on the reason that troubleshooting is needed.
So, if troubleshooting is required because the client's environment and/or third party systems (webserver hosting our app, etc.) is poorly maintained/configured, then I bill. It's not my responsibility to maintain these systems and time spent troubleshooting them is an additional service provided to the client, in lieu of them having proper admins.
If troubleshooting is required because the client has asked me to set-up a third party component which I'm not familiar with then I bill, but the client should be aware beforehand that this isn't my area of expertise. It's not reasonable for the client to expect me to be an expert in all third party software.
If troubleshooting is required when setting up one of our packages because I'm unfamiliar with the package, I bill for the set-up but not the troubleshooting. I class this as time spent learning, which the client could reasonably expect to have been done prior to the contracted work.
If troubleshooting is required because it turns out there is a bug in our software, then I do not bill. The client has already paid for the software licenses and should be able to expect it to work without also paying additional troubleshooting and rework, etc.
The decision to bill or not does not depend on the duration of the troubleshooting for me; small investigations can add up to days of work. Also, clients can start to abuse you as a free IT support/admin service if they think they can get away with it, although this probably applies less in the case of freelance development.