Can you tell if you are losing projects due to your price?
If you are getting as much work now as you were when the dollar was weaker, then I would not lower your rates.
However, if you are losing projects to competitors who are about $20 an hour cheaper than you, it may be a sign that you are too expensive and I would lower your rates to see if business picks up.
Personally, I would never feel "guilty" for charging more! You are taking all the risk to be in business for yourself and those who purchase your services get the benefit of your talent without having to support you as an employee. People won't pay more than they think something is worth.
The people who hire you can decide what they are willing to pay based on their budget, your value to them, and whether or not they can get something as good as they need for less money.
There is a concept called "marginal rate of return" -- As you raise prices, at some point the sales will drop. But since each sale is worth more, your income increases (more money for less work) - up to a point. As you lower prices, the number of sales will increase, but each sale is worth less so you may make more that way - up to a point.
The goal is to find that sweet spot where you are getting the most money for the least amount of work (or an amount of work you can handle).
I think you will also find when freelancing, the clients who pay more are actually less needy, better educated on what they want, easier to satisfy, and pay faster.