As a designer myself, my experience would tell me to work with you to find an logo you do want to use. I'd have a difficult time working on a logo and not satisfying the client in the end. If you haven't expressed clearly what you do not like and why you feel the current revision is unsuitable you should do so in a polite and respectful manner. Simply opening the discussion may be very beneficial. I know it would be for any client of mine.
In addition, perhaps I'm wrong but from the wording of your question it would see you saw his portfolio then a final design... This should not be the case. There should be multiple steps in between those two to ensure he is on the right track. If that's not the case, perhaps you need to ask why you aren't seeing preliminary ideas and only final designs and push him back a bit to show ideas before working towards final images. This is the most common path taken - discussion - 3-5 roughs - discussion - additional roughs or 1-3 comps (based on roughs) - discussion - 1-2 refined comps - discussion - 1 final.
If you have explained and the designer is still not "getting" it, and you are still unhappy with the work, and you just want to move on, then I personally, as a designer, would retain the initial 50% deposit/payment and not invoice for the remaining 50% (and not deliver final files). I can't feel good about charging final payments if the work was not satisfactory to the client. In reality though, every effort would be made to find something which is satisfactory before I reach this point with any of my clients. You really should speak to the designer about this. They may feel a small fee is warranted, or they may feel that a "kill" fee equal to the amount you've already paid is fair. Only the designer can decide what they feel is amicable to them.
As @NivF007 posted, emails can be construed as a "meeting of the minds" and therefore a contract. Any good designer I know is going to seek resolution rather than trying to charge you for work you don't find worthwhile.
All that being posted, forgive me, but without knowing you or the designer, it could be assumed the difficulty lies with you and your expectations. If your expectations are above and beyond what would be considered "normal" and the designer has made every effort to meet them, you are still unhappy, and the designer is no longer interested in seeking a solution for you. Then in those cases, it's not uncommon for both parties to agree to walk away for fees already collected/paid. Again, I don't think anyone can tell you what is "right" or "fair" except the designer you are working with. If unreasonable demands caused the designer many, many more hours of work than was anticipated, he/she may feel warranted in charging you to cover that extra work.