Imagine that I bought some items online (they could be consumables, materials or the like) to a shop based in a non-EU country. The goods were sent by post and weren't stopped at customs, so I haven't paid VAT on them. Can I still claim them as a business expense, if I'm a self-employed that doesn't issue VAT? (I don't know if it's relevant, just in case it is)
Yes, you can. You only need to issue VAT when you're VAT registered, which is what you do when you earn over a certain amount. Im not sure what it is, but its pretty high. I think you might be getting mixed up with the fees of taxes here. Claiming on expenses means you wont pay tax on what you earnt because the cost of that item was not considered profit. The amount of tax youpay on goods doesn;t really have to do with anything regarding your own earnings.
As has been said, you can claim that purchase as a business expense. But you will have to convert the cost of the purchase from the purchase currency to sterling. And whatever conversion rate you use will have to be used for all purchases in that currency during that tax year: so if you use the rate on the day you made the purchase, you will need to use that even if the rate changes later in the tax year. You only have to pay VAT if you’re VAT registered.