Waitrose is the top supermarket and has a great range of the sort of things you say you are looking for, eg Fudges' biscuits. There is a branch locator at http://www.waitrose.com/branches/index.aspx which will help you find one, and you can check out the products too.
You will also get excellent quality foods from Marks & Spencer. Its a while since I have been in one, but they used to have all the items you ask for.
Better than either of these will be to go into a good delicatessen. There is bound to be at least a couple on the main streets somewhere (sorry, its over 10 years since I was there). A quick search brought up www.picklesofharborne.co.uk but there will be plenty more.