I'm from Quebec City and trust me, you definitely want to stay in Upper town. I suggest you stay near Chateau Frontenac or the Hilton. But like Maximilien said, old Québec City is very compact, so you'll be able to walk to the restaurant.
Here are my suggestion:
Ristaurante Il Teatro (upper town):
Le Saint-Amour (upper town):
Laurie Raphaël (lower town):
L'Astral (upper town):
Le Café du monde:
and of course, if you can afford, dinner at Le Chateau Frontenac is always romantic:
If you go to La Grande-Allée, I suggest you eat at Cosmos, Voodoo or Savigny. Try to avoid the other because you'll pay too much for what you'll get in your plate.
In Lower Town, I suggest Le Lapin Sauté or Le Cauchon D'Ingue.
Also, make sure you eat a "Poutine" at Ashton for lunch.
Have fun in Quebec