Buster's Beach House at the Seaport Village

Oceanaire hands down in terms of choice and service. Go when they have happy hour at the oyster bar. Its only from 5-6pm, M-F.

FYI: Just one thing to keep in mind too chowhounds....95% of anything caught from the sea and delivered fresh here to restaurants in SD comes from one company. So eating at Anthony's fish grotto and the Oceanaire are pretty much no different!
More than likely anything you order from one restaurant will not differ from another. This is in regards to the fresh fish or shellfish.
The 3 things that set them apart are: ambiance, service and preparation. Pricetag fluctiation is associated with all 3.

Every post here has some great places for you to choose from. The Marine Room is probably the most solid all round choice in La Jolla. It is a staple there and Chef Bernard is one of the true original great chefs in SD that has risen the bar for dining out here from years past.

However if you choose to step out of La Jolla and head downtown....Chef Deborah Scott has some nice restaurants. Her newest is Island Prime and has what I consider, the best view of San Diego from the bay hands down. The food isnt too bad either ;-) Just google it