Obviously (or not) you should decide before shopping what kind of bike you want. This decision is based on what kind of riding you want to do. Pricepoint's also important, as all bikes involve making compromises and price has a lot to do with what compromises will need to be made. I'm going to assume someone who wants to do medium-distance road riding. What's medium distance? 20-100 miles in a day. I'm also going to assume a high-three figures price.
I think that a classic racing bike is the ultimate road machine. They have low rolling resistance, lots of places to put your hands, and while they're more a result of evolution than design, the influencing factors have been speed, handling and comfort.
The first part to consider is the frame. There's a bunch of frame materials out there, and most of them have good and bad qualities. At this pricepoint, titanium and carbon fiber are out, so it's a choice between steel and aluminum. Aluminum has a reputation for giving a bit of a harsh ride, but it's also lighter for the same strength. Because aluminum is weaker by volume than steel, the tubes have somewhat thicker walls and considerably larger diameters, and aluminum frames are very stiff. There's also a minimum level of crafstmanship that goes into aluminum frames that's higher than in a cheap steel frame. The more specialized welds affect the price too, of course. My bike's made out of boutique steel, but any chromoly steel should be fine too. Good steel bikes are still pretty light, and have a reputation for a very lively ride. They're slightly less stiff than aluminum, so they absorb a little more vibration. High tension steel, "Hi-ten," is not okay. It's heavy and the ride has a dead feeling.
The fork comes in the same breath as the frame. Mine's carbon, and I think it rocks. Carbon is good at absorbing road vibration, so it doesn't travel up into my hands as much. Unless you're planning to violate your racing bike by putting a suspension fork on it, it's the next best thing. Of course, they're a little expensive.
Once you've got a frame and fork, you need wheels and a component group. Low spoke-count wheels look kind of cool, but the performance advantage is pretty insignificant. Deep-V aerodynamic rims also look kind of cool, but cost and weigh more. For the type of riding I'm assuming, they're not better than the shallow-V aero rims that are popular lately. A well-made wheel spins freely and will keep spinning for a very long time after you spin it. Assuming that it's centered between the brake pads, it will never contact them until you squeeze the brake. It should also be perfectly round, so the tire shouldn't get closer or further from the brake arch during the spin. If any of these isn't true, and the wheel is seated properly in the dropouts and the brakes are adjusted right, the wheel needs to be repaired or replaced. Low quality wheels sometimes can't be fixed, or not without being rebuilt, and a good quality hub is very important - it's one of the main sources of resistance to going fast on a bike.
Most commercially distributed bikes come with a mixed group. That's a way that they can put better parts places where they think there's a significant performance advantage, or where they think their consumers think so, and save money where it's not as important. The big name brands here in the US are Shimano, SRAM and Campagnolo. A lot of people consider Campagnolo to be the best, but you definitely pay a premium. SRAM has a road group, but I've never seen it advertised on a complete, mass-market bike. My bike has Shimano 105 for all the drivetrain components. It's an all-metal group, which is important because plastic parts don't stay tuned, especially as they start to wear out. I think it's especially important to have good indexing shifters, because repairing them is difficult or impossible and they're expensive and a pain to replace. The next group down is Tiagra, which people on roadbikereview.com describe as being a little less than reliable, and not shifting smoothly under tension. That being said, Tiagra components have the same spacing as 105 components, so if you have a Tiagra-equipped bike, the parts can be replaced piecemeal.
So that's most of a bike. It still needs a seat post and saddle, a stem, and handlebars. It's good to have a seat post that allows both fore/aft and angle adjustments on the saddle because if a level (or whatever angle the seat post is at) saddle isn't comfortable for you, you'll have to spend a surprising amount of money on a new one. A good saddle is very important. Riding should be fun and a bike should be comfortable, and a bad saddle prevents both of those. This guy has a lot to say about saddles. My experience is that a saddle should be firm - not quite as hard as concrete, but very little give - and as narrow as supports your sit bones properly. Those are those two little bony bumps in the bottom of your butt. Sit bones are made to take weight, but a soft saddle makes other parts take weight too, and they're not made for it.
Most racing bikes now use the same threadless headsets that mountain bikes use. They're lighter and sturdier, but a pain to adjust. The handlebars are held by the stem. On a threadless headset, that clamps to the fork's steerer tube. You choose a height and cut off the excess. The problem I have with these is that if you choose too low a height, you're SOL. If you decide your handlebars are too high, you can remove the headset's top cap, the stem, and some spacers, then reorganize them with the stem lower, but sooner or later you'll have to get the steerer tube cut down some more. There's probably not much choice anymore about this. Speaking of headsets, there's something called an "integrated headset" that's showing up on some bikes. It uses the frame to perform the function of one of the parts of a headset. Headsets are relatively cheap and fairly easy to replace, but if the frame gets worn out you need a new bike. So avoid the "integrated" thing.
And that's a complete bike. What to give up if the sticker on the bike with all that is too high?
Carbon forks are pretty expensive. Steel forks were good enough for almost 100 years, and they're better now.
Off-brand brakes are fine. As long as there's no plastic. And frequently the brake pad, a cheap and easily replaceable part, is the only performance-effecting difference.
It's silly to spend a lot of money on the seat post and stem. And expensive headsets are kind of cool, but as long as they're sealed well and move smoothly, the less expensive one is good enough.
A less expensive crankset is probably fine. Hidden behind the crankset is the bottom bracket, which is worth spending money on. Of all the bearings in a bike, the bottom bracket sees the most rotations. It had better be good.
Derailleurs are pretty easy to replace if you can't start with a good one. But you eventually need a good one, or shifting can be a little less than reliable.
The chain, cassette and chainrings, as long as they're compatible with the higher-end equipment, don't need to be expensive.
My commuter had cheap foam handlebar tape. I loved it - it had little skulls on it.
Good pedals are worth it, but once you have something sturdy with a good bearing, it's probably not worth spending money on saving weight.
I like having an odometer, but I really don't need one.
Of course, it's much more economical to buy a complete bike than order by component, so after choosing a pricepoint, there may not be much selection of model. And ultimately, the bike that rides the best is the best choice. But if a shop carries different brands or models with different component mixes and similar ride quality, this should help in choosing. And if you like one bike a lot but it's not available in the right size, they can always order one.