There are many kayak features to consider when choosing a kayak and all the options available are important. The following are some (but not all options) that might be important to you for recreational or touring kayaks.
First, you should decide what you want to use your kayak for - ponds, streams, lakes, rivers, ocean, etc. The right kayak for ponds and lakes might not be the right kayak for ocean (and it might be dangerous).
Recreation kayaks are usually small kayaks, more stable, easy to pack and get away for the day. Recreation kayaks are easier to turn in a smaller radius and good for quiet waters.
Light touring kayaks are for paddling longer distances than recreational kayaks, such as light overnight trips. They have at least two hatches, are heavier than recreational kayaks and probably need two people to load the kayak on top of the car.
Touring kayaks are for paddling the farthest. They have at least two hatches, are longer than the light touring boats, have extra safety features (tow lines, etc.) and are much safer on the sea.
There are many shapes to consider (too many to even mention here). If you sit in the kayak on the floor at a kayak retail shop and the kayak is flat on the floor, it will be a more stable kayak in the water, yet will be harder to paddle, harder to keep on a straight track and harder to turn. If you sit in the kayak on the floor and it is harder to keep upright, the kayak will paddle faster and will be easier to turn. The rule is the safer you feel initially in your kayak, the faster you will outgrow your kayak. Wide kayaks are 28" or higher wide and narrow kayaks are 19"- 27". The narrow kayaks will paddle faster, but will feel very tipsy until you become used to the kayak.
The skeg and the rudder will keep the kayak going straight under troubling conditions such as high wind or strong currents. Contrary to most thinking, the rudder is not to steer a kayak but to assist in it's direction. A rudder can potentially impede rescues and is just another part of the kayak to break or need repair. With skeg boats, a stone might easily become caught between the skeg and the wall of the kayak. This may not allow you to lower the skeg during inclement weather conditions. If there is no skeg or rudder on the kayak, you have to learn and practice your kayaking skills to keep the kayak going straight. In very bad conditions, the rudder (or skeg) is an option that makes paddling much easier and it might make a difference of not being in trouble and being in trouble.
What material is best suited for the type of paddling you will be doing? Where will you be paddling and what will be the conditions of the water and the land? Will the land usually be rocky, have wood stumps, sand or dirt landing areas?
Molded Plastics are more durable, can be pulled up on the rocks and can withstand rough treatment. These kayaks are usually the heaviest of the kayaks.
Fibreglass is much stronger, needs to be treated very carefully, weighs less than Molded Plastics and usually more than the High Impact Plastic kayaks. You have to be careful not to hit rocks, not to land on rocky land, not to drop the kayak, etc.
Fiberglass repairs might have to be made on a fibreglass kayak. High Impact Plastics give you the benefit of strength, easier to maintain than Fibreglass and are usually lighter than Fibreglass.
Kevlar is usually the lightest, the strongest and the most expensive kayak. The kevlar kayak is the strongest kayak, but the hardest to repair. Hairline cosmetic cracks usually appear after many years of use and are difficult to repair. The kayak prices go from the most inexpensive to expensive: Plastic, High Impact Plastic, Fibreglass and Kevlar.
Hatches are very important features of a kayak. The hatches keep the kayak floating when the kayak is not upright and also limit the amount of water that can fill into the cockpit. They also protect everything from becoming wet that is carried inside the hatches. If the kayak is an open kayak (no hatches), special bags HAVE to be purchased to fill in the bow and stern to keep the kayak from sinking.
Do you expect to paddle in the ocean or in calm waters? If you expect to paddle in the ocean or in any body of water that has waves, strong currents, etc., the kayak you choose should have knee braces to help you steer and paddle. Do the knee braces fit you and are they comfortable?
Kayaks come in a multitude of colors and some colors fade or scratch more easily than other colors. When you select colors, ask the manufacturer if those colors will hold up well under the sun, salt water and all the elements.
Depending of the type of kayaking you expect to do, all the extras on the kayak make paddling less dangerous. If you will be paddling in the ocean, choose a kayak that has the most features available. Learn what each feature does and is that feature important to you.
Never choose a kayak unless you have paddled in the water. You will be disappointed and even may want to return the kayak if you haven't paddled in the kayak. A kayak is similar to footwear in that you would never buy shoes without trying them on. Just because your friend says that a certain model is the best of the kayaks, it doesn't mean that the kayak will fit you. Everyone is different and every kayak fits everyone differently.
Don't forget to learn how to maintain your kayak correctly because incorrect methods might damage your kayak and choose a kayak wisely.