Which way to go-where to start?

Hey Roy, don't let your head explode! Be excited because you are entering an amazing world. Much of this stuff is relatively new, and almost all of it changing fast.
Have you checked out Homepower Magazine? They also have a decent website. They are probably a great place to start. This website here is also a great spot, as well as the staff. Every employee I have spoken to seems to be very knowlegable. I bet the janitor knows more than I do.
Most of what I have read\heard is that solar hot water has the fastest payback. Does that mean it is the best for you? Hard to say seeing that It sounds like you are electricity dominate.
I am a big fan of radiant heating.
All in all, I feel the best "first place" to start is conservation. Not very sexy, but it is the real foundation of renewable energy of any type. A load analysis will help you see what you are using and in which areas you can start  making  reductions.
Electric base board heater - 1500 watts
5 100 watt bulbs           - 500 watts
1973 green fridge          - 8 billion watts
all equals about 4 kw of photovoltaics ($50K ?)

Radiant floor heating      - 5-10 watts
5 cfl bulbs                - 60 watts
high efficiency fridge        - 175 watts
this equates to like .8 kw pv.
Happy studies and hope to hear about your progress. -m

As others have said, you have to start with conservation.  Alternative energy is not generally the cheapest or easiest option.  It is only cheap and easy so long as you've minimized what you need to get from it.  That means first and foremost ensuring that your home is well-insulated and draft-proofed.  Put insulated curtains on your windows.  Weatherize all your doors and windows.  Consider how your landscape contributes to heat gain and loss.  Et cetera.  Also think about passive solar gain... paint the south side of your house a darker color, install extra windows on the south side, and ensure plenty of thermal mass in your sunlit areas.

After that, solar thermal is an excellent way to go assuming you have open skies to the south.  Shading of your panels can severely diminish their effectiveness.  Also, you have to think about how you will deliver the heat.  How is your "all electric" heat currently delivered?  Through forced air ducts?  Through base-board radiators?  If it would require a major renovation of your entire house, maybe solar thermal wouldn't work for you.  I would recommend radiant floor heat, although hot water radiators may work OK, and I think you can adapt it to a forced air system, but I don't know anything about how efficiently or effectively that would work.

Solar thermal works very well with an on-demand propane tankless water heater for backup.  You just put it in series.  If the solar is hot enough, the propane doesn't turn on.  If it's not hot enough, the propane tops it up.  Even if you don't have enough solar to heat your entire home, the solar can effectively pre-heat the water so that you use less propane. 

As far as energy prices going up, I'm sure you're right.  Especially electricity, and especially if we get cap-n-trade legislation.  However, propane should remain plentiful and relatively cheap for the foreseeable future.  I'm currently in negotiations to lease my land in Pennsylvania to a gas company to tap the Marcellus Shale gas under my feet.  The Marcellus Shale contains hundreds of trillions of cubic feet of natural gas -- more than in all of the Middle East.  We have at least a few hundred years' supply.  And there are other plentiful gas resources in the U.S.  Natural gas, propane, and other components of this resource should remain relatively cheap long after we're dead and buried.  The key word there is "relatively", since prices will increase along with inflation as long as the government keeps spending like nuts and the Fed keeps the printing presses going.  Solar is a good inflation hedge because all of your cost is up-front.  Once you own the system, there's nothing else to buy, so you don't have to worry about inflation.

Other than solar and propane, you may also want to consider an outdoor wood furnace.  If you have a wooded lot or can purchase cord wood inexpensively, these are great alternatives to traditional heat sources.  You just stoke it with a few logs once or twice a day and it provides plentiful heat to your home and hot water.  My parents have one and they keep their 5000+ sqft home, garage, guest apartment, and workshop at above-average temperatures all winter.  Most models have a backup propane or oil burner built-in so that you can go away on vacation and not have to worry about frozen pipes (or if you're just too lazy one snowy winter day to feed it with wood).  If you're concerned about greenhouse gases, remember that wood is carbon neutral -- each tree you burn is replaced by one that's growing.

Something else to consider is a nice high-efficiency wood stove inside your house.  I have a BIS Ultima, a built-in-stove which looks like a traditional fireplace.  It's 80% efficient and draws its combustion air from outside.  If you like the cozy look of a fireplace and the comfort of a point source of heat (so you can regulate your own temperature according to how close you are to it), then these are an excellent choice.  You can even duct heat into different rooms from it.  I sized my solar thermal system to maintain a baseline temperature in the house at around 60 degrees.  I can then burn wood for an hour or two in the morning and evening to bring the temperature up from there during the coldest months.  I also have a propane backup on my solar thermal, thus providing me multiple redundant heat sources.