Off grid different than on grid for location

I have my house and panels pointed to magnetic south.  Here in PA, that's about 15 degrees east of solar south.  I did that on purpose because the batteries will drain and the house will cool overnight, so I want to get them charged up first thing in the morning, not wait until mid-day.  If my batteries are full long before supper, that's ok, they'll hold me through the night.  A westward facing bank of panels at that point wouldn't do me much good other than maintain full batteries.  Most of the energy would probably go to waste.  The worse case is to have an overcast day then go through the night and not take advantage of first light to charge up again.

As far as vertical orientation, I pointed mine at 60 degrees to get the most direct exposure on the shortest day of the year.  This is important when off-grid, but not such much when grid-tied.  I may get much less power in the summer (and therefore for the year) than I otherwise could, but it's still much more in the summer than the winter.  The steep angle flattens out the seasonal curve to ensure I get decent power all year long.  It also deflects hail and snow this way.

A tracker sounds enticing to solve these issues, but I figure for the cost of a tracker, I could buy a lot more solar panels, so what really is the payoff?  I'd rather have the reliability of solid-state panels than a complex mechanical/electronic device.

Yep.  The same reason you need breakfast in the morning, your off-grid system needs morning sun.  After the night, you're more likely to need to fill up your batteries than in the evening.  That's why I think photovoltaic panels' marginal utility is higher by pointing them slightly east.  But that's only true for off-grid.  For grid-tie, pointing them at solar south is better because you'll get greater total energy from them and there is no storage issue.