Recumbent bikes
I could not recommend a quality recumbent bike more. I have two arthritic knees ( as well as calcium deposits in them and runners knee ) and a few arthritic toes from breaking them and them healing poorly. I am also still obese, but coming down slowly.
I have a Schwinn 220 that I bought in Aug 2011 for just over 400 bucks and it is wonderful. As you are sitting on a very wide seat and it has a back to it, there is no hunching over, reducing any neck, shoulder or back pain. As your feet never touch the ground, there is no pounding to already fragile knees.
When I first got it I was 50 and could only do about 12 minutes and thought I would never conquer it. Yesterday I did 23 miles in 62 minutes, so you can truly get stronger as you get older, next week is my 54th birthday.