If you want to start simple and see, get a Zoom H2 or H4 (find good used on eBay) and download Audacity.
http://audacity.sourceforge.net/Maybe some others have better suggestions for free editing software? Or maybe other equipment?
I won't go into what I have because I had the top floor of my house as a recording studio for over 20 years. Now I have parts and pieces and recorders and thousands of tapes, scattered around storage, the garage and the new house. Total mess!
One computer is dedicated to music and has a 8 channel DRM in it. (sitting in the corner after the last move) Winter project.