Welcome to the sport. Having been an avid geocacher since 2002, you will go places you never would have dreamed of going to. Geocaching is a great way to see the state and visit parks, historic sites or little known hangouts. You should check your area to see if there is a local club you can get involved with. I am one of the board members for our local club here in central Illinois and we love to hear from new folks and help them on the journey and answer questions in regards to the best gps to use, what is GSAK(a very useful tool in the world of geocaching), or give hints on that cache they just cant seem to find. Also be on the lookout for events in your area as they are a great way to network with other cachers in the area. For example, our club tries to hold one event each quarter, whether it be at an ice cream stand, local buffet, or city park...they are always a good time.
For those of you looking for a gps, shop around online to find the best price or package price. I am partial to GARMIN myself(gpsmap 76-csx) as I feel they are more user friendly, plus the CSX versions of their handheld gps's have some of the best reception out there when you get deep into the woods. Feel free to PM me if you have more specific questions....