PSP games come on PSP UMD discs. These discs can hold 1.8 GB of information.
Yes, they are expensive, but $120 is a good deal if you can get a brand new system.
Normal SD Cards do not fit into PSPs. Only Sony's Memory Stick DUO (Like an SD Card) will fit.
As for downloading games, I don't think your idea will work. You'll have to buy the UMD discs at the store - around $40 - $50.
Or, you could always use Ebay to find PSP systems and games for cheap. Hope that helps!