Reddit mentions of 1200mAh Li-ion Battery For Sony PSP 2th, Silm, Lite, PSP-2000, PSP-3000

Here's the blow-by-blow:

• There's a great modding guide over in the sidebar. It takes you through getting the most common custom firmwares onto any PSP device, and it's literally a ten minute job, if that.
  
• Everyone will give a different answer for batteries. Generally, most aftermarket batteries are terrible, so some recommend hunting down original ones - but you have no idea what sort of condition they'll be in at this point. Another option is removing the UMD drive and installing a LiPo in the void left behind, but that can be a bit intensive, and not everyone wants to lose UMD capability. You can also - with great care - dissect an original battery and install a new cell. Personally, I just picked up a 1200mAh Cameron Sino battery from Amazon two months ago and it seems to be holding up well.
  
• Don't bother with overpriced Memory Duo cards - just get a Micro SD to Memory Pro Duo adapter. It's a lot cheaper and you can get higher capacity cards.
  
• I've never used a Pandora battery, so I'll leave that to someone else. I don't think you need them now, though.

I'm having this problem at the minute with a 3rd party battery I've purchased recently and used just once. I picked up a 3rd party as the official small psp batteries seem to have a poor shelf life and wanted to find a good alternative. I am currently awaiting a reply from the seller (they have until the end of today).

My battery, when put on charge, takes just a few minutes before the orange light disappears. However it will not switch on unless the mains lead is connected to the wall. When on the home screen I can see the battery icon flip from being shown to being hidden. Clearly it's a battery issue - my other battery works fine for charging and using.

The link below shows the battery I purchased just in case anyone wants to know.

https://www.amazon.co.uk/Replacement-battery-PSP-2000-PSP-3000-PSP-3004/dp/B004WLOJ1E