Bank changes money, cash traveller cheques and cash advance on credit cards. There is also an ATM in front of the building that now accepts most foreign cards as of December 2008.
There are quite a few ATMs in Pakse, so there is no need to bring lots of cash when you head to Pakse. In addition, most hotels also changes USD to Kip with an exchange rate of USD 1 to 8,000 Kip.
If you are heading to Siphandon, it will help to withdraw some Kip from the ATM before you head down. There are no ATMs on the islands, however, if you desperately need some money, there is 1 on the mainland near the islands. You will have to pay for a boatman to bring you to the mainland and back which will probably set you back by a few USD.