@endso,
You have at least two options:

1. Use a 5V regulator and wire a diode in series with its output. Silicon diodes have about 0.6 to 1V forward voltage drop, depending on power rating (power diodes have a bigger drop). That should reduce your 5V to just the right level.
   The trouble with this solution is that the drop is non-linear. The stated 0.6 to 1V is the maximum drop, if the current is large enough. For very small currents it may be much less. What is "large enough current" depends on the diode, but as a rough guide, assume ~1mA for low power diodes.
   Better try it with a dummy load before exposing your precious to overvoltage.
2. Use a 3V or less regulator and wire it up to give the desired output. You will need two resistors. Connect them in series. The top end connects to your regulated output, the bottom end to the ground. The middle end now provides a reference voltage somewhere between the two, depending on the relative proportion of the two resistors. Connect the regulator's reference input (the pin that normally goes to the ground) to this middle point and hey presto, your regulated output is raised by the relevant value.

There are of course gazillions of other solutions but these are the simplest.

EDIT:
In case I am expressing myself very clearly, here is a page that is:
http://www.learningaboutelectronics....tage-regulator