Simple circuits reduce regulator noise floor
In the old days every instrument or system had a power supply board with discrete, homemade regulators on it to power the system. Then IC's like the LM7805 series of voltage regulators changed everything. No longer were system designers required to spend their time designing the power supply section also. These modern IC's even incorporated much improved current limiting and thermal protection. As such, the reliability of everything electronic improved overnight.
These venerable old regulators continued to change to match our design needs which led to low dropout varieties, improved transient response parts and now lower noise designs.
Noise has always been a constant and the LM7805 of 1972 has the same noise as the LM7805 purchased today. What has changed is our system's need for lower noise. This is especially true in RF Communications where we need low-phase-noise oscillators to be able to transmit and receive our complex digital communications. In data acquisition we commonly have 24-bit A/D converters which naturally demand low-noise support circuitry to be able to achieve their data sheet performance. Although not to be forgotten are our Audiophile friends who always worry about the "sound of noise" in their systems.
These advances have all gone a long way in helping us to design systems that meet specifications and are, small, cost effective and perhaps most importantly for those who pay our wages: they help us to stay on schedule.
Discrete Noise Reduction Circuits
Over the intervening years between the first regulators and the newest low-noise varieties there have been a few discrete circuits that have cropped up to lower the noise floor of our power supplies even further. As shown in Figures 1 and 2, the most common forms are the very popular "Capacitance Multiplier."
Figure 3 shows the less well known "Active Regulator" . There have also been a number of other active regulators proposed over the years, but most of them have been focused on ripple reduction, not noise floor reduction .
Figure 3: The Active Regulator, while not as popular as the Capacitance Multiplier, it does show up from time to time in published circuits. I first saw this circuit on Charles Wenzel's Website . Typically R4, R5 and or R8 need to be adjusted to get the noise reduction gain optimized for each circuit where this circuit is applied.
How they work
The Capacitance Multiplier as shown in Figure 1 works by isolating the load's influence on the capacitor by the current gain (or beta) of the transistor; hence whatever capacitor is used for C1 is multiplied in value by upwards of 100 times. The circuit of Figure 2 is a slight modification that strangely adds a noisy Zener diode  for improved line regulation, but in practice still can produce a low-noise output if R3 and C2 are chosen properly.
In most audio types of applications you will see these circuits implemented with a 1000uF or larger aluminum electrolytic capacitor. Here I took the more RF-oriented approach and used what might be found on a typical small PCB, and used a 10- or 100-uF/25V Tantalum  for C1. No special exotic or expensive parts were used in any of the testing.
The active regulator of Figure 3 works by sensing any noise at the input; transistor Q3 then amplifies and inverts the noise and subtracts it by adding an equal and opposite current across resistor R4. Thus in theory the noise reduction is perfect, but in practice greater than 40 dB noise reduction can be achieved.
The Active Regulator also has a high-frequency limitation because the transistor is working as an amplifier - it will have a gain-bandwidth limitation as well, which limits the circuit's effectiveness at higher frequencies. In practice I find that bandwidths of 1 MHz are not difficult to achieve and at that point the standard capacitor bypassing is very effective at keeping the noise floor low.
Note that while the Capacitance Multiplier usually works without any modification, the Active Regulator of figure 3 must usually be optimized for every application.
To get optimum noise reduction, the gain of the transistor stage must usually be adjusted and the easiest way to do that is to temporarily replace R8 with a 10- or 20-Ohm potentiometer. The potentiometer is then adjusted for optimum in-circuit gain that results in minimum output noise; the potentiometer can then be replaced with a fixed resistor.