Many modern CF double derringers are unreliable, with misfires common on one or both barrels. The problem is in the selector mechanism, which has to move the striker much further than in the old .41 RF guns, where it moved only about 1/8 inch, from the bottom of the top round to the top of the bottom round.
As to recoil, my limited experience has been that recoil in all except .22 RF and light .38 Special was awful. .45 Colt and .357 are way too much and .410 is over the top. I fired only one round from a .410 derringer and returned the gun to its owner; that was quite enough. The comments about trigger pulls are valid also; most are too heavy, plus the hammers are hard to cock.
FWIW, the original Remington double derringers have a spring that looks like it came off a Ford 250; they are very hard to cock. So the next time you see a Western hero draw his Remington and snap off two quick shots, putting the bullets in the same hole at 200 yards, just remember that it doesn't happen that way without help from a gunsmith (and the script writer).
As for carrying a derringer for SD, it would have to be the only gun available; I would prefer a .25 auto.
(I agree that the old High Standard derringer was good and in .22 Magnum would be a reasonable defense gun, but with the new .380 and 9mm autos, its time has passed.)