Paper Capacitors (once commonplace) have thankfully gone the way of the tubes they were connected to. Any of you old farts that had that TV that after about 7~8yrs you couldn't keep tubes in were victoms of dried out wax/paper capacitors. Replaced my share of 6AU6's and 12AU7's back in the day...
Paper electrolytic caps are still around though. (I think what Gunner is actually referring to). These eventually dry out and fail as well. "Supercapacitors" are basically large value elecrolytics, typically in the 1+ farad range as opposed to microfarads. First wide use of these was VCR clock power backup.
"dry piles" are carbon batteries as I recall... (also referred to as a "carbon pile")