I massaged my exhaust cut-outs just about as you describe. I found witness marks on the pipes, indicating to me that they were moving a little bit when the engine torqued, and the hole wasn't quite large enough. I used a die grinder with a couple different cutters to open to the shape I wanted, then dressed it with a combination of flat, half-round, and rat-tail files. Finished off with a "tootsie-roll" sanding spool on the die grinder, and then touched up the edges with body-color paint.
Jeff