Did you scale the design (made it smaller than the original)to fit in your hoop?
This possibility is limited- 5% is pretty much.
If you did not yet and have the possibility with a software you could try to enlage it a LITTLE to give it less density and to smooth out.
Try the free programs for converting we discussed in the software area, they can do scaling too.
And NEVER touch the screw of the hoop when the fabric is snapped in.
Tighten the screw with attatched fabric and stabilizer will cause puckering.
It will give enough tension to the fabric when pushing the inner frame throug the outer one.
And I forgot to mention: sometimes it is just poor digitising f.e. without an underlay. Did you purchase a quality design? If this is about quality, just forget about it and choose another one. Don't forget to delete the junk.
99% of the time it is improper stabilizer. Try floating another piece of tear away under the design. If that doesn't work you will have to use a cut away, then float a piece of tear away. By float, I mean put it on the bed of your machine under where your design will stitch. It is the back up, so it doesn't have to be hooped or adhered on.
you could make you design less dense by using software & yes more stabilizer will help
Are you stabilizing it sufficiently? Rule of thumb, for every 10,000 stitches use one layer of medium weight tear away.
The design might not be suitable for that type of fabric. The designs are usually digitized to go on cotton. It might be too dense for this fabric.