Hmmm...I might use 1/4" Fiber Board or Particle Board and make a slip cover from your fabric that the board would slide into...that way you would have a very sturdy base that could be removed and the slip cover removed for laundering, too...it is also fairly inexpensive and can be cut to size...

Yes to dragonflyer's answer, that is also my solution. I put that in my lighter weight bags. You can use any material including wood (I've used lauan the term typically refers to a plywood panel 1/4 or, less commonly, 1/8 inch thick.) that you can make a cover for. It depends on the weight that you will use inside the fabric box.
Am I understanding correctly that the boxes are made of fabric? Perhaps you will be storing fabric inside the boxes?

If you don't want them board stiff you could use plastic canvas and still cover it with fabric. or plastic like they use for rulers but without the numbers they will cut it to size at most hardware stores.

You can also use old cutting mats cut to size.