If you don't want them board stiff you could use plastic canvas and still cover it with fabric. or plastic like they use for rulers but without the numbers they will cut it to size at most hardware stores.
Yes to dragonflyer's answer, that is also my solution. I put that in my lighter weight bags. You can use any material including wood (I've used lauan the term typically refers to a plywood panel 1/4 or, less commonly, 1/8 inch thick.) that you can make a cover for. It depends on the weight that you will use inside the fabric box.
Am I understanding correctly that the boxes are made of fabric? Perhaps you will be storing fabric inside the boxes?
You are correct that I want to make boxes out of fabric. But knowing me, I will probably also put fabric in them. It's an addiction you know.
I know...lol-I agree about the luan and a 4x8 sheet would make a stack of boxes!
Hmmm...I might use 1/4" Fiber Board or Particle Board and make a slip cover from your fabric that the board would slide into...that way you would have a very sturdy base that could be removed and the slip cover removed for laundering, too...it is also fairly inexpensive and can be cut to size...