another solution would be to make it highly stackable, like up to 100. It would take up 1 slot. in the end you can drop it (becomes an item like other collectable resources) or keep it to raise terrain.
i would not allow to raise terrain without dirt in any of your bags and not allow digging without space where to put the dirt.
terraforming would become a teamwork where a digger provides dirt for the raisers.
the tribe could also keep a dirt pile somewhere on a digging site, so that it is readily on hand for further workings and stuff...
all in all, dirt is a great thing if it's implemented right!