there is also a timer on replanting a totem. forgot what it is...but it is a decent amount
its 360 minutes, (6 hours)
a rather long wait if you have a large number of bags and buildings, as anyone can just walk up and disassemble the walls/buildings in that time. or if you had any bags set to public and forgot.