The answer is, quite simply, they didn’t want to. The Empire and First Order both operated under the philosophy that stormtroopers were completely expendable and not worth providing with better armour. They were perfectly capable of developing it, hence why some classes of stormtrooper did have much sturdier armour, but they had such a large number of recruits that it just wasn’t necessary. Now, there is a bit of a disconnect here between what we see in the movies and how the armour is described in supplemental material. Safe to say, it appears a bit flimsier in the films than it’s described in the books because it’s convenient to have the stormtroopers die in one shot onscreen. The armour was often sufficient to deflect a glancing hit, and there are instances where we see the trooper get hit directly and get knocked out but still survive. Standard stormtrooper armour wasn’t completely useless, but it definitely wasn’t intended to ensure the safety of the troops.
Hope that helps!
~ Jacen