Fortifications do not add +1 defense per unit in a city. This is because the Fortifications building has a maximum roll of 6. Militia are eliminated before the fortifications building because the militia unit has a maximum defense roll of 4. Militia do not benefit from +1 defense. Any units with less than a defense of 6 will be eliminated before the Fortifications.
Give Fortifications a defense of 1, like the General, as Fortifications are advertised as being the same concept as the General.
The fortification works as intended, until you get to a unit that has less than 6 defense. In this case, militia comes to mind. But as long as you have a unit that is 6 defense or higher, than fortifications work. Most people only do make infantry (unless you're going SM/RA/GW).