I'd sad non-scalable software is the norm, not the exception. There are lots of excellent programmers who know to avoid exponential algorithms, but they have good jobs as far away from marketing as possible. There are far more AWFUL programmers out there peddling their shoddy wares for a quick buck. It's simple: if there are more than 4 or 5 competing software packages that do the same thing, at least 75% of those are going to be absolute garbage; such is the result of today's market forces and management overload, the people doing the purchasing either don't have the eye, or the time to properly examine their options and the result is that the incompetent coders make a sale.
To most experienced developers, bottlenecks are immediately apparent during the design phase. We just know, before the requirements meeting is over, exactly where and how the app will fail and that's where we focus the bulk of our effort. If you know that the key feature of an app is a nonsensical 14-table SQL join or a write-heavy backend, no amount of brilliant code-level optimization will make the app faster... the key is to break it into manageable chunks or sometimes completely rework the functionality.
If it weren't for these true engineering problems, programming would not be a career.