Over-Engineering Culture, Hacking, and Complexity

I was talking to a fellow escapee of The Conglomerate and we were talking about over-engineering culture. In the sense of “lol things built because it was time for someone to get promoted” and in the sense of complexity.

The thing about layers and layers of (debatably necessary) abstractions is that they make things that should be simple, complex. Which makes people feel stupid. The last change I committed there I was adding one field… and I had to change > 15 files. The biggest problem I had? Was thinking I must be doing something wrong because it couldn’t possibly be this hard to add one field, right?

I think of hacking as fixing one corner case only to make two more for later. So, reflecting on this, this is also true of a culture where things are hacked together. Because when hacks build on each other again and again, the result is that simple things become harder (and slower) than they should be. It makes people feel stupid.

We often talk about these two cultures like they are completely separate but this is not true. They feed on each other.

• Hacking can be a result of over-engineering culture, because things that can’t be done “correctly” may as well not be done properly at all.
• Hacking can be a reaction to an over-engineering culture, because people tired of things being so hard want to move fast and break things.
• Over-engineering can be a reaction to hacking because someone bitten by too many corner cases decided that it would be better if everything were a perfect circle.
• Over-engineering can be a reaction to hacking because perfectionism can be a reaction to things constantly breaking.

Personally, I like to think about the medium term. Essentially this means: don’t hack things because hacks are short-term. But just because there might be a reason why something won’t expand to fulfil some other purpose doesn’t mean that it should be generalised (yet). If that scenario is uncertain, and not in the current timeframe, document and move on.

Hacking to over-engineering isn’t a scale, it’s a circle. At the darkest part where they meet, simple things are hard, and engineers trying to get to grips with it feel stupid. Thinking about it, I’m not sure whether that codebase that required fifteen files to be changed to add one field was over-engineered or made of hacks, but does it matter? The effects were the same. Move slowly, with things that barely work.