Software engineering

Why is software engineering so difficult? Is this even engineering?

Must-read classics

How to build good software

Things you should never do

No Silver Bullet

The Architecture of Open Source Applications

Real software engineering


essential difficulty: invisibility

We speak human language to communicate ideas. Machines understand code. There is a gap between the two.

The reality of software though is not inherently embedded in space, and thus has no ready geometric representation.

Sometimes pictures can help in presenting ideas. But software visualizations will never be fully accurate.

Testing and prototyping

essential difficulty: conformity and scope

It is import to understand the problem before writing code. But how well do we understand the requirement?

We use prototyping to discover potential solutions. But there is more: prototyping also helps us understanding the problem and identifying new requirements. Testing, at the same time, validates the solution. Development cycles should alternate between those. With other words, consider exploration and consolidation phases.

Assuming that we completely understand all the requirements, software products still exist in a constantly changing environment: users, stakeholders and infrastructure changes over time. (software engineering is a service industry?)

Known unknowns and unknown unknowns

essential difficulty: changeability

How do we know that a software product is complete?

Software is constantly subjected to pressures for change

Software is constantly changing? We need to prepare for that.

But people tend to value known risks over unknowns. It makes us feel smart to think about risks others may not. As a result, we underestimate the unexpected. We rarely hear the following during roadmap planning: we don't need to prepare for that, let's just save those resources for the unexpected.

100:10:1 method: Brainstorm 100 things that could go wrong. Pick 10 on that list that feel like the most likely and investigate them. Find the 1 critical problem you’re going to focus on.


essential difficulty: complexity

What is software complexity? How can we measure that?

Software is limited not by the amount of resources put into building it but complexity.

Solution: alternate between extending and reducing complexity.

Solution: minimalism.

Code is not an asset, it is a liability. The more code you have, the more you have to optimize, debug, modernize, refactor, understand. In short, the more code you have — the harder it is to change. A smaller and simpler codebase is easier to understand, improve and extend. - Paul Copplestone


Programmers are most effective when they avoid writing code.

How to measure productivity?

Productivity varies tremendously across the profession, but it may not vary so much within a given company.

Maximising developer productivity is not the goal. Build things that will be used. It's not a problem if a team builds less. Estimate feature value and focus on delivering that. How to calculate feature value? It depends, but it's sure that it needs to be delivered first.

How to improve productivity? Reuse.

The best code is the code that you don't write.

Cut features that don't add value.

Top-down or bottom-up?

Working backwards:

“The idea is to start with the desired customer experience when designing new products, going so far as to draft “a press release that literally announces the product as if it were ready to launch and an FAQ anticipating the tough questions.” - Amazon working backwards

Working bottom-up:

Why software is easy to sell?

Software is largely a service industry operating under the persistent but unfounded delusion that it is a manufacturing industry. — The Cathedral and the Bazaar by Eric S. Raymond

How to find a direct competitor or alternative if it is not clear what problem the software solves?

See frameworks and obscurantism for more.

The main reason why software engineers earn more than other engineers because software is easy to sell.

Does anything work?

My own experience, validated by Cockburn’s thesis and Frederick Brooks in No Silver Bullet, is that software development projects succeed when the key people on the team share a common vision, what Brooks calls “conceptual integrity.” This doesn’t arise from any particular methodology, and can happen in the absence of anything resembling a process. – Why don't software development methodologies work

Use the feedback engine effectively. Cost of change is effected by the length of the feedback loop.

Iterative and incremental

Development cycles alternate between:

What is the goal? Feedback loop on different scales.

  1. A software system can best be designed if the testing is interlaced with the designing instead of being used after the design.
  2. A simulation which matches the requirements contains the control which organizes the design of the system.
  3. Through successive repetitions of this process of interlaced testing and design the model ultimately becomes the software system itself. – Alan Perlis

How big an increment should be?

Ship as soon as possible. Because shipping means more feedback.

Shipping early, frequently, and on a cadence allows you to shorten the time to feedback and accelerate getting you to where you really need to be.

Brendan O'Leary - What I learned at GitLab that I don't want to forget

Why are we doing these iterations? To get feedback on our decisions.


Architecture is a buzzword? Equals software design? It is team responsibility, everyone can do that on a certain level.

Remember that the builder has the inventive and creative responsibility for the implementation; so the architect suggests, not dictates. – The Mythical Man-Month

But communicating ideas during software design is hard. Separated architects perform worse, it is best done by a team.

Minimalism in software engineering


Minimalism improves quality:

Size is the best predictor of code quality

A long paper trail of software engineering studies has shown that many internal code metrics (such as methods per class, depth of inheritance tree, coupling among classes etc.) are correlated with external attributes, the most important of which is bugs. What the authors of this paper show is that when they introduce a second variable, namely, the total size of the program, into the statistical analysis and control for it, the correlation between all these code metrics and bugs disappears. However, this paper uncovers a strong association between most code metrics and the size of the code, both of which together lead to the externally observable attributes like bugs - Source

Faulty reward functions

AI agents fail miserably when trained with misspecified the reward functions. This also applies to software engineering: it doesn't matter if we have 100% test coverage, use proper semantic versioning for all software components.

Choose boring technology

Let’s say every company gets about three innovation tokens. You can spend these however you want, but the supply is fixed for a long while. You might get a few more after you achieve a certain level of stability and maturity, but the general tendency is to overestimate the contents of your wallet

Selecting dependencies is art

The Not-Invented-Here Syndrome is considered a classic management pathology, in which a team refuses to use a technology that they didn’t create themselves.

But at the same time:

Find the dependencies — and eliminate them. - In defense of not invented here syndrome

Characteristics of Software engineering

From We are not special


A software component rarely specifies tolerancy and it does not wear out (unlike most physical components). Connection to the real physical world can be explicitly declared and controlled.


Commonly heard: Why don't we fix this hardware issues from software? Software can be changed fast. Also changes can be reversed easily.

This leads to ever chainging requirements? If it is possible, then why not? How much does it cost to build a prototype?


Less hard constraints. If it is a hard constraint, software engineers turn to standards and real-time.

The Rational design process is a caricature for all engineering disciplines, real world software engineering is just one adaptation.

Your Coding Philosophies are Irrelevant

Ultimately it's your application/system/product that needs to work.

Updated 27 May 2023