Acceptance tests, integration tests, component tests. Developers discuss test strategy with the help of these phrases but sometimes the definitions remain unclear. Before specifying eg. a component test, we should first figure out what exactly we mean by a component. This applies to other test levels too: to avoid misunderstandings, first specify the context and the boundaries of the system component we plan to test.

Consider also the environment where the test runs, not just the functionality we want to verify. Sometimes it’s makes sense to run the same tests in different environment (eg. simulated vs physical hardware)

Test strategy

With the system architecture in mind, we can organise tests based on:

Test-driven development

TDD is successful because people started writing regression tests not because X% test coverage results in good code.

TDD is a discipline for programmers - Robert C Martin

Not a dogma.

Sometimes we don’t have the detailed architecture in front of us when writing code. So we create new units to see what works. Then ask ourselves if this unit is really worth having. This is an iterative design process we do on a daily basis. But how can we write unit tests if we don’t know what units are worth writing at all?

Test-first units leads to an overly complex web of intermediary objects and indirection in order to avoid doing anything that’s “slow”. - David Heinemeier Hansson

Some other benefits of TDD:

Against TDD:

Perform tests on different levels: TDD != Unit tests

What is a unit test?

  1. runs fast
  2. helps us to localise problems

Robert C. Martin: Working Effectively with Legace Code

It’s not a problem if the test covers more than one “unit”. But it needs to run fast.

It’s a unit when:

A test is not a unit test if:

Michael Feather

Yet it’s important to test integrations between units. And we can use unit test frameworks for this purpose, even though this is more than a unit test.

Minimal test “frameworks”

What is a test framework?

Heavy frameworks get in the way.

How to do unit testing in a constrained environment (embedded):

Yet another integration test framework

Want acceptance test driven development? Try using Robot. Robot is a DSL trying to be a scripting language. Decouple tests from the code as much as possible. Describe high level tests in a special language.

This is not your favourite scripting language:

Can we actually reuse keywords?

Or there will be few heavily-used keywords, and a really long tail of one-time keywords, which is a pain.

Does it with work with embedded?

Another complaint about robot framework is that when you have an expensive setup like a 4 minute flashing operation, you don’t want to repeat it more than necessary. So in a file, you might make the expensive setup a suite-level setup, followed by the tests cases that depend upon it. When this file grows, you might want to refactor it into a multi-file test suite in it’s own sub-directory. However, these tests no longer share a suite scope (because robot’s “suite scope” is actually a file scope” for legacy reasons) so in practice you may need to tolerate 3000 line files to avoid long setups. - elevation

Alternative if we don’t want another obscure DSL: pytest

Dealing with different test levels

Acceptance test

enable the user, customers or other authorized entity to determine whether to accept the system.

Acceptance test is normally black box testing. Developer test focuses on the code. We can combine these.

Component test

Once we have a definition for a component, we can call it a component test.


C/C++ Metrics: Line Counting metrics



Clean code

Clean code and C++

#sw #testing