What are acceptance tests, integration tests and component tests? Before discussing test strategy, make sure to have common understanding of these phrases.
To discuss what is covered by eg. a component test, we need to define what a component is. So first specify system or component context and boundaries.
Then we can organise tests based on:
TDD is successful because people started writing regression tests not because X% coverage results in good code.
TDD is a discipline for programmers - Robert C Martin
Not a dogma.
Sometimes we don't have the detailed architecture in front of us when writing code. So we create new units to see what works. Then ask ourselves if this unit is really worth having. This is an iterative design process we do on a daily basis. But how can we write unit tests if we don't know what units are worth writing at all?
Test-first units leads to an overly complex web of intermediary objects and indirection in order to avoid doing anything that's “slow”. - David Heinemeier Hansson
Some other benefits of TDD:
Testing is a spectrum: TDD != Unit TDD
- runs fast
- helps us to localise problems
Robert C. Martin: Working Effectively with Legace Code
It's not a problem if the test covers more than one “unit”. But it needs to run fast.
A test is not a unit test if:
- It talks to the database
- It communicates across the network
- It touches the file system
- It can't run at the same time as any of your other unit tests
- You have to do special things to your environment (such as editing config files) to run it.
Yet it's important to test integrations between units. And we can use unit test frameworks for this purpose, even though this is more than a unit test.
What is a test framework?
Heavy frameworks get in the way.
It is challenging to do unit testing in a constrained environment.
Want acceptance test driven development? Try using Robot. Robot is a DSL trying to be a scripting language. Decouple tests from the code as much as possible. Describe high level tests in a special language.
This is not your favourite scripting language:
Can we actually reuse keywords?
Or there will be few heavily-used keywords, and a really long tail of one-time keywords, which is a pain.
Does it with work with embedded?
Another complaint about robot framework is that when you have an expensive setup like a 4 minute flashing operation, you don't want to repeat it more than necessary. So in a file, you might make the expensive setup a suite-level setup, followed by the tests cases that depend upon it. When this file grows, you might want to refactor it into a multi-file test suite in it's own sub-directory. However, these tests no longer share a suite scope (because robot's “suite scope” is actually a file scope” for legacy reasons) so in practice you may need to tolerate 3000 line files to avoid long setups. - elevation
Alternative if we don't want another obscure DSL: pytest
enable the user, customers or other authorized entity to determine whether to accept the system.
Acceptance test is normally black box testing. Developer test focuses on the code. We can combine these.
Once we have a definition for a component, we can call it a component test.
C/C++ Metrics: Line Counting metrics
Clean code and C++