Book review: Test Driven Development for Embedded C

I wrote Japanese article to introduce “Test Driven Development for Embedded C” for embedded software developers in Japan last month. This entry is English version of it.

----------

As you can see with the title, this is the book about Test Driven Development for embedded software with C language. The author, James Grenning, is training, coaching and consulting agile software development or TDD over the world. He is also the inventor of agile estimation technique “planning poker” and one of the authors of agile manifesto.

Must read book for embedded software developer

Recently Japanese TDD passionates like @t_wada are holding TDD boot camp in major cities of Japan. This is an event for TDD beginners to have an exercise of TDD. People get together at one place, make pair for exercise and do TDD with specific problems.

Thanks to them, we can have more chance to exercise TDD and people are becoming to discuss practical topic about TDD on their blog or social media. TDD is not majority in the industry yet but I am feeling TDD is becoming common technique for us.

On the other hand, most of embedded software are left out of boom of TDD boot camp. They think that they have nothing is to do with TDD because every article on book or blog is written about enterprise or web system. They are things on other planets.

This is a book for you, embedded software developers :) This book deals with typical example of embedded software and C/C++ testing framework and explain how to do TDD with very tiny step of carefully. It also can be good textbook for module design of C code.

Why embedded software developer need TDD?

Embedded software developer might think why they need TDD. The author answer the question in the chapter 5 “Embedded TDD Strategy” of the book. He explain the we can solve “the target hardware bottleneck” problem, which is common for embedded software development.

The target hardware bottleneck is a problem that target hardware could be a bottleneck disturbs software testing. Generically, target hardware is different from host computer that is used to develop software. And we can have these situation:

• The problems leads to delay or increasing of working hour for software testing
• Target hardware are not ready till the end of project
• Target hardware is expensive and we can’t prepare enough number of target hardware 
• It takes long time to build and upload software to target hardware 
• The problems leads to difficulties to find bugs
• It is difficult to identify where bugs are because the target hardware also have bugs in the beginning
• Compiler for target hardware is different from one for host computer. Then we find unintentional behavior of software on the target hardware

To solve the target hardware bottleneck problem, the author recommend these approaches:

• Ensure quality of software by TDD on host computer before testing on the target hardware
• Do same tests on both of host computer and target hardware to find bug that are happend because of different between them.

And the author recommends TDD cycle for embedded software development as follows:

The book focuses on the stage 1 on the cycle. It doesn’t explain stage 2 to 5 in detail but you can find more information about step 5 on the author’s paper (PDF) “Scenario Testing with Executable Use Cases”.

I also write this cycle on my blog entry.

How is the book organized

The book consisted of three part.

In the first part “Test Driven Development”, the author introduces the process of TDD when we develop LED device driver. He also introduces why embedded software developer need TDD, test cycle including TDD and testing framework for C, Unity and CppUTest.

The second part is about technique to replace dependant module to test with C. It is very difficult to replace module with C, the only ways we can do are link substitution, preprocessor substitution and function pointer substitution. It also explains C++ mocking library “CppUMock” to test complex interaction with dependent module.

In the third part “Design and Continuous Improvement”, knowledge and technique to improve design continuously such as SOLID principle, test for legacy code and test patterns are introduced. If you are not familiar with object oriented design and programming, it is very difficult to understand. You need to read the books about object oriented design and programming first.

Please refer to the full table of contents on the official site.

How to read the book

For those who wants to know characteristics of TDD for embedded software

When I introduce the book to my coworker and he asked me “What is the difference between TDD for embedded and the others?” I think that basic points are same. For example, there is no difference between dependency to module and dependency to hardware.

I think that the most biggest difference is language constraint. Comparing to object oriented language, you need more technique to make test doubles, which is module to replace dependent module. In order to understand the technique, please refer to the chapter 7 “Introducing Test Doubles”, the chapter 8 “Spying on the Production Code” and the chapter 9 “Runtime-Bound Test Doubles”.

On the other hand, there is special bonus for embedded software developer. As I wrote before, we can solve the hardware bottleneck problem. Please refer to “Benefit for Embedded” at the chapter 1 and the chapter 5 “Embedded TDD Strategy”.

For those who wants to try TDD with example

To put TDD in practice, you need not only knowledge but technique. You must acquire technique by doing exercise. You can read the process to test and refactor your code for example of LED device driver in the chapter 3 “Starting a C Module” and the chapter 4 “Testing Your Way to Done”.  Try them.

Example in the chapter 3 and 4 is quite simple. You need test doubles in actual development. You can read about test doubles in the chapter 7 ”Introducing Test Doubles”, the chapter 8 ”Spying on the Production Code” and the chapter 9 “Runtime-Bound Test Doubles”.

For those who wants to know better C module design

TDD covers not only design and implement activity but also continuous improvement to better design by refactoring. You need to know what is better design to improve design of your software.

The chapter 11 “SOLID, Flexible and Testable Designs” introduces better design principles and implementation with C. You can understand how to design low-coupling module design with C.

There is also explanation to eliminate piles of if / switch-case statements by implementing polymorphism in the object oriented design concept with C. The code is tricky and it is very difficult to understand if you don’t understand object oriented design. In the case, you need to read object oriented software design book such as “Agile Software Development” first.

How Japanese readers say about the book

In conclusion, I introduce comments from the members of “Test Driven Development for Embedded C” studying group.

• It is very helpful to understand how TDD master think and do because every tiny step and design decisions are written at the beginning of the book.  
• I read the book and solve some exercises, then I could understand basic rhythm of TDD. 
• I was wondering how to apply object oriented design to C code before I read this book. Thanks to the book, I understood how to do it.
• I think that it is very difficult to understand the book at first if the readers understand object oriented design and test. Example of C code is not enough for beginners.
• This book is very good as index of modern software testing. It includes TDD, test for legacy code, SOLID principle and refactoring, xUnit Test Patterns. Every embedded software developer should read this!

Test Driven Development for Embedded C: Introducing runtime-bounded test doubles and the mock object

I had a meeting with reading group of "Test Driven Development for Embedded C" in Tokyo, Japan. We read the chapter 9 "Runtime-Bounded Test Doubles" and the chapter 10 "The Mock Object", then we discussed them. In this entry, I introduce the chapters and some topics from our discussion.

The chapter 9 "Runtime-Bounded Test Doubles"

When we implement product code with C, we have three ways to replace dependent module with test doubles. They are preprocessor substitution, link substitution and function pointer substitution.

Link-time substitution, which is introduced in chapter 7 "Introducing Test Doubles", is not flexible because it can't be replaced at runtime. If you want to use product code once and use test doubles at other time, function pointer substitution, which is introduced at chapter 9, is effective way.

Although function pointer substitution is very flexible, it has a side effect. Heavy usage of preprocessor or function pointer substitution reduces code readability. You should use them in a limited way.

By the way, the author doesn't explain preprocessor substitution. He may doesn't like it because of side effect.

The chapter 10 "The Mock Object"

The chapter 10 introduces the mock object, which is not so popular among embedded software engineers. The subject is a device driver for existent flash memory. The mock object is useful way to ensure quality of device driver before integrate it with hardware because:

• It is difficult to test complex interaction between software and hardware based on specific protocol with usual testing framework.
• It may be impossible to test error case in real device. 

Most part of the chapter 10 focuses on how to implement the mock object in C. You can also know "CppUMock", which is C++ mocking library, and "CMock", which is mock generation tool for C.

I found that mock object is usually complicated. If you implement mock object in C by yourself, you must to test it thoroughly before you use it. Test may be unnecessary if a mocking library assure quality of mock object.

You might think that making mock object is waste of time. But I think that it worth spending cost to make mock object and test with mock object. The more you standardize  device I/O, the more effectiveness of test grow. You can ensure quality of device driver more and more before integrate it with hardware.

At last we discussed why the mock object is popular among enterprise or web engineers but not among embedded engineers. Our conclusion is that mocking library with LL is very useful because LL has less constraint and mocking library is well maintained to use easily. On the other hand, library for C/C++ are still on the way to become useful.

This is one of difficulties for embedded engineers. To gain strength from TDD, we need to prepare way to overcome language constraint.

Boost quality of embedded software early with Dual Target Testing

I held the third monthly meeting of "Test Driven Development for Embedded C" reading group in Tokyo.

We read following two chapters then.

• The fifth chapter "Embedded TDD Strategy"
• This chapters introduces "Dual Target Testing" which is important to do TDD for embedded software development
• The sixth chapter "Year, but..."
• This chapter introduces opposing views for adopting TDD and answers for them

In this entry, I explain what is dual target testing and why we need it.


Concurrent development software and hardware makes embedded system development difficult

Concurrent development of software and hardware is common in embedded system development. As I told in the entry "Is TDD one of the solutions for difficulty of concurrent development of software and hardware?", concurrent development is one of difficulties in embedded system development. Bug whose cause are difficult to find often occurs because we move ahead while software and hardware are low quality.

I give you an example of concurrent development. I often develop embedded software as follows.

• I divide development period into several iterations.
• Each iteration has function scope.
• At the end of the iteration, we integrate software and hardware to make prototype. We evaluate it.
• I continue prototyping and create spec incrementally.

I think this kind of prototyping is common in embedded system development.

In this kind of prototyping, we have difficulties as follows.

• I don't have hardware at the beginning.
• I usually use minimum hardware to check booting OS.
• I also start from checking operation with previous product

After integration of hardware and software, QA team starts evaluation of prototype. They find lots of bugs. I recieved piles of bug reports and I start investigation of bug cause. This is very difficult task because quality of software and hardware is low then. Bugs are everywhere. Software engineers have to spend much time to prepare evidence to say software has no bugs.

This work is not productive. To realize productive embedded system development, software engineers have to spend enormous effort boosting software quality before hardware is ready.


Boost quality of embedded software early with Dual Target Testing

What can we do to boost quality of embedded software before hardware is ready ? James Grenning, the author of Test-Driven Development for Embedded C, recommend Dual Target Testing. This is a technique to make test with both of target hardware and PC possible at the beginning.

Here are key points.

• We build high quality software with exhaustive unit test and refactoring on PC from the beginning. Then we move to  unit test with eval hardware.
• This work reduce unproductive work like finding logical mistake with poor debug environment on eval hardware.
• Compiling for target processor and unit test on eval hardware and target hardware reduce bugs which causes are difference of development environment between host and target.

We can check only with unit test scope but we can start smoothly becausse we get high quality software before we move to eval hardware.

We don't need check logical mistake of module on eval hardware. Reducing unproductive work make us concentrate on spend enormous effort for advanced work.


We need to write unit tests consistently for dual target testing

We need to write many unit test code for dual target testing. This is more a time-consuming way than moving to eval hardware without tests.

It is very easy to check with eval hardware without unit test but everything needs to be done manually. Keeping quality is very difficult if everyone commit lots of code. We spend much time to find cause of bugs.

Writing unit tests for dual targeting test is very time-consuming way in the short term but it is very productive way in the long term because we can find mistake quickly. We reduce debug time and we can spend more advanced work.

We will talk which style is good for us at the next meeting.

How can we do dual target testing?

I will explain detailed techninque for dual target testing next time. Stay tuned :)

The second monthly meeting of the "Test-Driven Development for Embedded C" book club in Tokyo

The second meeting of "Test-DrivenDevelopment for Embedded C" book club in Tokyo took place on March 4th.

We read the three and fourth chapter

At this time, we read the third chapter "Starting C Module" and the fourth chapter "Testing Your Way to Done". We had first TDD session during these chapters. We wrote test code and product code for LED driver then we refactored both of them.

The third and fourth chapters are highlight of the first half of this book. If you start to read this book, I recommend you to try the example of these chapters by yourself.

There are sample codes on my github repository.

• Test code for LED driver
• LedDriverTest.c
• Product code for LED driver
• LedDriver.c
• LedDriver.h

Always make code stable

There are two very informative topics at this time. First one is to make code stable. Author describes TDD work like this:

And I saw this diagram and I found like these:

• There are two stable states of code, which are stable state that passed tests and one that don't pass.
• TDD doesn't make another state of code.

In the stable state that passes tests, there are not refactored code and refactored code. This topic is well known for TDD new comers.

The stable state that doesn't pass tests is a transient state. Compile error, link error, test failure. TDD makes this process obvious. TDD doesn't make unbalance state that passes tests but include incomplete implementation and mistakes.

You might think that you don't need to do such rigorous way. Of course, some people can implement perfect code at a time but most of us can't do it. We make mistakes. So we need way to find mistakes.

Do not care about performance too much at first but care about readability

Second topic is code readability. Here is informative description in the book.

don't let the performance factor outweigh improved design and readability unless there is proof the code is contributing to a specific performance problem.

Author doesn't mean that performance is not important. We had same opinion. When we deal with problem on performance, we have to find performance bottleneck with profiler. If you put emphasis on performance and write unreadable code while you don't know where bottleneck is, maintenance time increase and have bad effect on costs.

So here is better way:

• Considering a way to meet consider requirement at architecture design phase
• Finding performance bottleneck with profiler at the beginning of architecture implementation phase
• Refactoring code so that each module becomes more readable

We talked about own attempt

First topic is auto build and test with Eclipse. If you use this function, you can also do static analysis, code format and code coverage as follows. This is very easy.

Second topic is 'AdLint'. This is new open source static code analysis tool. I talked my plan to introduce own environment. I want to show demo next time.

Third topic is auto code formant with code formatting tool such as Artistic Style. You need to care when you use it on build process because it may make unintended code format. Manual execution may be better.

Fourth topic is code coverage tool. 'lcov' is good for gcc.

What's next?

Third meeting will take place on April. We will read the fifth chapter "Embedded TDD Strategy" and the sixth chapter "Yeah, but …".  First one is good to adopt TDD to embedded system development. Second one gives us hints to recommend TDD to other people. 

Is TDD one of the solutions for difficulty of concurrent development of software and hardware?

In my 5 year experience of embedded software engineer, I became to think concurrent development of software and hardware is the biggest risk of embedded systems. I thinks TDD is one of the solutions for the problem. So I held a book club of "Test Driven Development for Embedded C (Pragmatic Programmers)" in Tokyo. Here is my thought on how we fight with difficulty of  concurrent development of software and hardware.


Writing code for uncompleted hardware is so hard : (

General process of embedded software development is as follows:

1. Developing software on own computer
2. Debugging on an evaluation board
3. Testing on a target hardware

At the first step, we don't have the target hardware and have to write code for imaginary hardware that we have never seen. After we got the evaluation board, which is the second step, we try to debug on it to find bugs.

Most of engineers that I have met try to debugging on an evaluation board as quickly as possible, so they tend not to care about testability for unit test. As a result, we get high coupling code.

At the time, we have to advance the development with several risks as follows:

• Own computer, evaluation board and target hardware have a little bit different library and compiler.
• The evaluation board is bad in quality at the beginning of the development.
• There are many mistakes in logic of code
• Software engineer has to resolve underlying discrepancy in specification at the end of development.
• etc.

Problems occur someday. They make debugging time longer and distinguishing cause of bugs more difficult.

If we do agile software development for embedded software system, this will be a bottleneck.


Dual-Target Testing

For this problem, the author of "Test Driven Development for Embedded C (Pragmatic Programmers)" recommend hardware-independent software design and unit test on both of own computer and evaluation board from the beginning. This is called "Dual-Target Testing". If we do it from the beginning, we can eliminate bugs and make test easier.


What do we need for Dual-Target Tesing

At first, we need software design techniques to make software more testable. That is discussed in the books like "Test Driven Development: By Example" written by Kent-Beck or "Working Effectively with Legacy Code" written by Michael Feathers.

Second, we need cooperation with hardware engineers for low layer software development. For example, in the book of "Test-Driven Development for Embedded C ", there is a scene that software engineer and hardware engineer are discussing about design of interface between software and hardware so that software engineer can decide design of Test Double.


What "Test-Driven Development for Embedded C" is doing for this?

There are some code examples in my github. There is product code and test code for LED driver.

Key point of this example is a dependency injection at a constructor. Passing writing port address to constructor makes software independent with hardware. We can do unit test on both of own computer and the evaluation board.

This driver has writing port only. State of LED driver is stored at variable ledsImage once, then write to the port. This was an issue that software engineer and hardware engineer discussed in the book.

You might think that this example is quite simple and there are more difficult examples in real world. I think basic idea is the same as this example, which is interface design between software and hardware and low coupling software design against hardware and other software modules.

I will held the third monthly meeting of "Test Driven Development for Embedded C" book club in Tokyo and see this problem deeply : )

The book club of "Test Driven Development for Embedded C" started in Tokyo, Japan

The book club started today : )

Core members of the book club met at "TDD Boot Camp Tokyo for C++" in 2011. This was the event that embedded or game engineers got together and had a TDD practice.

At that time, I wrote my position paper like this: "I want to learn TDD for embedded software, so I want to hold a book club of Test Driven Development for Embedded C. Why don't you join? " To my delight, two engineers said to me that they would join. After I called for participants of this book club on Twitter, 10 engineers joined! That was how we started.

What is motivation of participants?

Participants are embedded software engineers of MFP, medical devices, communication devices, home electronics and etc. We have common point. We have legacy code, code without tests, in our projects and have a hard time of debugging. This is unpredictable activity.

To reduce time of debugging, to discovery bugs rapidly, we are paying attention to TDD.

What we read today

Today we read the first three chapters.

The first chapter "Test Driven Development" introduces what is TDD, purpose of TDD and benefits of TDD.

TDD is one of design techniques, so I can say that I do TDD to improve software design. But the biggest purpose of TDD is to prevent from bugs by getting quick feedback of test results and find mistake rapidly.

The second chapter "Test Driving Tools and Conventions" introduces Unity for C and CppUTest for C/C++ as testing frameworks that were introduced in this book.

CppUTest has a function to add tests into test suite automatically. This is very good for TDD. And it was developed with old function of C++ so that old compiles can compile it.

Unity doesn't have a function to add tests automatically but there is ruby script to do it. It is very convenient.

The third chapter "Starting a C Module " introduces the steps of TDD. We try practical example, LED driver. In this chapter, we can learn embedded software version of Dependency Injection. That is to put physical address of LED driver at runtime. With this way, we can separate software from hardware and we can get benefit to test without hardware.

What we didn't understand

We didn't understand following two points.

The first point. Can we design like this LED Driver example in real project? This example is OK because it is so simple. But in real project, dependency of hardware is so complicated. Changing physical address affects other hardware in some way. So we don't design like this.

Maybe, we can test only where we want to test without representing complicated hardware interaction. (I don't know how to do it)

Or, we don't test bottom of embedded software but focus on upper layer, which is hardware abstraction layer. That is easier to test than bottom layer. In this way, we can't test everything but we can reduce debugging time.

The second point is how we test mass of legacy code. If we need too much time before starting TDD, we lose motivation. To do TDD for real project, we need way to reduce initial cost and introduce TDD incrementally. The 13th chapter may introduce how to fight against legacy code. In near future, we can learn how to do it : )

How I felt after first time

We found that we have so many unknowns this time but we felt joyful that we got together at this book club. I want to say thank you to organizer of "TDD Boot Camp Tokyo for C++" and Mr. James W. Grenning. the author of Test Driven Development for Embedded C.

My job is to make the place that participants can learn a lot and enjoy : )