Boost quality of embedded software early with Dual Target Testing

I held the third monthly meeting of "Test Driven Development for Embedded C" reading group in Tokyo.

We read following two chapters then.

• The fifth chapter "Embedded TDD Strategy"
• This chapters introduces "Dual Target Testing" which is important to do TDD for embedded software development
• The sixth chapter "Year, but..."
• This chapter introduces opposing views for adopting TDD and answers for them

In this entry, I explain what is dual target testing and why we need it.


Concurrent development software and hardware makes embedded system development difficult

Concurrent development of software and hardware is common in embedded system development. As I told in the entry "Is TDD one of the solutions for difficulty of concurrent development of software and hardware?", concurrent development is one of difficulties in embedded system development. Bug whose cause are difficult to find often occurs because we move ahead while software and hardware are low quality.

I give you an example of concurrent development. I often develop embedded software as follows.

• I divide development period into several iterations.
• Each iteration has function scope.
• At the end of the iteration, we integrate software and hardware to make prototype. We evaluate it.
• I continue prototyping and create spec incrementally.

I think this kind of prototyping is common in embedded system development.

In this kind of prototyping, we have difficulties as follows.

• I don't have hardware at the beginning.
• I usually use minimum hardware to check booting OS.
• I also start from checking operation with previous product

After integration of hardware and software, QA team starts evaluation of prototype. They find lots of bugs. I recieved piles of bug reports and I start investigation of bug cause. This is very difficult task because quality of software and hardware is low then. Bugs are everywhere. Software engineers have to spend much time to prepare evidence to say software has no bugs.

This work is not productive. To realize productive embedded system development, software engineers have to spend enormous effort boosting software quality before hardware is ready.


Boost quality of embedded software early with Dual Target Testing

What can we do to boost quality of embedded software before hardware is ready ? James Grenning, the author of Test-Driven Development for Embedded C, recommend Dual Target Testing. This is a technique to make test with both of target hardware and PC possible at the beginning.

Here are key points.

• We build high quality software with exhaustive unit test and refactoring on PC from the beginning. Then we move to  unit test with eval hardware.
• This work reduce unproductive work like finding logical mistake with poor debug environment on eval hardware.
• Compiling for target processor and unit test on eval hardware and target hardware reduce bugs which causes are difference of development environment between host and target.

We can check only with unit test scope but we can start smoothly becausse we get high quality software before we move to eval hardware.

We don't need check logical mistake of module on eval hardware. Reducing unproductive work make us concentrate on spend enormous effort for advanced work.


We need to write unit tests consistently for dual target testing

We need to write many unit test code for dual target testing. This is more a time-consuming way than moving to eval hardware without tests.

It is very easy to check with eval hardware without unit test but everything needs to be done manually. Keeping quality is very difficult if everyone commit lots of code. We spend much time to find cause of bugs.

Writing unit tests for dual targeting test is very time-consuming way in the short term but it is very productive way in the long term because we can find mistake quickly. We reduce debug time and we can spend more advanced work.

We will talk which style is good for us at the next meeting.

How can we do dual target testing?

I will explain detailed techninque for dual target testing next time. Stay tuned :)

Rarejob lesson logs

December 23, 2012
And I took article discussion. Today we discussed about this article.http://online.wsj.com/article/SB10000872396390443675404578059311445951162.html

Discussion points are follows.
·What are possible problems that countries may encounter wh
en dealing with each other?
·How can people ensure harmony between nations with different cultures, values, or religions?

We usually have some problems with countries close to us. Japan has territory issues with China. Philippine has other issues with China.
I think that It is difficult to have harmony at political problem but we can do something at citizens level. Chinese people who have never been to Japan might think Japanese people are bad people but ones who live in Japan don't think like that because they knew what is reality. Reality is that most Japanese people are polite and kind. Once they visited Japan and knew reality, they will change his mind. Having chance to know each other is good way to cooperate each other.

December 23, 2012

I took TOEFL interview test for the first time. I was asked to hear following question, prepare answer in 15 minutes and answer for in 45 minutes. It is very hard.

"Some families think that having a pet is important to provide enjoyment to the members. Others think that a pet would only be a burden at home. Which opinion do you agree with and why? Give specific details and examples in your answer"

March 26, 2012
Total 13.5 hours. I used conversation material for intermediate level today.
In today's lesson, my tutor said that her mom loves the movie 'Hachiko the Akita dog' so they call her pet "Hachiko". That is nice.
I asked my tutor "What is your most favorite food?" Her answer is chocolate. Why do every girls like chocolate? I don't understand why because I don't eat it often.

March 18, 2012

Total 11.5 hours. I used conversation material for intermediate level today.

In today's lesson, we talked about my role in my family. It is to show my family outside of home. For example, I take them to overseas. I took them to Taiwan, Switzerland and Germany. Best city to visit is Zurich. I watched Italian opera with English subtitles. It was my first time to watch opera. So I was very impressed. My family is too.

March 17, 2012

Total 11.25 hours. I used conversation material for intermediate level today.

In English lesson, we talked about small family is better than big one. I don't think so. Big family is fun and you can learn social skill. You have to appeal to your parents in big family. That make you social person. In small family, you cannot get that experience.

But for my own family, I want to have small one. Raising many children costs very much in Tokyo. Two children are enough for me. My tutor has same opinion with me. She grew up in big family but she wants to have small family.

The second monthly meeting of the "Test-Driven Development for Embedded C" book club in Tokyo

The second meeting of "Test-DrivenDevelopment for Embedded C" book club in Tokyo took place on March 4th.

We read the three and fourth chapter

At this time, we read the third chapter "Starting C Module" and the fourth chapter "Testing Your Way to Done". We had first TDD session during these chapters. We wrote test code and product code for LED driver then we refactored both of them.

The third and fourth chapters are highlight of the first half of this book. If you start to read this book, I recommend you to try the example of these chapters by yourself.

There are sample codes on my github repository.

• Test code for LED driver
• LedDriverTest.c
• Product code for LED driver
• LedDriver.c
• LedDriver.h

Always make code stable

There are two very informative topics at this time. First one is to make code stable. Author describes TDD work like this:

And I saw this diagram and I found like these:

• There are two stable states of code, which are stable state that passed tests and one that don't pass.
• TDD doesn't make another state of code.

In the stable state that passes tests, there are not refactored code and refactored code. This topic is well known for TDD new comers.

The stable state that doesn't pass tests is a transient state. Compile error, link error, test failure. TDD makes this process obvious. TDD doesn't make unbalance state that passes tests but include incomplete implementation and mistakes.

You might think that you don't need to do such rigorous way. Of course, some people can implement perfect code at a time but most of us can't do it. We make mistakes. So we need way to find mistakes.

Do not care about performance too much at first but care about readability

Second topic is code readability. Here is informative description in the book.

don't let the performance factor outweigh improved design and readability unless there is proof the code is contributing to a specific performance problem.

Author doesn't mean that performance is not important. We had same opinion. When we deal with problem on performance, we have to find performance bottleneck with profiler. If you put emphasis on performance and write unreadable code while you don't know where bottleneck is, maintenance time increase and have bad effect on costs.

So here is better way:

• Considering a way to meet consider requirement at architecture design phase
• Finding performance bottleneck with profiler at the beginning of architecture implementation phase
• Refactoring code so that each module becomes more readable

We talked about own attempt

First topic is auto build and test with Eclipse. If you use this function, you can also do static analysis, code format and code coverage as follows. This is very easy.

Second topic is 'AdLint'. This is new open source static code analysis tool. I talked my plan to introduce own environment. I want to show demo next time.

Third topic is auto code formant with code formatting tool such as Artistic Style. You need to care when you use it on build process because it may make unintended code format. Manual execution may be better.

Fourth topic is code coverage tool. 'lcov' is good for gcc.

What's next?

Third meeting will take place on April. We will read the fifth chapter "Embedded TDD Strategy" and the sixth chapter "Yeah, but …".  First one is good to adopt TDD to embedded system development. Second one gives us hints to recommend TDD to other people. 

Is TDD one of the solutions for difficulty of concurrent development of software and hardware?

In my 5 year experience of embedded software engineer, I became to think concurrent development of software and hardware is the biggest risk of embedded systems. I thinks TDD is one of the solutions for the problem. So I held a book club of "Test Driven Development for Embedded C (Pragmatic Programmers)" in Tokyo. Here is my thought on how we fight with difficulty of  concurrent development of software and hardware.


Writing code for uncompleted hardware is so hard : (

General process of embedded software development is as follows:

1. Developing software on own computer
2. Debugging on an evaluation board
3. Testing on a target hardware

At the first step, we don't have the target hardware and have to write code for imaginary hardware that we have never seen. After we got the evaluation board, which is the second step, we try to debug on it to find bugs.

Most of engineers that I have met try to debugging on an evaluation board as quickly as possible, so they tend not to care about testability for unit test. As a result, we get high coupling code.

At the time, we have to advance the development with several risks as follows:

• Own computer, evaluation board and target hardware have a little bit different library and compiler.
• The evaluation board is bad in quality at the beginning of the development.
• There are many mistakes in logic of code
• Software engineer has to resolve underlying discrepancy in specification at the end of development.
• etc.

Problems occur someday. They make debugging time longer and distinguishing cause of bugs more difficult.

If we do agile software development for embedded software system, this will be a bottleneck.


Dual-Target Testing

For this problem, the author of "Test Driven Development for Embedded C (Pragmatic Programmers)" recommend hardware-independent software design and unit test on both of own computer and evaluation board from the beginning. This is called "Dual-Target Testing". If we do it from the beginning, we can eliminate bugs and make test easier.


What do we need for Dual-Target Tesing

At first, we need software design techniques to make software more testable. That is discussed in the books like "Test Driven Development: By Example" written by Kent-Beck or "Working Effectively with Legacy Code" written by Michael Feathers.

Second, we need cooperation with hardware engineers for low layer software development. For example, in the book of "Test-Driven Development for Embedded C ", there is a scene that software engineer and hardware engineer are discussing about design of interface between software and hardware so that software engineer can decide design of Test Double.


What "Test-Driven Development for Embedded C" is doing for this?

There are some code examples in my github. There is product code and test code for LED driver.

Key point of this example is a dependency injection at a constructor. Passing writing port address to constructor makes software independent with hardware. We can do unit test on both of own computer and the evaluation board.

This driver has writing port only. State of LED driver is stored at variable ledsImage once, then write to the port. This was an issue that software engineer and hardware engineer discussed in the book.

You might think that this example is quite simple and there are more difficult examples in real world. I think basic idea is the same as this example, which is interface design between software and hardware and low coupling software design against hardware and other software modules.

I will held the third monthly meeting of "Test Driven Development for Embedded C" book club in Tokyo and see this problem deeply : )

The book club of "Test Driven Development for Embedded C" started in Tokyo, Japan

The book club started today : )

Core members of the book club met at "TDD Boot Camp Tokyo for C++" in 2011. This was the event that embedded or game engineers got together and had a TDD practice.

At that time, I wrote my position paper like this: "I want to learn TDD for embedded software, so I want to hold a book club of Test Driven Development for Embedded C. Why don't you join? " To my delight, two engineers said to me that they would join. After I called for participants of this book club on Twitter, 10 engineers joined! That was how we started.

What is motivation of participants?

Participants are embedded software engineers of MFP, medical devices, communication devices, home electronics and etc. We have common point. We have legacy code, code without tests, in our projects and have a hard time of debugging. This is unpredictable activity.

To reduce time of debugging, to discovery bugs rapidly, we are paying attention to TDD.

What we read today

Today we read the first three chapters.

The first chapter "Test Driven Development" introduces what is TDD, purpose of TDD and benefits of TDD.

TDD is one of design techniques, so I can say that I do TDD to improve software design. But the biggest purpose of TDD is to prevent from bugs by getting quick feedback of test results and find mistake rapidly.

The second chapter "Test Driving Tools and Conventions" introduces Unity for C and CppUTest for C/C++ as testing frameworks that were introduced in this book.

CppUTest has a function to add tests into test suite automatically. This is very good for TDD. And it was developed with old function of C++ so that old compiles can compile it.

Unity doesn't have a function to add tests automatically but there is ruby script to do it. It is very convenient.

The third chapter "Starting a C Module " introduces the steps of TDD. We try practical example, LED driver. In this chapter, we can learn embedded software version of Dependency Injection. That is to put physical address of LED driver at runtime. With this way, we can separate software from hardware and we can get benefit to test without hardware.

What we didn't understand

We didn't understand following two points.

The first point. Can we design like this LED Driver example in real project? This example is OK because it is so simple. But in real project, dependency of hardware is so complicated. Changing physical address affects other hardware in some way. So we don't design like this.

Maybe, we can test only where we want to test without representing complicated hardware interaction. (I don't know how to do it)

Or, we don't test bottom of embedded software but focus on upper layer, which is hardware abstraction layer. That is easier to test than bottom layer. In this way, we can't test everything but we can reduce debugging time.

The second point is how we test mass of legacy code. If we need too much time before starting TDD, we lose motivation. To do TDD for real project, we need way to reduce initial cost and introduce TDD incrementally. The 13th chapter may introduce how to fight against legacy code. In near future, we can learn how to do it : )

How I felt after first time

We found that we have so many unknowns this time but we felt joyful that we got together at this book club. I want to say thank you to organizer of "TDD Boot Camp Tokyo for C++" and Mr. James W. Grenning. the author of Test Driven Development for Embedded C.

My job is to make the place that participants can learn a lot and enjoy : )

How to compile googletest with gcc and CMake on Mac OS X SnowLeopard

I often found the way with Visual Studio C++ or XCode on web but I don't use them on my laptop. So I researched the way without IDE.

My settings

• Mac OS X 10.6.8（Snow Leopard）
•  gcc 4.5.4
• cmake 2.8.5

According to googletest's README, you need more than 2.6.4 of cmake but the version of gcc is not clear.

Process

1. Get source code of google test from svn repository
2. Build googletest with cmake
3. Create source file that has main function
4. Create test code
5. Build test code with g++
6. Run the test

(1) Get source code of google test from svn repository

Place the source where you want.

svn checkout http://googletest.googlecode.com/svn/trunk/ gtest-svn

(2) Build googletest with cmake
Make a directory that cmake runs, which is good for anywhere.  {GTEST_DIR} is where theres is googletest. It is success when you get libgtest.a and libgtest_main.a after build.

mkdir mybuild
cd mybuild
cmake ${GTEST_DIR}
make

(3) Create source file that has main function

Create source file that has main function. You can find sample as below link.

http://code.google.com/p/googletest/source/browse/trunk/src/gtest_main.cc

You have to call below functions in main function.

• testing::InitGoogleTest(&argc, argv); 
• RUN_ALL_TESTS();

#include <iostream>
#include "gtest/gtest.h"

GTEST_API_ int main(int argc, char **argv) {
  std::cout << "Running main() from testmain.cc\n";

  testing::InitGoogleTest(&argc, argv);
  return RUN_ALL_TESTS();
}

(4) Craeate test code

#include "gtest/gtest.h"

TEST(firstTest, abs)
{
  EXPECT_EQ(1, abs( -1 ));
  EXPECT_EQ(1, abs( 1 ));
}

(5) Build test code with g++

You have to add the directory where there is googletest's header file into include path. You also have to build test code with static library of google test that you made at (2).

As below, testmain.cc is a file that has main function and mytest.cc is a file that has test code.

g++ -I{GTEST_DIR}/include testmain.cc mytest.cc libgtest.a libgtest_main.a -o mytes

(6) Run the test

Run the test. It is success if you see the test result as below.

auto_ptr and shared_ptr for resource management

Resource management is common troublesome issue for C/C++ programmer. If you don't free a resource that you acquire, you will run out of resource some day.

Scott Meyers, the author of Effective C++, recommended RAII(Resource Acquisition Is Initialization) for resource management. There are several ways to do it but simple way is to use auto_ptr and shared_ptr if your concern is about memory.

Give you a couple of example.

In this example, an error occurs before you delete an object that you acquirer, so resource leak happens. So you have to care about somebody except you because he can write a code that resource leak happens.

Investment* createInvestment() {
  return new Investment;
}

// an example of memory leak                                                                              
void f1(){
  std::cout << "f1" << std::endl;
  try {
    Investment *pInv = createInvestment();

    //If some one write code that can throws error,                                                       
    //destructor of Investment may not be called.                                                         
    throw 1;

    delete pInv;
  } catch(...) {
    std::cout << "catch" << std::endl;
  }
}

In this example, I used auto_ptr for resource management. Now you don't need to care about resource. When the control flow leaves the scope, auto_ptr frees a pointer.

//an example of auto_ptr                                                                                  
void f2(){
  std::cout << "f2" << std::endl;

  try {
    //If you pass a pointer to auto_ptr,                                                                  
    //auto_ptr frees it when the control flow leaves the scope.                                           
    std::auto_ptr pInv(createInvestment());
    throw 1;
  } catch (...) {
    std::cout << "catch" << std::endl;
  }
}

auto_ptr has several limitation. For example several auto_ptr cannot contain the same pointer. So you cannot put auto_ptr into STL container because STL container assume that every object can copy.

void f3(){
  std::cout << "f3" << std::endl;
  //several auto_ptr cannot contain the same pointer.                                                     
  //If you copy auto_ptr, then the source becomes null pointer and                                        
  //the destination can contain the pointer.                                                              
  Investment* pInv1 = createInvestment();
  std::auto_ptr pInv2(pInv1);
  std::auto_ptr pInv3(pInv2);
}

If you need to contain the same pointer with several auto_ptr, you should use shared_ptr in Boost Libraries. But shared_ptr has limitation. You cannot use it for an array.

void f4(){
  //several shared_ptr objects can have the same pointer.                                                 
  boost::shared_ptr pInv1(createInvestment());
  boost::shared_ptr pInv2(pInv1);

  //If every shared_ptr objects are freed, then the pointer                                               
  //that they have also freed.                                                                            
}

Here is a sample code for this entry.
https://github.com/yohei1126/effectivecpp/blob/master/chap3/term13/Investment.cpp

There are other ways for resource management. I will write it some day :)