AVR Microcontroller Testing Procedures

In this article, my colleague Anton Kozlov tell us how to cover the main methods of simulation and firmware testing for AVR microcontrollers. Let's start! AVR microcontrollers Why do we need to test ? Any testing of the software is necessary in order to make sure that the program is working. Testing also ensures that the program or certain parts of it meet specific requirements. However, passed tests do not guarantee a complete absence of errors in the project. They only increase the probability of issue detection at early stages of development. An error during compilation has zero cost. At the stage of initial testing, the error cost is equal to the cost of the developer’s time. At the stage of alpha or beta testing, the cost of errors is increasing. The price of fixing errors found after launching a product in a series can cover the cost of the entire project. You are lucky if it’s just about releasing a hotfix (though it might be not so easy for hardware projects). In the worst case, you will have to revoke the whole batch. The main purpose of testing is to save money and time. Comparison of testing in software and hardware projects Software testing is performed on standardized hardware, at a high level of abstraction. The standard environment and ready-made test frameworks allow for unambiguous test results, regardless of the hardware on which the product is running. The situation with the testing of microcontrollers (and hardware devices in general) is simpler but still quite confusing. Testing is easier because microcontrollers are mainly designed for simple tasks. The examples of such tasks are LED blinking, sensor data gathering, simple data processing, controlling display output, communication with other peripheral modules. The main difficulty is that the environment where the program executes depends on the model and manufacturer of the controller. Firmware runs at the low level. Given the stringent performance requirements, testing becomes a daunting task. What kind of testing can I use depending on the task? There are three main options for running tests on embedded systems. Let’s list them all: (on the host machine). 1. Local testing tests start and work quickly, no need to access the device. Advantages: limited testing area. It will not work with external peripherals. A good example of using this method is the testing of a platform-independent computational algorithm that requires a dataset from sensors. The testing of such an algorithm with a real data source would be very inconvenient. It’s much better to prepare a dataset in advance. You can do this by using a small logger program on your unprocessed sensor data. Disadvantages: 2. Testing the MCU in simulation. : no device needed for testing. You can program your own environment for the MCU. Advantages limited accuracy of MCU and environmental simulations, difficulty in creating and configuring such an emulator. Disadvantages: 3. Testing the firmware on real MCU. it is possible to work with the periphery of the MC, the firmware will work out the same way as in production. Advantages: : you need to have a ready-made device with all peripherals and electronic components. The test cycles will take a long time due as you will have to constantly reflash the MCU. It is very difficult to automate testing using this method. Disadvantages In this article we will analyze the first two methods as the most promising for automation. Local Testing Local tests are very good for testing of the firmware parts that are not dependent on the environment. Examples can be any computing algorithms, various containers, lists, queues, trees, high-level exchange protocols, finite machines, etc. Let’s consider an example for AVR microcontrollers with the possibility of local testing of the platform-independent firmware parts. Testing firmware compatible with AVR MCU For AVR MCU the most convenient and productive development environment is . Unfortunately, this environment is not cross-platform and is only available for Windows. Atmel Studio For a clear comprehension of examples in this project, I use open-source tools. I rely on on Ubuntu for source code, for compile, and link firmware, for compiling tests and simulator. VSCode AVR GNU Toolchain gcc The project build process (compile, link, firmware) is done with the utility. This approach allows me to automate test execution and firmware upload to the target system. make For example, let’s consider the firmware for the microcontroller that implements the functionality of a simple thermometer. Atmega1284 Temperature measurement is performed by reading the voltage on the voltage divider (which includes a thermistor), converting the ADC value into temperature values, and displaying it on the 1602 ( ) LCD screen. hd44780 The function of communication with the internal and external periphery can be performed via operating the microcontroller registers. Nevertheless, our firmware has functionality which it is necessary to test (converting ADC value into temperature and then displaying it on the LED screen). Let’s consider an example of native testing of the firmware functionality. Repo link: https://github.com/maddevsio/AVR_Testing You will need to compile. AVR GNU Toolchain ├── builds — executable files │ ├── firmware │ └── tests └── src — sources files ├── inc — peripherials drivers, computing source │ ├── adc.c │ ├── adc.h │ ├── calculate.c │ ├── calculate.h │ ├── lcd.c │ └── lcd.h ├── main.c — entry point ├── Makefile ├── obj — objects linking firmware ├── test_obj — objects linking tests └── tests — test sources ├── test.c └── test.h for for To create a project with tests, you need to separate sources by function. What runs only for MCU should not be mixed with source files that can run on local systems. That is not an easy task. Also, you should use two compilers, making some modules platform-dependent and others not. Let’s look at for more details. main.c #include #include #ifdef TEST directive, than use testing librarry #include # drivers, initialize global variables #include #include #define ADC_PIN uint16_t adcValue; float temperature; #endif int main( ) { #ifdef TEST is defines RUN_TESTS(); # adcInit(); lcdInit(); (;;){ char temp[ ] = { }; adcValue = adcRead(ADC_PIN); temperature = adcToTemp(adcValue); sprintf(temp, , temperature); lcdPrintLn(temp); } #endif } "inc/calculate.h" // Testing module //If we define TEST "tests/test.h" else // Else we connect peripherials "inc/adc.h" "inc/lcd.h" 1 void // Run test if TEST directive else // Else do what firmware needs to do for 10 0 "%f" In local testing there are two sections of : main.c For testing functionality For implementing the main program cycle. Let’s consider testing conversion of ADC values to temperature (src/tests/test.c file): struct CalcTestCase structure, consist ADC value and calculated temperature value according to ADC { uint16_t adcVal[ ]; double temperature[ ]; }; struct CalcTestCase calcCase = { .adcVal= { , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , }, .temperature = { , , , , , , , , , , , , , , , , } }; double temperatureDeviation = ; Maximum absolute deviation RUN_TESTS( ){ test_AdcCalc(); } test_AdcCalc( ){ all testcase values uint8_t adc_caseSize = sizeof(calcCase.adcVal)/sizeof(calcCase.a dcVal[ ]); (int i = ; i = value - diviation))) { ; } { ; } } // Testcase 100 100 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200 // … … … 810 820 830 840 850 860 870 880 890 900 910 920 930 940 950 960 970 980 990 -55.609 -45.834 -39.719 -35.168 -31.5 -28.4 -25.699 -23.293 -21.116 // … … … 83.642 87.215 91.284 95.869 101.222 107.639 115.620 125.990 0.12 // void void void void // Iterate over 0 for 0 -1 // Using from // if return true else return false When implementing such a test, it is necessary to split the in two sections: the first section will use a compiler from AVR Toolchain — (compiler for AVR microcontrollers), and the second section will use for compiling of the platform-independent executable files. Makefile avr-gcc gcc This example enables you to test the functionality of converting ADC values to temperature using calculated temperature values depending on the input voltage of the microcontroller. When you change the conversion functionality, you just need to run the tests and make sure that they have not detected any errors. To compile and run the tests, do the following: $ cd AVR_Testing/src $ make tests after a successful compilation, the executable test file will appear in the directory. AVR_Testing/builds/tests/ Native test development is quite a huge task, and implementing it takes a lot of time. Therefore, it will be more productive to use one of the existing frameworks for testing. Testing with Unity framework Let’s run the same tests using Unity, a simple framework for testing of embedded systems. In the same project repository (the unity-framework branch) you will find an example of the framework usage. When using Unity, the compilation of tests and firmware is done separately, like in native tests. The testing of each module has its own entry point and can be run independently if you use Unity. To start using the framework, simply clone the framework’s repository. git clone https: //github.com/ThrowTheSwitch/Unity.git is used to generate sources of an executable file from a test case file. To install it, do the following: Ruby sudo apt install ruby-full Another remark: to use unity in your project, you need to specify the absolute path to it in the variable of your . $(UNITY_ROOT) Makefile An example of a testcase source: #include #include double temperatureDeviation = ; struct CalcCase { uint16_t adcVal[ ]; double temperature[ ]; }; struct CalcCase calcTestCase = { .adcVal= { , , , , , , , , , , , , , , , , , , }, .temperature = { , , , , , , } }; setUp( ) { } tearDown( ) { } test_adcToTemp( ) { int8_t caseSize = sizeof(calcTestCase.adcVal)/sizeof(calcTe stCase.adcVal[ ]); #ifdef SHOW_TEST_RESULT printf( ); #endif (int i= ;i<caseSize - ;i++){ double calculatedTemp = adcToTemp(calcTestCase.adcVal[i]); #ifdef SHOW_TEST_RESULT printf( ,calculatedTemp,calcTestC ase.temperature[i]); #endif TEST_ASSERT_DOUBLE_WITHIN(temperatureDevi ation,calcTestCase.temperature[i],calcula tedTemp); } } "calculate.h" "unity.h" 0.12 100 100 10 20 30 40 50 60 70 80 90 100 // ... ... ... 910 920 930 940 950 960 970 980 990 -55.609 -45.834 -39.719 -35.168 // ... ... ... 107.639 115.620 125.990 void void void void void void 0 "Adc to temp assertion. calculated value:actual value\n" for 0 1 "%f:%f\n" In this example, we only need to specify the header file of the sources of the module under test and implement the verification of the operation of converting ADC values to temperature with the acceptable value for the observational error. Besides writing test cases for Unity and specifying the path to the framework, you should also add special compiler directives. In this case, we use a comparison of numbers with a floating point of increased accuracy. Testing of types is disabled in the framework by default, you need to add directives to enable the double types: double -DUNITY_INCLUDE_DOUBLE #Enables double type -DUNITY_DOUBLE_PRECISION= f — e # presiccion double comparsion 0.001 Set of After you hit , Ruby will generate test entry point files, they will get compiled and run. make tests Simulator using testing In most hardware projects, the microcontroller is used to interact with external and internal peripherals, whose behavior is very difficult and sometimes impossible to simulate during tests. Target system simulators are very helpful in solving this problem. For microcontrollers with AVR architecture, there are several simulation systems listed below: is the development environment for AVR microcontrollers that I have mentioned earlier. (The one that works only for Windows.) It includes a built-in simulator for almost all AVR microcontroller models and a handy toolkit for creating and controlling both internal and external peripherals. The drawback is the lack of cross-platformity and cumbersomeness — it’s hard to automate the simulation. Atmel Studio is an open-source cross-platform C/C++ project that offers its own AVR microcontroller simulation systems. There are several models of microcontrollers, the ability to write debug scripts on Tcl/Tk and Python. The disadvantages are the complicated mechanism of adding new MC models to the simulator, complex documentation, and the fact that the project remains unsupported for several years. Simulavr — relatively new (and at the same time quite flexible) cross-platform project for writing AVR MCU simulators in C/C++. Because of the similar name, it can be confused with the previous tool, however, these are completely different projects. Simavr has a large number of AVR MCU models, the architecture enables you to easily add new devices and models. It also has integration with PlatformIO (vscode extension for embedded systems development) and clear examples of how to use the tool with different kinds of peripherals. Disadvantages: the lack of a description of the project build and rather obscure documentation. simavr I chose the as the most promising option. simavr To start using simavr, just clone and bulild the repository. git clone https: cd simavr make //github.com/buserror/simavr/ After a successful build, you can test the simulator’s performance by running tests or examples. To run the executable files of this simulator, you need to create a symbolic link to the sources in the firmware directory. For example, in the directory: simavr/tests atmega88_timer16.axf - mcu firmware that teed to test obj%your_system_architecture% /test_atmega 88_timer16.tst - compiled enviroment, peripherials and simulator Here’s the symbolic link for executable file creation: ln -s obj%your_system_architecture% ega88timer /test_atmega 88_timer16.tst mega88timer ./m In the you will find folders with sources of simulations, and the parts folder contains the sources of common peripherals. simavr/examples For a more immersive effect, you can run examples with graphics — such as board_hd44780, board_ssd1306. Simavr provides a wide range of tools for the following types of tasks: Development of custom virtual boards with microcontrollers and peripherals.Development of virtual electronic components.Managing simulation behavior on time segments up to microcontroller tact.Connection of avr-gdb debugger. A full description of the simulator’s features can be found in the project repository. Structurally any simulator on a simavr represents sources of board, peripherals, and compiled firmware (by the way, a simavr simulator enables you to load the same firmware like on a real device, changes are not required). For the example of testing let’s look at the tests of the same thermometer while using a simulator (see the simavr-testing branch). The structure of directory is as follows: src/tests/sim ├── adcToLcd.c - board sources ├── main_wrapper.c — wrapper firmware entry point - main.c ├── Makefile ├── obj-x86_64-linux-gnu — objects folder(name depends system achitecture) └── parts — board peripherials (virtual electronic components) ├── hd44780.c ( lcd implementation) └── hd44780.h for of 1602 Schematic of the virtual board: Simavr has a very simple, though not quite obvious project structure. Let us have a look at the most important moments of a board implementation: is the wrapping of the entry point for firmware. It provides the compiler and the simulator with additional information about firmware, power supply voltage, and other parameters (full description of all parameters can be found in ). main_wrapper.c simavr/simavr/sim/avr/avr_mcu_section.h #undef F_CPU #define F_CPU #include #define VCC #define AVCC #define AREF AVR_MCU(F_CPU, ); frequency and model AVR_MCU_VOLTAGES(VCC, AVCC, AREF); power supply, adc, aref voltage #include 8000000 "avr_mcu_section.h" 5000 5000 5000 "atmega1284" //Set MCU //Set "../../main.c" contains sources of the board as well as descriptions for manipulations with the periphery, data ports and operating time intervals between actions of the periphery. adcToLcd.c int main(int argc, char *argv[]) { firmwareInit(firmware,argv[ ]); initializing, firmware flashing hd44780_init(avr, &hd44780, , ); setConnections(); together avr_cycle_timer_register_usec(avr, lcdTimer, lcdDataGathering, NULL); lcdDataGathering after lcdTimer usec (!simulationCompleted){ until simulation is complete avr_run(avr); } (run_test()) simulation data { printf( ); } { printf( ); } } 0 //MCU 16 2 //LCD initializing // Connecting all parts //Timer registration. It will fire while //Wait if //Run test with "TEST PASSED.\n" else "TEST FAILED.\n" When initializing firmware, the board searches for the specified firmware elf-file, then creates MCU and uploads the firmware to it. To initialize and work with the display, in the parts directory find the sources of the LCD screen on the controller (for the project I just took them from and refactored the function of displaying the data on the screen for parsing and returning a double-value). hd44780 simavr/examples/parts Next, the function of connecting peripheral parts to the controller is applied, which uses such methods as: setConnections avr_io_getirq(avr_t *avr, uint32_t ctl, int index) — it returns a pointer to the unique identifier of the I/O port PIN. Arguments are controller identifier, port, PIN. (See simavr/simavr/sim/sim_io.h for details). avr_connect_irq(avr_irq_t *src, avr_irq_t *dst) — this is the function of connecting one PIN I/O (peripheral or microcontroller) to another. The example of a use for this function: avr_connect_irq(avr_io_getirq(avr, AVR_IOCTL_IOPORT_GETIRQ(‘D’), \ ),hd44780.irq + IRQ_HD44780_RS); 4 — connecting PORTD PIN 4 of MCU to RS PIN of LCD. You can raise signals to be sent to the I/O ports using the following function: avr_raise_irq(avr_irq_t *irq, uint32_t value) — it accepts the port identifier and the required value. Value can be equal 1 or 0 for digital inputs, or store a value in millivolts for analog inputs. Simavr has a convenient mechanism for setting time for external events by registering timers. On triggering of the timer, a callback function is executed. There is a function for registering the timer: avr_cycle_timer_register_usec(struct avr_t *avr, uint32_t when, avr_cycle_timer_t timer, *param) void void — it accepts the controller identifier, the actual time after which the timer will trigger, the callback which will trigger and its optional parameters. In our board, the callback performs the function of reading the screen output, switching to the ADC output of the next voltage value and registering a new timer. For adequate reading of information from the screen, the board uses a timer setting the delay until the display is filled with new data. To perform every tick of MCU please use: avr_run(*mcu identifier) — using this mechanism enables you to suspend the simulation for some time to verify the data or perform calculations. It also serves as a mechanism for launching the simulator in a separate thread. In our example, we use a simple implementation with running the simulation in a single thread, collecting the simulation results, and testing the collected data. In order not to overload the code with two testing methods, it was decided to put the test case into a separate file which connects to the project both when compiling tests on Unity and when using the simulator. adc-temp_test.c To compile and run the simulation, just specify the absolute path to simavr in , in the variable, and execute it in the src directory: src/tests/sim/Makefile $(SIMAVR) make sim-test cd tests/sim ./adcToLcd Unlike modular tests, tests using a simulator (in fact, these are integration tests) allow to fully evaluate the correctness of firmware operation. Simulation of ADC conversion is a clear example of the quality of testing in our project. Due to the limitations of ADC discretization, when calculating low values the simulation goes with considerable observational error than it is found out during the unit tests. For larger projects, you can easily integrate the simulator with the Unity framework and automate the testing process via CI/CD. Testing the firmware on real hardware The launch of hardware tests is mostly done manually, using on-chip debugging systems such as JTAG. To automate hardware tests, it is necessary to emulate the behavior of the external periphery using another microcontroller. The adequacy of the behavior of the emulated periphery is questionable. In this case, test might even be more expensive to write than the firmware itself. Anyway, this approach ensures maximum accuracy and closeness to real conditions. Description of test preparation methods deserves a separate article, and we will tackle this topic in the next publications. Conclusion There is a lot more to tell about firmware testing. There is no standard method for testing a platform-dependent code. However, general recommendations when writing tests and simulators for specific models of AVR microcontrollers allow to fully automate the firmware testing process. I hope this article has helped you to acquire basic knowledge about the testing of AVR microcontrollers. Previously published at https://blog.maddevs.io/avr-mcu-testing-f925e84b7023 .