PHiLIP on the HiL: Automated Multi-platform OS Testing with External Reference Devices

07/15/2021
by   Kevin Weiss, et al.
0

Developing an operating system (OS) for low-end embedded devices requires continuous adaptation to new hardware architectures and components, while serviceability of features needs to be assured for each individual platform under tight resource constraints. It is challenging to design a versatile and accurate heterogeneous test environment that is agile enough to cover a continuous evolution of the code base and platforms. This mission is even morehallenging when organized in an agile open-source community process with many contributors such as for the RIOT OS. Hardware in the Loop (HiL) testing and Continuous Integration (CI) are automatable approaches to verify functionality, prevent regressions, and improve the overall quality at development speed in large community projects. In this paper, we present PHiLIP (Primitive Hardware in the Loop Integration Product), an open-source external reference device together with tools that validate the system software while it controls hardware and interprets physical signals. Instead of focusing on a specific test setting, PHiLIP takes the approach of a tool-assisted agile HiL test process, designed for continuous evolution and deployment cycles. We explain its design, describe how it supports HiL tests, evaluate performance metrics, and report on practical experiences of employing PHiLIP in an automated CI test infrastructure. Our initial deployment comprises 22 unique platforms, each of which executes 98 peripheral tests every night. PHiLIP allows for easy extension of low-cost, adaptive testing infrastructures but serves testing techniques and tools to a much wider range of applications.

READ FULL TEXT

page 1

page 2

page 12

page 14

research
09/02/2020

Agile SoC Development with Open ESP

ESP is an open-source research platform for heterogeneous SoC design. Th...
research
12/01/2017

ARTENOLIS: Automated Reproducibility and Testing Environment for Licensed Software

Motivation: Automatically testing changes to code is an essential feat...
research
08/31/2022

OSC Community Lab: The Integration Test Bed for O-RAN Software Community

O-RAN Software Community (OSC) is an open-source project collaborated by...
research
11/17/2020

Ginkgo – A Math Library designed for Platform Portability

The first associations to software sustainability might be the existence...
research
04/17/2020

Automated System Performance Testing at MongoDB

Distributed Systems Infrastructure (DSI) is MongoDB's framework for runn...
research
12/18/2020

Simulation Environment for Safety Assessment of CEAV Deployment in Linden

This report presents a simulation environment for pre-deployment testing...
research
09/03/2020

Linking Stakeholders' Viewpoint Concerns and Microservices-based Architecture

Widespread adoption of agile project management, independent delivery wi...

Please sign up or login with your details

Forgot password? Click here to reset