View page as slide show

Communications software and architecture

Testing and verification

Testing and verification - Goals

  • Bug free software
  • Correct behaviour
    • Behaves as intended
    • Meets the requirements described in use cases
    • Follows the specifications and standards.
  • Acceptable performance
    • Memory leak avoidance
    • Usability

Tests in communication software

  • Traditional tests
    • Unit testing
    • Integration testing
    • Acceptance testing
      • Conformance testing
      • Load testing
      • In-field testing
  • Cleanroom engineering (not in scope of course)
    • Formal verification
    • Statistical usage testing

Unit testing

  • testing that individual unit of source code works properly.
    • unit = smallest testable part
      • function, method, procedure, class
  • test cases should be independent from each other
  • Unit testing for communication software is similar than to any other software
    • test cases for individual FSM state transitions and complex FSm transactions
  • testing frameworks like in traditional Software engineering
    • JUnit,CPPUinit …

Integration testing

  • Testing that the independently working units are working together correctly
  • Units that are not available can be replaced with imitators (simulators)
    • drivers: generate input messages for the real units under test
    • stubs: passive imitators that accept outputs from units under test. May send also replies to outputs.
  • Communication software fits well for integration testing
    • Well defined hierarchy: Layered architecture
    • Message based communications
    • Well defined interfaces
    • → Creating simulation is easy

Conformance testing

  • Testing that the implementation conforms to the original requirements
    • Conforms to the standards and specifications
  • Important to guarantee that different implementations will work together.
    • Implementers may interpret standards differently and take own short cuts
    • Testing will point out the flaws
  • Interest in the external behaviour of the implementation
    • Check functional correctness of implementation under test without checking it inner works
    • Black box testing
  • Behaviour is specified with set of scenarios
    • Testing scenarios described in TTCN (Tree and Tabular Combined Notation)
    • TTCN testing suite provided by the protocol designer /standardisation body / testing facility.

Load testing

  • Goal is to test the performance of the implementation
  • Reliability under load
    • how overload protection mechanisms are working.
    • Requests coming from several independent sources
    • Requests from the sources interleaved
  • Sustainability of connection
    • How the implementation handles really slow and long lasting connections
  • load generators used to create simulated environment rather than using real independent sources
    • Programmable device that offers selection of predefined scenarious that can be run.
    • not just generate load, but also analyses the responses. (e.g. conformance in heavy load)

in-field testing

  • Testing of the system in work.
    • detect, locate and eliminate bugs that are found in real work
  • Log files of events happening in system
    • e.g. logs of messages coming that do not fit for current state.
    • Logs can be analysed remotely and bug fixes then created.
Last modified: 2013/07/01 14:42