Virtual System Testing

• Extract API Traces Steps from Sh-archives in stage -Directory
• Launch test-runner.rb for API Trace Steps in virtual-system-test-input -directory
• Version information

List self extracting archives in ../stage directory

ls ../stage/*.sh

and observe self extracting archive files with names corresponding Possibility operators used in Formal Modeling

../stage/arch_isCompanyRegisteredFails.sh
../stage/arch_isCompanyRegistered.sh

Extract test cases for interface registerCompany(post) into directory virtual-system-test-input with the following command:

# Cleanup previous tests
rm virtual-system-test-input/*
# first extract
../stage/arch_isCompanyRegisteredFails.sh \
        --dir virtual-system-test-input \
          interface '/registerCompany\\(post\\)'  2>&1
# second extract
../stage/arch_isCompanyRegistered.sh \
       --dir virtual-system-test-input \
        interface '/registerCompany\\(post\\)' 2>&1

After these commands the content of virtual-system-test-input contains files:

001-01-inp-TimeoffRegisterCompany.d6bb3494bdda70ae1ad54ebca1fc01ae890db84a
001-02-api-TimeoffRegisterCompany.d6bb3494bdda70ae1ad54ebca1fc01ae890db84a
001-03-ret-TimeoffRegisterCompany.d6bb3494bdda70ae1ad54ebca1fc01ae890db84a
001-04-out-TimeoffRegisterCompany.d6bb3494bdda70ae1ad54ebca1fc01ae890db84a
001-01-inp-TimeoffRegisterCompany.426759921aa71f5075e6fa6e33e828cfef8762ab
001-02-api-TimeoffRegisterCompany.426759921aa71f5075e6fa6e33e828cfef8762ab
001-03-ret-TimeoffRegisterCompany.426759921aa71f5075e6fa6e33e828cfef8762ab
001-04-out-TimeoffRegisterCompany.426759921aa71f5075e6fa6e33e828cfef8762ab

with files name using format 'STEP-TYPE-INTERFACE.SHA1', where

• STEP: is three digits for API trace step
• TYPE: is Api trace step
  • 01-inp formal state before executing API trace STEP
  • 02-api formal parameter values passed to INTERFACE
  • 03-ret formal parameter return values for calling INTERFACE
  • 04-out formal state after executing API trace STEP
• INTERFACE: name of interface called in API trace STEP
• SHA1: suffix is SHA1 of the archive from, which the file was extracted identifying virtual system test, of which API trace step is part of

To launch RSpec test runner to execute API trace step test cases in directory virtual-system-test-input issue the command

bundle exec rspec spec/virtual-system-test/test-runner.rb 2>&1

and observe the output:

------------------------------------------------------------------
Step 001 - interface TimeoffRegisterCompany SHA1=d6bb3494bdda70ae1ad54ebca1fc01ae890db84a
    Check beforeState
    Start application
.   Make API call
    Check API response
        Check API response - expect success
.   Stop application
    Check afterState
------------------------------------------------------------------
Step 001 - interface TimeoffRegisterCompany SHA1=426759921aa71f5075e6fa6e33e828cfef8762ab
    Check beforeState
    Start application
.   Make API call
    Check API response
        Check API response - expect failure
.   Stop application
    Check afterState


Finished in 13.14 seconds (files took 0.26002 seconds to load)
4 examples, 0 failures

Test runner groups files in directory virtual-system-test-input by SHA1, INTERFACE and STEP and

• initializes and checks system state using API trace step before state
• calls API with parameter value corresponding the INTERFACE of the API trace step
• checks that API response corresponds to INTERFACE return values of the API trace step
• checks that system state, after the API call, matches after state of the API trace step

• Ruby language environment : ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
• Tester environment
  • rspec GEM = * rspec (3.7.0)=
  • sqlite3-ruby GEM = * sqlite3-ruby (1.3.3)=
  • capybara GEM = * capybara (2.17.0)=
  • selenium-webdriver GEM = * selenium-webdriver (3.8.0)=
• Sbuilder Tools Set
  • tla-trace-arch GEM create self extracting archive = * tla-trace-arch (0.1.1)=