Evaluation Strategy

MIDAS partners have defined the evaluation strategy as the way the results of the project will be assessed.

The evaluation strategy provides:
1.    the main success factors of the project,
2.    the main key performance indicators of the project,
3.    a description of how the project will be evaluated as a whole, including the result of technical and pilots tasks,
4.    a description of how each test method, component or approach at technical level will be evaluated at scientific level,
5.    a description of how the adoption of MIDAS will be evaluated in the healthcare and logistics domains and,
6.    a description of how the results will be evaluated from the standardization view.

We have identified a set of Sucess Factors:

  • Quality
  • Cost
  • Effort
  • Risk
  • Usage-based testing is effective and efficient and can be applied without consulting
  • Functional Testing Scheduling is effective and completely automatic.
  • Binding Functional testing Scheduling and Automated Test Case Generation improve the automation workflow
  • Automatic Test case generation and test execution is effective and efficient.
  • Test methods can easily be developed without consulting and can be deployed on the MIDAS platform with minimal human intervention.
  • The MIDAS platform usage can be transparently monitored and billed on a tenancy basis.

As well as a set of Key Performance Indicators:

  • Number of defects found
  • Difficulty to learn MIDAS DSL
  • Effort for learning the MIDAS DSL
  • Difficulty to create the test architecture
  • Effort for creating the test architecture with the MIDAS DSL
  • Difficulty of usage-based testing
  • Effort for usage-based testing
  • Efficiency of usage-based testing
  • Difficulty of evidence-driven testing
  • Effort for evidence-driven testing
  • Efficiency of evidence-driven testing
  • Difficulty of security testing
  • Effort for security testing
  • Efficiency of security testing
  • Difficulty for integrating test methods in the MIDAS platform
  • Effort for deploying the MIDAS platform on the Cloud
  • Effectiveness of the monitoring and billing functionalities for the usage and the corresponding billing of the MIDAS services and underlying Cloud resources
  • Effort for the interpretation of the test results
  • End-user without a consultant
  • MBT methodology adoption in practice
  • Adoption by SMEs in practice