Evaluation Strategy
MIDAS partners have defined the evaluation strategy as the way the results of the project will be assessed.
The evaluation strategy provides:
1. the main success factors of the project,
2. the main key performance indicators of the project,
3. a description of how the project will be evaluated as a whole, including the result of technical and pilots tasks,
4. a description of how each test method, component or approach at technical level will be evaluated at scientific level,
5. a description of how the adoption of MIDAS will be evaluated in the healthcare and logistics domains and,
6. a description of how the results will be evaluated from the standardization view.
We have identified a set of Sucess Factors:
- Quality
- Cost
- Effort
- Risk
- Usage-based testing is effective and efficient and can be applied without consulting
- Functional Testing Scheduling is effective and completely automatic.
- Binding Functional testing Scheduling and Automated Test Case Generation improve the automation workflow
- Automatic Test case generation and test execution is effective and efficient.
- Test methods can easily be developed without consulting and can be deployed on the MIDAS platform with minimal human intervention.
- The MIDAS platform usage can be transparently monitored and billed on a tenancy basis.
As well as a set of Key Performance Indicators:
- Number of defects found
- Difficulty to learn MIDAS DSL
- Effort for learning the MIDAS DSL
- Difficulty to create the test architecture
- Effort for creating the test architecture with the MIDAS DSL
- Difficulty of usage-based testing
- Effort for usage-based testing
- Efficiency of usage-based testing
- Difficulty of evidence-driven testing
- Effort for evidence-driven testing
- Efficiency of evidence-driven testing
- Difficulty of security testing
- Effort for security testing
- Efficiency of security testing
- Difficulty for integrating test methods in the MIDAS platform
- Effort for deploying the MIDAS platform on the Cloud
- Effectiveness of the monitoring and billing functionalities for the usage and the corresponding billing of the MIDAS services and underlying Cloud resources
- Effort for the interpretation of the test results
- End-user without a consultant
- MBT methodology adoption in practice
- Adoption by SMEs in practice