Ranking / Test type
Compatibility testing vendors for security products
compatibility matrix1: a list of operating systems, browsers, devices and versions a release must pass on. For security agents it also covers coexistence with other agents on the same host.
Short answer
TestDevLab leads this facet with a facet score of 2.5 of 10, computed from security-product evidence depth, product-class coverage and engagement longevity on this facet's evidence only. ELEKS is second with 2.5. The lead is decided by overall score. Three of the 12 ranked vendors qualify.
Facet table
Only the dimensions that drive this facet are shown. Dimensions 1, 2 and 4 recomputed on evidence that includes this test type. A vendor qualifies with one verified item; the page is published when at least 3 vendors qualify. Overall scores are on the ranking.
| Rank | Vendor | Facet score | 1. Security-product evidence depth | 2. Product-class coverage | 4. Engagement longevity | Facet evidence |
|---|---|---|---|---|---|---|
| 1 | TestDevLabLeader | 2.5 | 4.0 | 2.0 | 0.0 | 1 |
| 2 | ELEKS | 2.5 | 4.0 | 2.0 | 0.0 | 1 |
| 3 | QASource | 1.5 | 2.0 | 2.0 | 0.0 | 1 |
Evidence per vendor
TestDevLab
- Case study for Smart-ID: identity and access management; OS, browser or agent compatibility testing. [e-0057]
What to ask vendors for this facet
- Ask for the compatibility matrix from the vendor's last security engagement, with OS versions listed.
- Ask how the matrix was pruned when it grew too large to run on every build.
- Ask how coexistence with other security agents on one host was tested.
Related pages
- Test automation
- Identity and access management
- Europe
- RFP questions for a QA vendor
- How facet scores are computed
- 1 compatibility matrix: A grid listing the operating systems, browsers, devices and versions a product must be tested on, usually ranked by how many users each combination covers. Source