Tool Review Criteria
During the process of preparing for this presentation, I did a survey of a number of products to help implement a secret storage strategy. I did not approach these from a security analysts perspective but more from the devops side.
The criteria I used are as follows:
- Ease of setup - is the product straight-forward to deploy and do initial configuration?
- Easy of use - once set up, is the product intuitive? do you need to spend a lot of time going to the docs to figure out how to use it?
- Cloud readiness - does the product qualify as "cloud native": easily deployable, stateless, loosely couple dependencies, etc...?
- Datacenter readiness - will the product function easily in a more traditional corporate datacenter?
- Automation / pipeline readiness - does the product integrate well with continuous integration and continuous delivery environments or is it likely to be come the manual bottleneck?
- Product maturity - has the product been around and gone through some iterations so the wrinkles have been ironed out?
- Developer friendliness - does the APIs and/or command-line parameters are straightforward and the product has a good story for integrating into existing code?
- Documentation - RTFM
- Stability - does it crash or corrupt data?
- Auditability - does it provide decent auditing for both change and access or are you forced to build a layer on top?
Admittedly, some criteria proved to be more meaningful than others. For example, the "data center readiness" attribute ended up pretty flimsy. Any product I looked at that wasn't "data center ready" wasn't "cloud ready" either, due to product maturity.
Should I start reviewing any SAAS-type products, I think that will change so I'll leave the criteria up there.
Written on September 21, 2015