Evaluation Framework

The privacy evaluation process combines transparency and qualitative questions in a single streamlined framework. This requires organizing all the evaluation questions into categories and sections derived from the Fair Information Practice Principles (FIPPs) that underlie international privacy laws and regulations. In addition, the questions and the categories that organize them all are mapped to a range of statutory, regulatory, and technical resources that provide background information on why each question is relevant to the privacy evaluation process.

The evaluation process incorporates our policy annotator tool that allows a reviewer to annotate and associate a product's specific policy text with each evaluation question. For example, the following evaluation question requires a reviewer to read the policies of the application or service and determine whether or not it transparently discloses the issue raised in the question by providing a yes or no response:

Question: Do the policies clearly indicate whether or not the vendor collects personally identifiable information (PII)?

"Image of policy annotator"

If the reviewer responds yes to this question, that means the application or service discloses whether or not it collects personally identifiable information, and the overall transparency score is increased. Given a yes transparent response to this question, the reviewer is then asked a follow-up question of whether or not the application or service discloses it engages in the particular practice described. A yes or no response that personally identifiable information is or is not collected will increase or decrease the overall quality score based on whether the practices described are considered qualitatively better or worse for the purposes of our evaluation process.

The evaluation tiers and evaluation scores describe in more detail how responses to questions affect the tier and overall roll-up score for an application or service.