Common Sense Privacy Training: For Vendors

The Privacy Program

The Common Sense Privacy Program helps clarify privacy policies with its policy annotator so that teachers can make smart choices about the learning tools they use with students, and schools and districts can participate in evaluating the technology used in K–12 classrooms.

Training Guide

Privacy Evaluations

Our evaluation process for online applications and services attempts to address some of the common barriers to evaluating privacy practices. Privacy concerns and needs vary widely based on the app and the context where it is used. For example, it makes sense for a student information system to collect a home address; however, it wouldn't make sense for an online calculator to collect a home address. Because our evaluation process pairs a transparency evaluation with a qualitative evaluation, we can track what a policy covers alongside the strengths and weaknesses of how data is handled. Lastly, our summary evaluation allows us to highlight the implications of an app's privacy practices alongside the goals and contexts within which the app can be used.

For further information about the Privacy Program click here.

Policy Annotator

The policy annotator was developed to address common barriers to evaluating privacy policies. The easy-to-use tool makes evaluating privacy policies easier to complete while also identifying privacy concerns in context. The policy annotator pairs a transparency evaluation with a qualitative evaluation to highlight the implications of the application or service's privacy practices, providing an overall score for the product.

To launch the policy annotator, please visit https://policy-annotator.commonsense.org.

Login

If users already have an account, they can click Log in. If not, they can click Log in and then click the Create Account button.

After users have clicked Log in, they must enter the username and password used with their account registration to log in to the service. If users have forgotten their password, they can click Reset Password; the login information will be sent to the email address we have on file for that account. Otherwise, users can click the Create Account button, and an email containing login information will be sent to them. All user accounts will become active upon approval.

Evaluation Process

The evaluation process contains four steps:

Step 1: Overview

After logging in, select a product from the Product Families drop-down list to evaluate the policies of that application or service.

The selected Product Family drop-down list contains all the publicly available policies for each product that we have copied into our policy annotator for evaluation.

After a product is selected, the product's policies are displayed (in the gray box) along with information on the policies themselves, such as the readability statistics (on the right). There are several options for navigating these questions and learning more about a product's privacy practices.

Step 2: Triage

The next stage of the evaluation process is the observational stage that asks just the bare minimum number of questions to determine whether a privacy evaluation can proceed. These questions, which relate not to the policy text itself, but to the vendor's practices, are summarized in the following topics. To read the full questions without citations, visit here.

1. Assessment

• 0.1.1: Privacy Policy and Terms of Service are made available
• 0.1.2: App store policies are the same as policies available for its applications and services
• 0.1.3: Provides encryption for user information transmitted during login, account creation, and usage
• 0.1.4: Requires encryption for user information transmitted during login, account registration, and usage

2. Policy Available

• 0.2.1: Policies are labeled and easy to find
• 0.2.2: Policies are provided in accessible formats
• 0.2.3: Discloses no prohibition on automated copying of the policies
• 0.2.4: Policies are available on all product pages
• 0.2.5: Policies are available on a new account registration webpage

3. Account Type

• 0.3.1: Provides a free sample account with the application or service
• 0.3.2: Requires an access code or purchase of additional hardware or devices to use the application or service
• 0.3.3: Offers additional paid services or in-app purchases

4. Policy Errors

• 0.4.1: Policies are available in a clear and understandable manner

Step 3: Transparency

The next stage of the privacy evaluation process requires transparency about answers to expected questions from the text of the policies. Questions include the following elements:

• Transparency selection: Do the policies discuss the issues raised in the question?
• Qualitative selection: Does the vendor indicate whether or not it engages in the particular practice described?
• Notes: Is there anything noteworthy, exceptional, or egregious regarding the details of the question that should be noted?
• Policy references: Can text within the various policies be highlighted and associated with a particular question?

(Note that not all parts of the evaluation process may be available depending on the answers given to a transparency question.)

Transparency Questions

The following sections categorize questions that require answers related to the policy text itself.

1. Transparency

• 1.1: Policy Version (2 questions)
• 1.2: Policy Coverage (1 question)
• 1.3: Policy Changes (2 questions)
• 1.4: Policy Notice (3 questions)
• 1.5: Policy Contact (1 question)
• 1.6: Policy Principles (1 question)
• 1.7: Policy Language (1 question)
• 1.8: Intended Use (6 questions)

2. Focused Collection

• 2.1: Data Collection (9 questions)
• 2.2: Data Source (2 questions)
• 2.3: Data Exclusion (2 questions)
• 2.4: Data Limitation (1 question)

3. Data Sharing

• 3.1: Data Shared with Third Parties (2 questions)
• 3.2: Data Use by Third Parties (4 questions)
• 3.3: Data Not Shared with Third Parties (1 question)
• 3.4: Data Sold to Third Parties (1 question)
• 3.5: Third-Party Data Acquisition (1 question)
• 3.6: De-identified or Anonymized Data (2 questions)
• 3.7: Third-Party Links (1 question)
• 3.8: Third-Party Data Access (1 question)
• 3.9: Third-Party Data Collection (1 question)
• 3.10: Third-Party Data Misuse (1 question)
• 3.11: Third-Party Service Providers (2 questions)
• 3.12: Third-Party Affiliates (1 question)
• 3.13: Third-Party Policies (1 question)
• 3.14: Third-Party Data Combination (2 questions)
• 3.15: Third-Party Authentication (3 questions)
• 3.16: Third-Party Contractual Obligations (2 questions)

4. Respect for Context

• 4.1: Data Use (2 questions)
• 4.2: Data Classification (1 question)
• 4.3: User Classification (1 question)
• 4.4: Data Combination (1 question)
• 4.5: Data Notice (1 question)
• 4.6: Data Changes (1 question)
• 4.7: Enforcement Context (1 question)

5. Individual Control

• 5.1: User Content (2 questions)
• 5.2: User Consent (1 question)
• 5.3: Remedy Process (1 question)
• 5.4: Data Settings (1 question)
• 5.5: Data Disclosure (3 questions)
• 5.6: Data Rights (1 question)
• 5.7: Data License (3 questions)

6. Access and Accuracy

• 6.1: Data Access (3 questions)
• 6.2: Data Integrity (1 question)
• 6.3: Data Correction (3 questions)
• 6.4: Data Retention (2 questions)
• 6.5: Data Deletion (5 questions)
• 6.6: Data Portability (2 questions)

7. Data Transfer

• 7.1: Data Handling (2 questions)
• 7.2: Transfer Consent (1 question)
• 7.3: Transfer Request (1 question)
• 7.4: Onward Contractual Obligations (1 question)

8. Security

• 8.1: User Identity (1 question)
• 8.2: User Account (3 questions)
• 8.3: Third-Party Security (1 question)
• 8.4: Data Confidentiality (2 questions)
• 8.5: Data Transmission (1 question)
• 8.6: Data Storage (2 questions)
• 8.7: Data Breach (1 question)

9. Responsible Use

• 9.1: Social Interactions (3 questions)
• 9.2: Data Visibility (3 questions)
• 9.3: Report Content (2 questions)
• 9.4: Monitor and Review (7 questions)
• 9.5: Internet Safety (1 question)

10. Advertising

• 10.1: Vendor Communications (1 question)
• 10.2: Traditional Advertising (1 question)
• 10.3: Behavioral Advertising (1 question)
• 10.4: Ad Tracking (3 questions)
• 10.5: Filtered Advertising (2 questions)
• 10.6: Marketing Communications (2 questions)
• 10.7: Unsubscribe (2 questions)
• 10.8: Do Not Track (2 questions)

11. Compliance

• 11.1: Children Under 13 (7 questions)
• 11.2: Teens 13-18 (1 question)
• 11.3: Students in K–12 (5 questions)
• 11.4: Parental Consent (9 questions)
• 11.5: Legal Requirements (4 questions)
• 11.6: Certification (1 question)
• 11.7: International Laws (2 questions)
• 11.8: Compliance Assessment (1 question)

For more information about the questions, please see the following resources:

• To read the full question set, with no additional citations, visit here.
• For people who want additional background, we have also included the Fair Information Practice Principles, with links to additional reading.

Step 4: Summary

The next step requires users to summarize and describe the various policy details for four concerns -- (1) Safety, (2) Privacy, (3) Security, and (4) Compliance -- and a general overview of the complete application or service based on the answers provided in the transparency questions.

The Safety concern categorizes all the safety-related transparency questions with their relative transparency and qualitative answers so that a user can write a summary that is reflective of the better or worse safety practices of that product.

Likewise, the Privacy concern categorizes all the privacy-related transparency questions with their relative transparency and qualitative answers so that a user can write a summary that is reflective of the better or worse privacy practices of that product.

The Security concern categorizes all the security-related transparency questions with their relative transparency and qualitative answers so that a user can write a summary that is reflective of the better or worse security practices of that product.

The Compliance concern categorizes all the compliance-related transparency questions with their relative transparency and qualitative answers so that a user can write a summary that is reflective of the better or worse compliance practices of that product. Lastly, the general overview summary section is where the user summarizes the key points from all four concerns into a general overview of the complete application or service.

Results

The final step of the privacy evaluation, called Results, provides several different types of information about the product's policies that are based on the answers given to the transparency questions. The first Details tab displays a short-form privacy policy that is automatically generated by the evaluation process.

The second Scoring tab displays the transparency, qualitative, and overall score for the product based on the answers given to the transparency questions.

The Scoring< tab also displays both graphical and tabular results for Transparency, Quality, and Overall Score from each of the four concerns and various statutes derived from answers to the questions used in the evaluation process. 

In addition, for even more detailed results, a drop-down menu is available to view both the transparency and qualitative impact for each question by scoring or notes.

The results from the evaluation process provide an overall score based on transparency and quality and a tier for each product. The following three tiers are available: (1) Use Responsibly, which indicates that the application or service meets our minimum criteria, but more research should be completed prior to use; (2) Use with Caution, which indicates that the application or service does not clearly define the safeguards to protect child or student information; and (3) Not Recommended, which indicates that the application or service does not support encryption or lacks a detailed privacy policy.

Interpretation of each tier is highly contextually dependent and varies based on use, audience, and local concerns. For more information about the tiers and how they may apply to your situation, please read our 2021 State of Kids' Privacy Report.

Once a privacy evaluation is complete, our team publishes the product's evaluation on our website at https://privacy.commonsense.org/.

For example, here is a sample privacy evaluation that displays the evaluation date, the product's name, icon, key points, tier, and overall score.

Each privacy evaluation displays a general overview of the product and includes links to read the privacy policies and access the application or service.

In addition, each evaluation displays key points of the product based on question responses for Intended Use.

Each evaluation displays key points of the product based on question responses for Data Collection.

Lastly, each evaluation displays key points of the product based on question responses for Parental Consent.

Additionally, more details about a product's key points are available in a short-form privacy policy shown earlier in the Details tab on the Results stage of the policy annotator.

An evaluation score and curated summary written by the user are included for each of the four categories (Safety, Privacy, Security, and Compliance). Following is an overall score and summary for the concern of Safety.

This is an overall score and summary for the concern of Privacy.

This is an overall score and summary for the concern of Security.

This is an overall score and summary for the concern of Compliance.

For further information about the policy annotator or to search evaluations, please join us at https://privacy.commonsense.org.