Rating Questions

The following ratings a product could receive are described below:

Pass Rating

Meets our minimum requirements for privacy and security practices.

Applications and services that received a Pass rating have met a minimum criteria for transparency and qualitatively better practices in their policies. Before using an application or service in this rating, parents, teachers, schools, and districts are strongly advised to read the full privacy evaluation as a starting point for the process of vetting the application or service. In addition, a more detailed review should happen before any child or student data is shared with a service. In 2019, approximately 20% of applications and services are rated Pass, which is a 10% increase in the percentage of products with overall better rating question practices since 2018.

Warning Rating

Does not meet our recommendations for privacy and security practices.

Applications and services that received a Warning rating have risks narrowly focused around data use related to selling data, third-party marketing, creating profiles that are not associated with any educational purpose, and/or using data to target advertisements. We include data use from both the first party (i.e., the vendor that builds the service) and third parties (any company given access to data by the vendor). Using data to profile students for advertising purposes can potentially violate multiple state laws and in some cases federal law. An application or service can be given a Warning rating for either a lack of transparency around data use—which creates the potential for profiling and behavioral targeting—or for clearly stating the service uses data to target advertisements and/or create profiles. As with any application being considered for use within schools, school and/or district staff should review the privacy policies and terms of service to ensure that they meet the legal and practical requirements of their state laws and school policies. Unclear or qualitatively worse responses to the questions listed below trigger inclusion in the Warning rating:

1. Do the policies clearly indicate the version or effective date of the policies?

2. Do the policies clearly indicate whether or not a user's personal information is sold or rented to third parties?

3. Do the policies clearly indicate whether or not a user's personal information is shared with third parties for advertising or marketing purposes?

4. Do the policies clearly indicate whether or not behavioral or contextual advertising based on a user's personal information is displayed?

5. Do the policies clearly indicate whether or not third-party advertising services or tracking technologies collect any information from a user of the application or service?

6. Do the policies clearly indicate whether or not a user's personal information is used to track and target advertisements on other third-party websites or services?

7. Do the policies clearly indicate whether or not the vendor allows third parties to use a user's data to create a profile, engage in data enhancement or social advertising, or target advertising?

In 2019, approximately 60% of applications and services are rated Warning, which is a 20% decrease from 2018 in the percentage of products rated Warning. However, this decrease was due to a respective 10% increase in the number of applications and services rated Pass and Fail. On the bright side, a majority of applications and services (68%) disclosed that they do not rent, lease, trade, or sell data. However, a majority of applications and services are unclear or explicitly allow third-party marketing, behavioral advertising, and third-party tracking, track users across other websites, or allow the creation of data profiles. This use of educational data for noneducational purposes, even if legal, is contrary to user expectations about edtech.

Fail Rating

Does not have a privacy policy and should not be used

Applications and services that received a Fail rating have issues narrowly focused on whether a detailed privacy policy is available for evaluation prior to login or account creation in order to protect child and student data. Unclear or qualitatively worse responses to the question listed below trigger inclusion in the Fail rating:

1. Is a privacy policy available?

The criteria for Fail measure whether or not a vendor has done the bare minimum to provide users with a rudimentary understanding of how the vendor protects user privacy. The single criteria listed above is a basic privacy requirement for all products. Applications and services that do not meet this basic requirement can potentially run afoul of federal and state privacy laws. In 2019, approximately 20% are rated Fail, which is a negative trend since 2018 and a 10% increase in the percentage of products with overall worse rating question practices since 2018. This increase is likely the result of a more representative selection of applications and services evaluated in 2019. Among the applications or services we evaluated, only a small number did not have a privacy policy and/or terms of service available on their website at the time of our evaluation. Nonetheless, as with the Warning criteria described above, a Fail rating is not a sign that a vendor is necessarily doing anything illegal or unethical, but it could mean, based on how the application or service is used, that it could be violating either federal or state laws. It is a sign that, based on publicly available policies and observed security practices, their services do not provide adequate guarantees that information stored in their information systems will be protected.