*Privacy (and context) not included

The IoD supplemental assessments are available here

The Mozilla foundation recently published a Valentines version of their "*privacy not included" lists. These lists detail the common gifts around a holiday season and document some of the good, the bad and the ugly of the security and privacy practices of those devices.  The site is far from exhaustive, but serves to give consumers an idea of the sort of threats to their security and privacy that exist in modern "smart" products and the questions they need to ask to protect themselves if they are concerned.

Being valentines day related, a number of IoD devices were listed along side smart beds and other tangentially related accessories.  Normally I'm happy to see attention given to the issues around privacy and security of IoD, however I was not too pleased with some aspects of their assessments and criteria.  After being involved in researching these devices and interacting with many of the vendors, their assessments baffled me and I felt them to be possibly misleading to consumers.  I said as much on Twitter, asking Mozilla if they would answer questions about their list and the criteria used:

Within a few hours, the researchers behind the list reached out to me and were more than happy to have a video call with me to discuss my concerns. I was impressed with this accountability and response. They already knew of the project and they were happy to hear the opinions of someone far more involved than they were.  I have a great deal of respect for them for stepping up.

They revealed that their criteria and methodology was based on what was available from vendors websites, privacy policies, and communications with the vendors and their answering questions from the researchers.  As they did not have the devices themselves, they could not assess these claims directly. As a result, they had limited details to work with in their assessment and had to rely on vendors information and trust that it was accurate.  This is why the Lovense devices were deemed to meet minimum standards but the We-Vibe device didn't. This is despite the fact that I hold the We-Vibe We-Connect app as the high water mark for app security for the industry (Lovense is very good as well, don't get me wrong); Lovense replied to the researchers and had a documented vulnerability disclosure process, We-vibe did not reply and has no documented vulnerability disclosure process. So being forthcoming was a benefit for Lovense.  Something for We-Vibe to consider.

Additionally, they were applying their criteria for evaluating IoT security and privacy and their minimum IoT security standards and applying it to these adult devices.  The problem is that this is "one size fits all" fails to take into account some of the nuances of these devices connections and the unique nature of IoD devices in general and the data associated with them.  I found that each section of their methodology had it's own issues in relations to IoD, and even some subsections therein had problems.

My comments on each of their criteria is broken down here:

• "Can it spy on me": This criteria is based on if a device has or uses "a camera, microphone, and location tracking".  While they admit that "just because something can spy on you, doesn’t me it will", their assessment pages for these devices doesn't make that very clear.  The phrase "Can it spy on me" gives the hint that maybe these devices are spying if any of them use these functions.  If the devices need these functions to do what they were designed to do (i.e. video/audio calling), then it's more a matter of them being used responsibly and properly in a secure fashion by the vendor. That's the part that matters for the consumer and their privacy.
• "Location Tracking": Most of the IoD devices they list require location permission in their android app. The phrasing seems to indicate that the app is tracking the users movements in some nefarious fashion by asking for this.  This is far from the situation. Since Android 6.0, Bluetooth connections have required the location permission in order to scan the local area for what devices are nearby. If you read the Android developer documents regarding Bluetooth Permissions and Bluetooth Low Energy, you would see that Bluetooth use by an app requires
  the ACCESS_COARSE_LOCATION permission at a minimum in order to function. The reasons are kind of complex and will be documented further in a later article, but it's a requirement of Android to have the permission for Bluetooth to function, not the apps function or vendor desire. My analysis shows that most of the IoD devices that need the location permission, never call any functions to query the users location. So the app, while it could, never knows where the users location is. Those that do call that function appear to do so as it's part of a feature of the app and it's operation, not anything nefarious.
• "What does it know about me?": This issue looks at the use of encryption, data sharing and understandability of the privacy policy. This is actually 3 very distinct criteria in one and should be broken out.  Encryption should relate to SSL/TLS usage (is all communication sent over secure https connections) and the strength of the implementation as determined by tools like SSLlabs.com. The privacy policy readability is a very subjective thing. While you need to have the lawyers happy with cryptic legalese, most consumers can't understand it. I always encourage vendors to have a "plain language" section that, in good faith, explains what all that jargon actually means to them in real world terms. Data sharing should be clarified by Mozilla. Sharing with third parties can have many meanings and impacts on privacy, some good and some bad. Do normal things like crash analytics using a third party service like Instabug count? Is that different than using aggregate (and properly anonymity of scrubbed) data to make products better? Their criteria is not clear as to what counts as "unexpected".  I did approve of their inquiry of if there was a way to delete your user account and all information. This is an idea I will be advising vendors on providing in the future.
• "Can I control it?": This item deals with passwords, automatic updates, data handling and parental controls. Passwords are problematic because they take into account Bluetooth pairing passwords in addition to any login passwords. Last I checked, none of the devices had a keyboard to change the password from defaults (if PIN needed at all).  I agree that use of strong passwords can be improved by vendors, however this is a contentious issue that needs to be weighed against support costs and usability. Parental controls often refers to limiting access to adult material. The Mozilla definition fails to take into account that many apps can require a PIN to prevent anyone else from opening the app. Does that count as a parental control? It seems to be an unnecessary inclusion and should have been focused on "privacy locks" to prevent someone snooping in the app to see past conversations and connections for instance.
• "Company shows it cares about its customers?": This section revolves around companies public commitments to users data privacy and security. This was established by the presence of a vulnerability reporting program or bug bounty and how easy the company was to contact with concerns. The IoD project has helped many vendors establish such programs and the fact they are being used as a criteria here shows how necessary they are to have.
• "What could happen if something goes wrong?": This section is problematic for many reasons. Primarily, it's "worst case scenario" which often contains links to scaremongering articles about past violations. It's because of these past issues that these worst case scenarios are unlikely to happen. The industry is learning from mistakes and being careful not to repeat them. That said, there is always potential for things to happen. Problem is that the way they are presented, it's not clear that it's theoretical and seems like it's whats happening with these devices right now and not that the issues have been fixed or mitigated. Usage of terms like "potentially" and "could" may not catch the attention of readers and lead to wrong conclusions.

So what is the real deal with those IoD devices? What is the real status?

I generally prefer to stay objective and not compare or rank products or vendors, however in this case, I feel I can provide consumers with more context and nuanced information than the Mozilla list does, at least in some of the more technical matters.

Utilising the spirit of the criteria that Mozilla laid out, along with my own critiques of them and supplemental details, I've assessed the devices they they had in their list and provided further insight that hopefully consumers will use to guide their choices.

Read the IoD Supplemental Assessments

(Please note that these are a snapshot in time and these assessments may change tomorrow with an update. Versions and release dates have been noted where applicable.)