Generic Sensor API

1. Introduction

Increasingly, sensor data is used in application development to enable new use cases such as geolocation, counting steps or head-tracking. This is especially true on mobile devices where new sensors are added regularly.

Exposing sensor data to the Web has so far been both slow-paced and ad-hoc. Few sensors are already exposed to the Web. When they are, it is often in ways that limit their possible use cases (for example by exposing abstractions that are too high-level and which don’t perform well enough). APIs also vary greatly from one sensor to the next which increases the cognitive burden of Web application developers and slows development.

The goal of the Generic Sensor API is to promote consistency across sensor APIs, enable advanced use cases thanks to performant low-level APIs, and increase the pace at which new sensors can be exposed to the Web by simplifying the specification and implementation processes.

This lacks an informative section with examples for developers. Should contain different use of the API, including using it in conjunction with requestAnimationFrame.

2. Scope

This section is non-normative.

The scope of this specification is currently limited to specifying primitives which enable expose data from local sensors.

Exposing remote sensors or sensors found on personal area networks (e.g. Bluetooth) is out of scope. As work in these areas mature, it is possible that common, lower-level primitives be found, in which case this specification will be updated accordingly. This should have little to no effects on implementations, however.

This specification also does not currently expose a sensor discovery API. This is because the limited number of sensors currently available to user agents does not warrant such an API. Using feature detection, such as described in §4 A note on Feature Detection of Hardware Features, is good enough for now. A subsequent version of this specification might specify such an API, and the current API has been designed with this in mind.

3. Background

This section is non-normative.

This section is ill-named. It principally covers default sensors and explains the reasoning behind them. It should be renamed accordingly and moved, either to another section of the spec or to an external explainer document.

The Generic Sensor API is designed to make the most common use cases straightforward while still enabling more complex use cases.

Most devices deployed today do not carry more than one sensor of each sensor types. This shouldn’t come as a surprise since use cases for more than a sensor of a given type are rare and generally limited to specific sensor types, such as proximity sensors.

The API therefore makes it easy to interact with the device’s default (and often unique) sensor for each type simply by instantiating the corresponding Sensor subclass.

Indeed, without specific information identifying a particular sensor of a given type, the default sensor is chosen.

let sensor = new GeolocationSensor({ accuracy: "high" });

sensor.onchange = function(event) {
    var coords = [sensor.latitude, sensor.longitude];
    updateMap(null, coords, sensor.accuracy);
};

sensor.onerror = function(error) {
    updateMap(error);
};
sensor.start();

Note: extension to this specification may choose not to define a default sensor when doing so wouldn’t make sense. For example, it might be difficult to agree on an obvious default sensor for proximity sensors.

In cases where multiple sensors of the same type may coexist on the same device, specification extension will have to define ways to uniquely identify each one.

var sensor = new DirectTirePressureSensor({ position: "rear", side: "left" });
sensor.onchange = _ => console.log(sensor.pressure);
sensor.start();

4. A note on Feature Detection of Hardware Features

This section is non-normative.

Feature detection is an established Web development best practice. Resources on the topic are plentiful on and offline and the purpose of this section is not to discuss it further, but rather to put it in the context of detecting hardware-dependent features.

Consider the below feature detection examples:

if (typeof Gyroscope === "function") {
    // run in circles...
}

if ("ProximitySensor" in window) {
    // watch out!
}

if (window.AmbientLightSensor) {
    // go dark...
}

// etc.

All of these tell you something about the presence and possible characteristics of an API. They do not tell you anything, however, about whether that API is actually connected to a real hardware sensor, whether that sensor works, if its still connected, or even whether the user is going to allow you to access it. Note you can check the latter using the Permissions API [PERMISSIONS].

In an ideal world, information about the underlying status would be available upfront. The problem with this is twofold. First, getting this information out of the hardware is costly, in both performance and battery time, and would sit in the critical path. Secondly, the status of the underlying hardware can evolve over time. The user can revoke permission, the connection to the sensor be severed, the operating system may decide to limit sensor usage below a certain battery threshold, etc.

Therefore, an effective strategy is to combine feature detection, which checks whether an API for the sought-after sensor actually exists, and defensive programming which includes:

1. checking for error thrown when instantiating a Sensor object,

2. listening to errors emitted by it,

3. handling all of the above graciously so that the user’s experience is enhanced by the possible usage of a sensor, not degraded by its absence.

try { // No need to feature detect thanks to try..catch block.
    var sensor = new GeolocationSensor();
    sensor.start();
    sensor.onerror = error => gracefullyDegrade(error);
    sensor.onchange = _ => updatePosition(sensor.latitude, sensor.longitude);
} catch(error) {
    gracefullyDegrade(error);
}

5. Security and privacy considerations

Privacy risks can arise when sensors are used with each other, in combination with other functionality, or when used over time, specifically with the risk of correlation of data and user identification through fingerprinting. Web application developers using these JavaScript APIs should consider how this information might be correlated with other information and the privacy risks that might be created. The potential risks of collection of such data over a longer period of time should also be considered.

Variations in sensor readings as well as event firing rates offer the possibility of fingerprinting to identify users. User agents may reduce the risk by limiting event rates available to web application developers.

Note: do we really want this mitigation strategy?

If the same JavaScript code using the API can be used simultaneously in different window contexts on the same device it may be possible for that code to correlate the user across those two contexts, creating unanticipated tracking mechanisms.

User agents should consider providing the user an indication of when the sensor is used and allowing the user to disable it.

Web application developers that use sensors should perform a privacy impact assessment of their application taking all aspects of their application into consideration.

Ability to detect a full working set of sensors on a device can form an identifier and could be used for fingerprinting.

A combination of selected sensors can potentially be used to form an out of band communication channel between devices.

Sensors can potentially be used in cross-device linking and tracking of a user.

5.1. Browsing Context

Sensor readings must only be available in the top-level browsing context to avoid the privacy risk of sharing the information defined in this specification (and specifications extending it) with contexts unfamiliar to the user. For example, a mobile device will only fire the event on the active tab, and not on the background tabs or within iframes.

Note: Feature Policy should allow securely lifting those restrictions once it matures.

5.2. Secure Context

Sensor readings are explicitly flagged by the Secure Contexts specification [POWERFUL-FEATURES] as a high-value target for network attackers. Thus all interfaces defined by this specification or extension specifications must only be available within a secure context.

5.3. Obtaining Explicit User Permission

Permission to access sensor readings must be obtained through the Permissions API [PERMISSIONS].

6. Concepts

6.1. Sensors

A sensor measures different physical quantities and provide corresponding raw sensor readings which are a source of information about the user and their environment.

Each reading is composed of the values of the different physical quantities measured by the sensor at time t_n.

Known, predictable discrepancies between raw sensor readings and the corresponding physical quantities being measured are corrected through calibration.

Known but unpredictable discrepancies need to be addressed dynamically through a process called sensor fusion.

Calibrated raw sensor readings are referred to as sensor readings, whether or not they have undergone sensor fusion.

6.2. Sensor Types

Different sensor types measure different physical quantities such as temperature, air pressure, heart-rate, or luminosity.

For the purpose of this specification we distinguish between high-level and low-level sensor types.

Sensor types which are characterized by their implementation are referred to as low-level sensors. For example a Gyroscope is a low-level sensor type.

Sensors named after their readings, regardless of the implementation, are said to be high-level sensors. For instance, geolocation sensors provide information about the user’s location, but the precise means by which this data is obtained is purposefully left opaque (it could come from a GPS chip, network cell triangulation, wifi networks, etc. or any combination of the above) and depends on various, implementation-specific heuristics. High-level sensors are generally the fruits of applying algorithms to low-level sensors—for example, a pedometer can be built using only the output of a gyroscope—or of sensor fusion.

That said, the distinction between high-level and low-level sensor types is somewhat arbitrary and the line between the two is often blurred. For instance, a barometer, which measures air pressure, would be considered low-level for most common purposes, even though it is the product of the sensor fusion of resistive piezo-electric pressure and temperature sensors. Exposing the sensors that compose it would serve no practical purpose; who cares about the temperature of a piezo-electric sensor? A pressure-altimeter would probably fall in the same category, while a nondescript altimeter—which could get its data from either a barometer or a GPS signal—would clearly be categorized as a high-level sensor type.

Because the distinction is somewhat blurry, extensions to this specification (see §10 Extensibility) are encouraged to provide domain-specific definitions of high-level and low-level sensors for the given sensor types they are targeting.

Sensor readings from different sensor types can be combined together through a process called sensor fusion. This process provides higher-level or more accurate data (often at the cost of increased latency). For example, the readings of a three-axis magnetometer needs to be combined with the readings of an accelerometer to provide a correct bearing.

Smart sensors and sensor hubs have built-in compute resources which allow them to carry out calibration and sensor fusion at the hardware level, freeing up CPU resources and lowering battery consumption in the process.

But sensor fusion can also be carried out in software. This is particularly useful when performance requirements can only be met by relying on application-specific data. For example, head tracking for virtual or augmented reality applications requires extremely low latency to avoid causing motion sickness. That low-latency is best provided by using the raw output of a gyroscope and waiting for quick rotational movements of the head to compensate for drift.

Note: sensors created through sensor fusion are sometimes called virtual or synthetic sensors. However, the specification doesn’t make any practical differences between them, preferring instead to differentiate sensors as to whether they describe the kind of readings produced--these are high-level sensors—or how the sensor is implemented (low-level sensors).

6.3. Reporting Modes

This feature is at risk. It is not clear whether there is value in splitting up sensor types between those that fire events at regular intervals and those which don’t.

Sensors have different reporting modes. When sensor readings are reported at regular intervals, at an adjustable frequency measured in hertz (Hz), the reporting mode is said to be periodic. On sensor types with support for periodic reporting mode, periodic reporting mode is triggered by requesting a specific frequency.

Sensor types which do not support periodic reporting mode are said to operate in an implementation specific way. When the reporting mode is implementation specific, sensor readings may be provided at regular intervals, irregularly, or only when a reading change is observed. This allows user agents more latitude to carry out power- or CPU-saving strategies, and support multiple hardware configurations. Periodic reporting mode, on the other hand, allows a much more fine-grained approach and is essential for use cases with, for example, low latency requirements.

Sensors which support periodic reporting mode fallback to implementation specific reporting mode when no requirements are made as to what frequency they should operate at.

Note: reporting mode is distinct from, but related to, sensor readings acquisition. If sensors are polled at regular interval, as is generally the case, reporting mode can be either periodic or implementation specific. However, when the underlying implementation itself only provides sensor readings when it measures change, perhaps because is is relying on smart sensors or a sensor hubs, the reporting mode cannot be periodic, as that would require data inference.

This lacks a description of the different data acquisition modes, notably polling vs. on change, both at the platform and HW layer.

It would be useful to describe the process of sensor polling and how increased sensor polling frequency decreases latency.

A definition of sensor accuracy and how it affects threshold, and thus "on change" sensors would be useful.

7. Model

A diagram would really help here.

7.1. Sensor Type

A sensor type has an associated interface whose inherited interfaces contains Sensor.

A sensor type has one or many associated sensors.

If a sensor type has more than one sensor, it must have a set of associated identifying parameters to select the right sensor to associate to each new Sensor objects.

A sensor type may have a default sensor.

7.2. Sensor

A sensor has an associated set of activated Sensor objects. This set is initially empty.

A sensor has an associated latest reading map which holds the latest available sensor readings.

does the latest reading map need to be tied to an origin?

The latest reading map contains an entry whose key is "timestamp" and whose value is a high resolution timestamp of the time at which the latest reading was obtained expressed in milliseconds that passed since the time origin. latest reading["timestamp"] is initially set to null, unless the latest reading map caches a previous reading.

The other entries of the latest reading map hold the values of the different quantities measured by the sensor. The keys of these entries must match the attribute names defined by the sensor type's associated interface, so that the getter of the foo attribute can simply return latest reading["foo"].

The [map/value] of all latest reading entries is initially set to null.

A sensor supports periodic reporting mode if its associated sensor type does.

A sensor has an associated reporting flag which is initially unset.

A sensor has an associated periodic reporting mode flag which is initially unset.

A sensor has an associated current polling frequency which is initially null.

A sensor has an associated abstract operation to retrieve its permission which takes a Sensor object as input and returns a permission and, eventually, its associated PermissionDescriptor.

8. API

8.1. The Sensor Interface

[SecureContext]
interface Sensor : EventTarget {
  readonly attribute SensorState state;
  readonly attribute DOMHighResTimeStamp timestamp;
  void start();
  void stop();
  attribute EventHandler onchange;
  attribute EventHandler onactivate;
  attribute EventHandler onerror;
};

dictionary SensorOptions {
  double? frequency;
};

enum SensorState {
  "unconnected",
  "activating",
  "activated",
  "idle",
  "errored"
};

A Sensor object has an associated sensor.

Each Sensor object has a task source called a sensor task source, initially empty. A sensor task source can be enabled or disabled, and is initially enabled. When enabled, the event loop must use it as one of its task sources. The task source for the tasks mentioned in this specification is the sensor task source.

When the visibility state of the Document in the top-level browsing context changes, let current_visibility_state be the result of running the steps to determine the visibility state of the Document. If current_visibility_state is "visible", enable the sensor task source, otherwise, disable it.

Note: user agents are encouraged to stop sensor polling when sensor task sources are disabled in order to save battery.

8.1.1. Sensor internal slots

Instances of Sensor are created with the internal slots described in the following table:

8.1.2. Sensor.state

The getter of the state attribute returns this.[[state]].

8.1.3. Sensor.timestamp

The getter of the timestamp attribute returns latest reading["timestamp"].

8.1.4. Sensor.start()

8.1.5. Sensor.stop()

8.1.6. Sensor.onchange

onchange is an EventHandler which is called whenever a new reading is available.

Should this be renamed onreading? Should we instead add an ondata EventHandler for continuous data and use onchange when the data changed?

8.1.7. Sensor.onactivate

onactivate is an EventHandler which is called when this.[[state]] transitions from "activating" to "activated".

8.1.8. Sensor.onerror

onerror is an EventHandler which is called whenever an exception cannot be handled synchronously.

8.1.9. Event handlers

The following are the event handlers (and their corresponding event handler event types) that must be supported as attributes by the objects implementing the Sensor interface:

8.2. The SensorErrorEvent Interface

[SecureContext, Constructor(DOMString type, SensorErrorEventInit errorEventInitDict)]
interface SensorErrorEvent : Event {
  readonly attribute Error error;
};

dictionary SensorErrorEventInit : EventInit {
  required Error error;
};

8.2.1. SensorErrorEvent.error

Gets the Error object passed to SensorErrorEventInit.

9. Abstract Operations

9.1. Construct Sensor Object

9.2. Connect to Sensor

9.3. Register a Sensor Object

9.4. Unregister a Sensor

9.5. Set Sensor Settings

9.6. Observe a Sensor

9.7. Is Current Reporting Mode Periodic

9.8. Find the polling frequency of a Sensor

9.9. Update latest reading

9.10. Update Observers

9.11. Handle Errors

9.12. Request Sensor Access

10. Extensibility

This section is non-normative.

Its purpose is to describe how this specification can be extended to specify APIs for different sensor types.

Extension specifications are encouraged to focus on a single sensor type, exposing both high and low level as appropriate.

10.1. Security

All interfaces defined by extension specifications should only be available within a secure context.

10.2. Naming

Sensor interfaces for low-level sensors should be named after their associated sensor. So for example, the interface associated with a gyroscope should be simply named Gyroscope. Sensor interfaces for high-level sensors should be named by combining the physical quantity the sensor measures with the "Sensor" suffix. For example, a sensor measuring the distance at which an object is from it may see its associated interface called ProximitySensor.

Attributes of the Sensor subclass that hold sensor readings values should be named after the full name of these values. For example, the Thermometer interface should hold the sensor reading's value in a temperature attribute (and not a value or temp attribute). A good starting point for naming are the Quantities, Units, Dimensions and Data Types Ontologies [QUDT].

10.3. Unit

Extension specification must specify the unit of sensor readings.

As per the Technical Architecture Group’s (TAG) API Design Principles [API-DESIGN-PRINCIPLES], all time measurement should be in milliseconds. All other units should be specified using, in order of preference, and with the exception of temperature (for which Celsius should be favored over Kelvin), the International System of Units (SI), SI derived units, and Non-SI units accepted for use with the SI, as described in the SI Brochure [SI].

10.4. Exposing High-Level vs. Low-Level Sensors

So far, specifications exposing sensors to the Web platform have focused on high-level sensors APIs. [GEOLOCATION-API] [ORIENTATION-EVENT]

This was a reasonable approach for a number of reasons. Indeed, high-level sensors:

• convey developer intent clearly,

• do not require intimate knowledge of how the underlying hardware sensors functions,

• are easy to use,

• may enable the User Agent to make significant performance and battery life improvements,

• help avoid certain privacy and security issues by decreasing the amount and type of information exposed.

However, an increasing number of use cases such as virtual and augmented reality require low-level access to sensors, most notably for performance reasons.

Providing low-level access enables Web application developers to leverage domain-specific constraints and design more performant systems.

Following the precepts of the Extensible Web Manifesto [EXTENNNNSIBLE], extension specifications should focus primarily on exposing low-level sensor APIs, but should also expose high-level APIs when they are clear benefits in doing so.

10.5. When is Enabling Multiple Sensors of the Same Type Not the Right Choice?

TODO: provide guidance on when to:

• allow multiple sensors of the same type to be instantiated,

• create different interfaces that inherit from Sensor,

• add constructor parameters to tweak sensors settings (e.g. setting required accuracy).

10.6. Definition Requirements

The following definitions must be specified for each sensor type in extension specifications:

• An interface whose inherited interfaces contains Sensor. This interface must be constructible. Its [Constructor] must take, as argument, an optional dictionary whose inherited dictionaries contains SensorOptions. Its attributes which expose sensor readings are readonly and have getters must return the if disturbed, and null otherwise.

• A abstract operation to retrieve sensor permission which takes an object implementing the sensor type's associated interface as input and returns a permission and, eventually, its associated PermissionDescriptor.

An extension specification may specify the following definitions for each sensor types:

• A dictionary whose inherited dictionaries contains SensorOptions.

• A default sensor. Generally, devices are equipped with a single sensor of each type, so defining a default sensor should be straightforward. For sensor types where multiple sensors are common, extension specifications may choose not to define a default sensor, especially when doing so would not make sense.

• A set of identifying parameters. TODO: replace that by an abstract operation.

10.7. Extending the Permission API

Provide guidance on how to extend the Permission API [PERMISSIONS] for each sensor types.

10.8. Example WebIDL

Here’s example WebIDL for a possible extension of this specification for proximity sensors.

[SecureContext, Constructor(optional ProximitySensorOptions proximitySensorOptions)]
interface ProximitySensor : Sensor {
    readonly attribute unrestricted double distance;
};

dictionary ProximitySensorOptions : SensorOptions {
    double? min = -Infinity;
    double? max = Infinity;
    ProximitySensorPosition? position;
    ProximitySensorDirection? direction;
};

enum ProximitySensorPosition {
    "top-left",
    "top",
    "top-right",
    "middle-left",
    "middle",
    "middle-right",
    "bottom-left",
    "bottom",
    "bottom-right"
};

enum ProximitySensorDirection {
    "front",
    "rear",
    "left",
    "right",
    "top",
    "bottom"
};

11. Acknowledgements

First and foremost, I would like to thank Anssi Kostiainen for his continuous and dedicated support and input throughout the development of this specification.

Special thanks to Rick Waldron for driving the discussion around a generic sensor API design for the Web, sketching the original API on which this is based, providing implementation feedback from his work on Johnny-Five, and continuous input during the development of this specification.

Special thanks to Boris Smus, Tim Volodine, and Rich Tibbett for their initial work on exposing sensors to the web with consistency.

Thanks to Anne van Kesteren for his tireless help both in person and through IRC.

Thanks to Domenic Denicola and Jake Archibald for their help.

Thanks also to Frederick Hirsch and Dominique Hazaël-Massieux (via the HTML5Apps project) for both their administrative help and technical input.

Thanks to Tab Atkins for making Bikeshed and taking the time to explain its subtleties.

The following people have greatly contributed to this specification through extensive discussions on GitHub: Anssi Kostiainen, Boris Smus, chaals, Claes Nilsson, Dave Raggett, David Mark Clements, Domenic Denicola, Dominique Hazaël-Massieux (via the HTML5Apps project), Francesco Iovine, Frederick Hirsch, gmandyam, Jafar Husain, Johannes Hund, Kris Kowal, Marcos Caceres, Marijn Kruisselbrink, Mark Foltz, Mats Wichmann, Matthew Podwysocki, pablochacin, Remy Sharp, Rich Tibbett, Rick Waldron, Rijubrata Bhaumik, robman, Sean T. McBeth, smaug----, Tab Atkins Jr., Virginie Galindo, zenparsing, and Zoltan Kis.

We’d also like to thank Anssi Kostiainen, Dominique Hazaël-Massieux, Erik Wilde, and Michael[tm] Smith for their editorial input.

Conformance

Document conventions

Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this specification.

All of the text of this specification is normative except sections explicitly marked as non-normative, examples, and notes. [RFC2119]

Examples in this specification are introduced with the words "for example" or are set apart from the normative text with class="example", like this:

Because this document doesn’t itself define APIs for specific sensor types—that is the role of extensions to this specification—all examples are inevitably (wishful) fabrications. Although all of the sensors used a examples would be great candidates for building atop the Generic Sensor API, their inclusion in this document does not imply that the relevant Working Groups are planning to do so.

Informative notes begin with the word "Note" and are set apart from the normative text with class="note", like this:

Note, this is an informative note.

Conformant Algorithms

Requirements phrased in the imperative as part of algorithms (such as "strip any leading space characters" or "return false and abort these steps") are to be interpreted with the meaning of the key word ("must", "should", "may", etc) used in introducing the algorithm.

Conformance requirements phrased as algorithms or specific steps can be implemented in any manner, so long as the end result is equivalent. In particular, the algorithms defined in this specification are intended to be easy to understand and are not intended to be performant. Implementers are encouraged to optimize.

Conformance Classes

A conformant user agent must implement all the requirements listed in this specification that are applicable to user agents.