How accurate is the self-reported energy consumption of connected devices?
By Michael Poplawski and Anay Waghale
Connected devices—such as lights, sensors, power strips and other equipment in buildings—can bring greater functionality and services, which often entails operating modes beyond simple “on” and “off.” Device energy consumption can vary depending on how much data they are producing or analyzing, and devices may operate in low-power or stand-by modes when they are not active. This makes it more difficult to estimate energy consumption, because the energy performance of connected devices depends on what operating modes they use and how much time they spend in each mode. The uncertainty about energy performance can be largely mitigated if connected systems can accurately report their own energy consumption, and many emerging connected lighting systems already offer some form of this capability.
But how accurate is this energy self-reporting? To find out, Pacific Northwest National Laboratory (PNNL), on behalf of the U.S. Department of Energy (DOE), conducted a study that explored the energy-reporting accuracy of market-available connected devices, in this case connected electrical outlets. The study, conducted at PNNL’s Connected Lighting Test Bed, was intended to generate awareness of building systems that are capable of reporting their own energy consumption, and to further interest in the value of energy data for a variety of uses. The findings draw attention to how the accuracy of reported metrics can be characterized and quantify the performance variation found in market-available products.
THE STUDY CONSIDERED two residential-market products and three commercial-market products with the ability to report the power drawn and/or energy consumed by devices plugged in to their connected electrical outlets. Energy-consumption data reported by the devices were compared to measurements taken by a reference meter under 10 test conditions. The residential products reported power draw but not cumulative energy consumption; the commercial products reported both power draw and cumulative energy consumption.
Analysis of the results revealed variations across test conditions and units, as summarized in Figures 1A and B. The average relative reporting error (RRE) for the two residential products, derived from the reported power draw, was −0.02% and −1.20%. The average RRE for two of the three commercial products, derived from reported power draw, was greater than those of the residential products (-2.40%, -2.72%, -0.36%). The average commercial-product RRE derived from reported energy consumption should be very consistent and better than performance based on reported power draw. However, the average RREs derived from reported energy consumption varied significantly (17%, -10.78%, -1.5%) across the three makes of commercial-market products and were uniformly less accurate than performance based on reported power draw. Subsequent analysis identified a number of root causes for this decrease in performance, most of which were related to reporting resolution.
The power draw required to provide the energy-reporting functionality can vary significantly with the connected load. While all makes and models showed very little dependence on the device being tested, the power draw of the commercial units varied considerably across load conditions, as shown in Figure 2. The average power draws for the three commercial units (9.76 watts, 8.10 watts, 21.76 watts) were significantly higher than those of the residential units (1.21 watts, 2.11 watts), as were their average no-load power draws (5.13 watts, 5.72 watts and 20.50 watts for the commercial units, vs. 1.31 watts and 1.95 watts for the residential units).
THE RESULTS OF THIS STUDY and subsequent related work may be relevant to industry specification and standards development organizations. The methods this study employs could inform test and measurement procedures and performance classifications for connected outlets, lighting products and other building systems capable of reporting their own energy consumption.
For example, there is a need for application-specific performance classifications that end users can understand and relate to their energy-data use needs (e.g., 2% accuracy class for utility streetlight energy billing needs, or 10% accuracy class for ESCO performance verification needs). Additional recommendations from the study include development of internal power-draw limits for energy-reporting devices and systems under well-defined operating conditions, and test methods to verify whether energy-reporting devices and systems comply with established internal power-draw limits and application-specific performance classifications.
Owners, operators and specifiers of energy reporting can support the development of relevant industry standards and specifications by analyzing the dependency of current and planned energy-data use cases on accuracy—noting in particular the degree of dependence on relative vs. absolute accuracy and on trueness vs. precision (i.e., repeatability)—and communicating use-case needs to industry-standards and specification organizations.
In the meantime, manufacturers developing products that report energy consumption should:
- Characterize the accuracy of the reported metrics using a reference meter calibrated by an independent laboratory that was accredited by an ILAC MRA signatory whose scope of accreditation explicitly covers energy measurement and should be sure to include this information on product data sheets.
- Clearly document resolution for all reported metrics, via all reporting interfaces (e.g., hardware display, software user interface, data exports, API), on product data sheets.
- Contribute to the development of industry-standard test methods for characterizing energy-reporting accuracy, such as those currently being developed by the ANSI C136 – Standards for Roadway and Area Lighting Equipment and ANSI C137 – Standards for Lighting Systems committees.
- Develop product designs that enable efficient characterization of reporting accuracy by independent laboratories and other interested parties.
- Report the internal power draw of energy-reporting devices and systems under well-defined conditions (e.g., minimum, maximum, noload), and contribute to the development of industry-standard power-draw limits.
- Implement vendor-neutral common Representational State Transfer (REST) APIs for common product types and applications, as has been done for some commercial power distribution units.
- Report energy data using industry-standard information and semantic models (such as those underway in ASHRAE AP Working Group, the Open Connectivity Foundation, and the ZigBee Alliance) and contribute to their ongoing development of these models.
The accuracy of self-reported energy consumption in connected devices is an important topic, but it’s also a complex one. For additional details— including a normality analysis that attempted to separate systematic and random error sources—download the full report at www.energy.gov.