[Bldg-sim] New Report: Review & Comparison of Web- and Disk-based Tools for Residential Energy Analysis

[Jason Glazer]

An interesting report was recently posted by LBL titled:

  Review & Comparison of Web- and Disk-based Tools
  for Residential Energy Analysis

It can be downloaded at:

  http://eetd.lbl.gov/emills/PUBS/SoftwareReview.html

Here is the abstract:

There exist hundreds of web- and disk-based building energy 
software tools. These tools exhibit considerable range in 
approach and creativity, with some being highly specialized 
and others able to consider the building as a whole. 
However, users are faced with a dizzying array of choices 
and, often, conflicting results. The fragmentation of 
development and deployment efforts has hampered tool 
quality and market penetration. The purpose of this review 
is to provide information for defining the desired 
characteristics of residential energy tools, and to 
encourage future tool development that improves on current 
practice. We evaluated 50 web-based residential 
calculators, and 15 disk-based calculators. The comparison 
shows that some tools require a relatively small number of 
well-considered inputs while others ask a myriad of 
questions and still miss key issues. The value of detail 
has a lot to do with the type of question(s) being asked by 
the user (e.g., the availability of dozens of miscellaneous 
appliances is immaterial for a user attempting to evaluate 
the potential for space-heating savings by installing a new 
furnace). More detail does not, according to our 
evaluation, automatically translate into a “better” or 
“more accurate” tool. Efforts to quantify and compare the 
"accuracy" of these tools are difficult at best, and prior 
tool-comparison studies have not undertaken this in a 
meaningful way. The ability to evaluate accuracy is 
inherently limited by the availability of measured data. 
Furthermore, certain tool outputs can only be measured 
against “actual” values that are themselves calculated 
(e.g., HVAC sizing), while others are rarely if ever 
available (e.g., measured energy use or savings for 
specific measures). Similarly challenging is to understand 
the sources of inaccuracies. Many factors conspire to 
confound performance comparisons among tools. Differences 
in inputs can range from weather city, to types of HVAC 
systems, to appliance characteristics, to occupant-driven 
effects such as thermostat management. Differences in 
results would thus no doubt emerge from an extensive 
comparative exercise, but the sources or implications of 
these differences for the purposes of accuracy evaluation 
or tool development would remain largely unidentifiable. 
For the tools that we tested, the predicted energy bills 
for a single test building ranged widely (by nearly a 
factor of three), and far more so at the end-use level. 
Most tools over-predicted energy bills and all over-
predicted consumption. The deviations (over-predictions) we 
observed from actual bills corresponded to up to $1400 per 
year (approx. 250% of the actual bills). Energy savings 
estimates automatically generated by the web-based tools 
varied from $46/year (5% of predicted use) to $625/year 
(52% of predicted use). The estimates reflect widely 
different packages of measures proposed by the tools, and 
thus a diversity of “messages” sent to users about the 
opportunities for saving energy. Based on spot checks, we 
also discovered a remarkable number of results that suggest 
errors in programming or algorithm accuracy. There are 
numerous potential avenues for improvement of residential 
energy tools. Various important building science issues and 
energy efficiency features cannot be sufficiently well 
evaluated using existing tools. Synthesizing the 
information gathered, we developed best-practice guidelines 
that may be useful to developers of residential-energy 
tools.
=========================================================
Jason Glazer, P.E.  mailto:jglazer at gard.com  847 698 5686
GARD Analytics - http://www.gard.com/
1028 Busse Highway, Park Ridge, IL 60068
Building Energy Simulation and Analysis
Admin of BLDG-SIM list for building simulation users


===========================
You received this e-mail because you are subscribed 
to the BLDG-SIM at GARD.COM mailing list.  To unsubscribe 
from this mailing list send a blank message to 
BLDG-SIM-UNSUBSCRIBE at GARD.COM