[Bldg-sim] Energy model calibration - normalizing the utility bills to month start-end

[Macumber, Daniel]

Since we have your ear Jeff, I have wondered if Guideline 14 describes any penalty to apply to the goodness of fit metric when allowing unknown parameters to be varied to improve the fit to monthly data?  Something like how adjusted R-squared or AIC penalize models with more parameters?  It seems that if you achieve a given level of goodness of fit after fiddling with a bunch of parameters then that is not as good as achieving the same level of fit with fewer free parameters.  Is there any way to estimate the number of degrees of freedom lost when changing a single numeric model parameter like ACH vs a more complicated one like changing out entire HVAC curves?

Dan


On Jun 27, 2015, at 9:00 AM, Jeff Haberl <jhaberl at tamu.edu<mailto:jhaberl at tamu.edu>> wrote:


Hello Maria,



Thanks for actually reading Guideline 14-2014. However, I'm sorry to disagree, but Section 5.3.3.3.7 in Guideline 14-2014 seems pretty clear to me (...since I had a hand in writing it). In fact, I can't imagine doing a calibrated simulation without checking the defaults (as a minimum). Unfortunately, understanding the performance curves of the installed equipment can be tricky since few if any systems installed today come with factory-generated performance curves that are plug-and-play with simulation curve inputs. However, I think Guideline 14 is right in shining a light on this issue. Don't forget, one of the purposes of a guideline is to bring issues to the surface so knowledgeable users can decide to do something or justify why they did not do it.



In the same light, the advice in Section 5.3.3.3.6 also seems reasonable (....I think I wrote this too...). If you have actual blower door or duct blaster measurements, then these would be preferred over an empirical adjustment. There are many studies out there that used the "brute force" ACH adjustment for infiltration, which can improve a model's fit after all other variables are considered.



Your suggestion to more closely link the calibrated simulation to  the intended ECM is reasonable. In fact, now that Guideline 14 is published, we'll probably be starting in on the "next version" in a year or so and I would invite you to join the Committee.



Jeff


PS: Did you download and try the software that now comes with Guideline 14-2014?...this includes the IMT from RP1050 and Diversity Factor toolkit from RP1093.

8=!  8=)  :=)  8=)  ;=)  8=)  8=(  8=)  8=()  8=)  8=|  8=)  :=')  8=) 8=?
Jeff S. Haberl, Ph.D.,P.E.inactive,FASHRAE,FIBPSA,......jhaberl at tamu.edu<mailto:........jhaberl at tamu.edu>
Professor........................................................................Office Ph: 979-845-6507
Department of Architecture............................................Lab Ph:979-845-6065
Energy Systems Laboratory...........................................FAX: 979-862-2457
Texas A&M University...................................................77843-3581
College Station, Texas, USA, 77843.............................http://esl.tamu.edu
8=/  8=)  :=)  8=)  ;=)  8=)  8=()  8=)  :=)  8=)  8=!  8=)  8=? 8=) 8=0
________________________________
From: Bldg-sim [bldg-sim-bounces at lists.onebuilding.org<mailto:bldg-sim-bounces at lists.onebuilding.org>] on behalf of Maria Karpman [maria.karpman at karpmanconsulting.net<mailto:maria.karpman at karpmanconsulting.net>]
Sent: Thursday, June 25, 2015 10:48 AM
To: 'Jim Dirkes'; 'Justin Spencer'
Cc: 'Building Simulation'
Subject: Re: [Bldg-sim] Energy model calibration - normalizing the utility bills to month start-end

Jim,

FYI attached is a presentation from 2014 ASRHAE/IBPSA summarizing our findings on accuracy of the calibrated simulations based on projects that went through an incentive program in NJ. (You presented there also – loved your Sherlock Holmes hat and pipe :).) To clarify the language, ECM savings estimated via calibrated simulation are referred in the presentation as “projected savings”; ECM savings estimated via “whole building” approach are referred to as “realized savings”.  Slide 7 summarizes high level results – while there was a relatively good alignment between the total projected and realized savings for the entire sample, individual projects were all over the map.

I agree with your last point below that the greatest source of error is too little measured data, but given the finite budget that each project has, I hoped that documents such as Guideline 14 would emphasize the link between the minimum scope of measurements and ECMs considered for a given project. For example Guidelines 14 says the following concerning simulation defaults, which in my opinion is impractical:

5.3.3.3.7 Minimizing Default Values. Check and thoroughly
understand all default input variables in the simulation
program, as many of the default values have little resemblance
to the actual building being simulated. The fewer the
number of default values used, the more representative the
simulation will be—but only if the changes are well reasoned.
This also includes inspection of the default performance
curves of the various systems and plant equipment, because
such curves can significantly impact the results of the simulation.
Any program default values that are altered, however,
should be well documented.

There is also a section on adjusting infiltration / ventilation rates to calibrate the model (quoted below), but it should only apply if project does not include ECMs involving air-sealing (which are common in multifamily and school projects) or ventilation controls such as DCV. If project includes infiltration or ventilation related measures, the related inputs cannot be used to calibrate the model even as “a last resort”.

5.3.3.3.6 Estimating Infiltration Rates. Infiltration
rates are difficult to measure and may be treated as an
unknown that is iteratively solved with the simulation program
once the other major parameters are determined. This
approach is only recommended as a last resort. To solve for
the infiltration rate and/or the ventilation rate iteratively, conduct
a series of simulation runs such that only the infiltration
and/or ventilation rates are changed from one-tenth to as
much as ten times the expected rates. Next, compare the simulation
outputs produced to the measured building data as discussed
below. In addition, supporting evidence should be
used to justify the final choice of variables.

I think calibrated simulation should be treated as an extension of retrofit isolation approach, and site measurements performed in support of simulation must at minimum include (a) parameters that are modified to model ECM savings, and (b) simulation inputs that do not change but drive ECM savings. For lighting fixture replacement ECM, pre-retrofit wattage would be an example of (a), and lighting runtime hours would be an example of (b). On the other hand, if there are no ECMs that reduce base load (e.g. plug load, lighting, etc.), but there are ECMs involving HVAC controls, site measurements should focus on operation of existing controls, and establishing existing lighting wattage is not necessary because model calibrated to capture the overall base load (lighting + plug loads) usage may be good enough in this case. We had to develop guidelines for incentive programs that rely on calibrated simulation to specify minimum scope of measurements and acceptable “estimated” inputs for each common ECM type. I hope that a similar guidance would be included at some point in one of ASHRAE standards / guidelines. As it stands now, there is a sea of difference between the specificity of simulation requirements in 90.1 Appendix G compared to calibrated simulation of existing buildings. Unless I am missing some key reference?

Maria

--
Maria Karpman LEED AP, BEMP, CEM
________________
Karpman Consulting
www.karpmanconsulting.net<http://www.karpmanconsulting.net/>
Phone 860.430.1909
41C New London Turnpike
Glastonbury, CT 0603
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 139044 bytes
Desc: image001.png
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20150627/bfb02575/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 58565 bytes
Desc: image002.png
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20150627/bfb02575/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 6574 bytes
Desc: image003.png
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20150627/bfb02575/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 6574 bytes
Desc: image003.png
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20150627/bfb02575/attachment-0003.png>