[Bldg-sim] Comparing ASHRAE 90.1 App G Models to Real Buildings (UNCLASSIFIED)

[Eurek, John S NWO]

Classification: UNCLASSIFIED
Caveats: NONE

Fred,

Thank you for the link to the article. It was fascinating. That answered exactly what I was wondering about.


John Eurek PE, LEED AP
Mechanical Engineer,


-----Original Message-----
From: Fred Betz [mailto:fbetz at aeieng.com] 
Sent: Wednesday, May 27, 2015 4:03 PM
To: Eurek, John S NWO; bldg-sim at lists.onebuilding.org
Subject: [EXTERNAL] RE: [Bldg-sim] Comparing ASHRAE 90.1 App G Models to Real Buildings (UNCLASSIFIED)

Just catching up with this thread so I'm sorry if I'm repeating anything that's already been mentioned. 

Take a look at the paper from Pam Berkeley et al published last year at SimBuild. 
https://www.ashrae.org/membership--conferences/conferences/ashrae-ibpsa-usa-papers

10 experienced energy modelers modeling the same building in a 3hr period. Fascinating results. 


There are emerging methods to do more rigorous QC using a variation on Monte Carlo for energy models to calculate a confidence interval for the model rather than fully rely on modeler experience and 3rd party QC. Georgia Tech has integrated this into their version of e+, which I hope gets integrated into a future version of e+. It's computationally intense so cloud computing is probably the right way to do this, which I believe e+ is heading towards. 


Fred


FRED BETZ  PhD., LEED AP ®BD+C
SENIOR SUSTAINABLE
DESIGN CONSULTANT
 
AEI | AFFILIATED ENGINEERS, INC.  
5802 Research Park Blvd. | Madison, WI  53719

P: 608.236.1175 | F: 608.238.2614  
fbetz at aeieng.com  |  www.aeieng.com  


-----Original Message-----
From: Eurek, John S NWO [mailto:John.S.Eurek at usace.army.mil] 
Sent: Wednesday, May 27, 2015 11:28 AM
To: bldg-sim at lists.onebuilding.org
Subject: Re: [Bldg-sim] Comparing ASHRAE 90.1 App G Models to Real Buildings (UNCLASSIFIED)

Classification: UNCLASSIFIED
Caveats: NONE

This is a slightly different question: 
How close do you expect 2 energy models to be created by 2 different modelers (using the same program) if you give them the same plans and information?

(Ask Pablo Picasso, Salvador Dali and Rembrandt to draw a tree)

I assume most companies don't double up on the energy modeling efforts which would show how consistent or non-consistent energy models are. (assuming the energy modelers are experienced and competent.)

Somebody who teachers energy modeling may be able to provide insight and good examples.

As far as an energy model matching the actual utilities bills..... If you have a 1000 modelers, making models on 1000 computers for 1000 years......



-----Original Message-----
From: Bldg-sim [mailto:bldg-sim-bounces at lists.onebuilding.org] On Behalf Of Jacob Dunn
Sent: Friday, May 22, 2015 2:37 PM
To: bldg-sim at lists.onebuilding.org
Subject: [EXTERNAL] Re: [Bldg-sim] Comparing ASHRAE 90.1 App G Models to Real Buildings

Thank you all for your thoughtful insight on this matter!  It’s an important debate – both in understanding the capabilities/limitations of our craft as energy modelers, but also to communicate our value to the community at large. 

 

To clarify slightly, my question revolved around how the specific modeling protocol of Appendix G could account for the “performance” gap between modeled and actual use.  Thus, the fact that buildings aren’t operated as the energy model specified and the lack of building commissioning, while true and important, are not inherent to the intent of Appendix G modeling.  The most interesting question is, “If you model a LEED App G model perfectly according to protocol, AND the building was operated according to the modeled schedules, will it predict the right number?”  Lots of your responses lent insight into this question, thanks again!  

 

I’ve revised my list below based on your responses:

 

Added:

-   Plug load values are assumed, which can have a huge impact on overall energy (Thanks Christoph and Chris Hadlock for the insight)

-  Insulation values are largely specified without thought to thermal bridging

-  HVAC controls simulation is often simplified

- Performance curves are often not simulated due to increased effort and unavailability of performance data from manufacturers

 

Original:

-          Appendix G does not take into account external shading, which can be critical in urban environments for accurate energy predictions

-          Schedules are typically not created with the intent of being predictive.  Overall building hours are adhered to, but detailed schedule creation is not usually in the scope of a LEED model (or is it, in your experience?).  For instance, typical plug load base values during unoccupied times are .3, this is a pretty big assumption.

-          The App G model uses a TMY weather file, which can vary from the current weather year (I wonder on average by how much?)

-          Infiltration values are assumed, unless blower door testing has been done (which is rare for commercial buildings).

-          Thermostat values are modeled as consistent across the building, which is rarely the case in an actual operating building

Cheers,

 

Jacob Dunn LEED AP BD+C



ESKEW+DUMEZ+RIPPLE, APC

2014 AIA National Architecture Firm Award

 

365 Canal Street Suite 3150 

New Orleans LA 70130

504.561.8686

eskewdumezripple.com <http://www.eskewdumezripple.com/> 

 

From: Bldg-sim [mailto:bldg-sim-bounces at lists.onebuilding.org] On Behalf Of Chris Hadlock
Sent: Friday, May 22, 2015 1:08 PM
To: Christoph Reinhart
Cc: bldg-sim at lists.onebuilding.org
Subject: Re: [Bldg-sim] Comparing ASHRAE 90.1 App G Models to Real Buildings

 

All,

 

I would agree with all of the factors mentioned that absolutely can result in deviations between actual and modeled building performance. I would also echo the sentiment that following modeling rules shouldn't necessarily preclude us from attempting to better predict actual building performance through the LEED process. Applying careful attention to important details and a healthy dose of experience (bringing together real life building performance knowledge as it relates to the grey areas - namely schedules, equipment controls, occupant behavior, etc) can truly help close the gap. At the end of the day, a rating system should be attempting to reward buildings that actually perform well, not theoretically perform well (and as modeler's we should take a leading role in making good (i.e. fair) assumptions).

 

My colleague (Janine Vanry) has recently completed research (to be published soon) for her masters thesis at the University of Waterloo (Ontario, Canada) which studied how LEED certified academic buildings in southwestern Ontario performed in comparison to government energy intensity benchmarks, campus-wide energy intensities, and in general how LEED (modeled) results compare to actual building performance (as measured through M&V). Consistent with Dr Samuelson's (et al.) research findings, the discrepancies between the modeled results and the actual energy intensities showed that there was an under-prediction anywhere from 2% to 44%. 

 

While energy modeling professionals understand (as is evident by this thread) that there will be differences between the documented EAc1 energy savings and actual building energy usage, this isn't always communicated and understood by the building owners and the professionals we work with.

 

Chris

 

On Thu, May 21, 2015 at 10:56 AM, Christoph Reinhart <tito_ at mit.edu> wrote:

	Dear Jacob,

	 

	This is an eternal debate and there are many reasons for moving away from the use of 90.1 Appendix G to evaluate the performance of a building designs.  To answer your question directly, we worked a few years ago with Enermodal in Canada on a comparison between design phase building energy models (BEM) prepared for LEED Canada certification (slightly different to Appendix G) to calibrated BEM and measured energy use for 18 buildings. The main findings are quoted below:

	 

	Analysis of a Simplified Calibration Procedure for 18 Design-Phase Building Energy Models

	H W Samuelson, A Ghorayshi and C F Reinhart

	Journal of Building Performance Simulation, DOI: http://dx.doi.org/10.1080/19401493.2014.988752 <http://dx.doi.org/10.1080/19401493.2014.988752>  

	This paper evaluates the accuracy of 18 design-phase building energy models, built according to LEED Canada protocol, and investigates the effectiveness of model calibration steps to improve simulation predictions with respect to measured energy data. These calibration steps, applied in professional practice, included inputting actual weather data, adding unregulated loads, revising plug loads (often with submetered data), and other simple updates. In sum, the design-phase energy models underpredicted the total measured energy consumption by 36%. Following the calibration steps, this error was reduced to a net 7% underprediction. For the monthly energy use intensity (EUI), the coefficient of variation of the root mean square error improved from 45% to 24%. Revising plug loads made the largest impact in these cases. This step increased the EUI by 15% median (32% mean) in the models. This impact far exceeded that of calibrating the weather data, even in a sensitivity test using extreme weather years.

	 

	Best,

	 

	Christoph

	Christoph Reinhart 

	Associate Professor 

	Department of Architecture

	Massachusetts Institute of Technology

	77 Massachusetts Ave, Rm 5-418, Cambridge, MA 02139, USA

	t: 617 253 7714 <tel:617%20253%207714> , f: 617 253 6152 <tel:617%20253%206152> , creinhart at mit.edu <mailto:creinhart at mit.edu>  

	Sustainable Design Lab <http://mit.edu/SustainableDesignLab/>  | DIVA <http://www.diva4rhino.com/>  | Daysim <http://daysim.ning.com/>  | mapdwell <http://www.mapdwell.com/>  | umi <http://www.urbanmodeling.net/> 

	-------------------------------------------------------------------------

	Events  Modeling Urban Sustainability <http://architecture.mit.edu/event/modeling-urban-sustainability-energy-daylight-and-walkability>  | DIVA Day 2015 <http://diva4rhino.com/diva-day-2015> 

	 

	 

	 

	From: Bldg-sim [mailto:bldg-sim-bounces at lists.onebuilding.org] On Behalf Of Brooks, Alamelu
	Sent: Thursday, May 21, 2015 10:43 AM
	To: Jim Dirkes; Nathan Kegel
	Cc: bldg-sim at lists.onebuilding.org
	Subject: Re: [Bldg-sim] Comparing ASHRAE 90.1 App G Models to Real Buildings

	 

	I believe Appendix G is not meant to measure the performance of the existing building. ASHRAE 90.1 Appendix G Technical Committee is the right source to answer this question. They can clarify the intention of the APP G modeling methodology.

	 

	Best,

	Alamelu

	Alamelu  Brooks LEED AP (BD+C), HBDP, BEAP, EIT| Senior Associate | +1.443.718.4881 <tel:%2B1.443.718.4881>  direct | Alamelu.Brooks at icfi.com <mailto:Alamelu.Brooks at icfi.com>  | icfi.com 

	ICF INTERNATIONAL | 7125 Thomas Edison Drive, Suite 100, Columbia, MD 21046 USA 

	Connect with us on social media <http://www.icfi.com/social> .

	 

	 

	From: Bldg-sim [mailto:bldg-sim-bounces at lists.onebuilding.org] On Behalf Of Jim Dirkes
	Sent: Thursday, May 21, 2015 10:36 AM
	To: Nathan Kegel
	Cc: bldg-sim at lists.onebuilding.org
	Subject: Re: [Bldg-sim] Comparing ASHRAE 90.1 App G Models to Real Buildings

	 

	I agree fully with all of the above comments and would like to add these:

	*	Even buildings that are commissioned properly will see their performance erode over time.  There are hundreds of reason for this, ranging from poor maintenance to well-intentioned maintenance people not having time to monitor operations well.  There is NO BUILDING that operates well for long.
	*	Buildings often see changes in operation, occupancy and schedule.  These are oftimes gradual changes over a period of years, but can be substantial

	 

	On Thu, May 21, 2015 at 10:22 AM, Nathan Kegel <nathan.kegel at iesve.com> wrote:

	Climate files used in the simulations versus the actual weather.

	 

	I’m in the midst of a project that shows a variance in EUI of up to 200% just by changing the climate file for the DOE primary school.  Full results to be presented in September.

	 

	Add in all the other factors already mentioned, and if your 90.1 model comes anywhere close the real buildings’ it’s far more likely that the 90.1 model was extremely “lucky” than it is that the model used accurate assumptions.

	 

	Regards,

	 

	Nathan

	 

 <http://www.iesve.com/> 

Nathan Kegel
Business Development Manager

O:

  763.276.9981

M:

  415.420.9314

http://www.iesve.com <http://www.iesve.com/>  

Integrated Environmental Solutions Limited. Registered in Scotland No. SC151456 Registered Office - Helix Building, West Of Scotland Science Park, Glasgow G20 0SP

Email Disclaimer <http://www.iesve.com/disclaimer.html>  

	 

	 

	From: Bldg-sim [mailto:bldg-sim-bounces at lists.onebuilding.org] On Behalf Of Maria-Lida Kou
	Sent: Thursday, May 21, 2015 9:17 AM
	To: Jacob Dunn
	Cc: bldg-sim at lists.onebuilding.org
	Subject: Re: [Bldg-sim] Comparing ASHRAE 90.1 App G Models to Real Buildings

	 

	Jacob,

	 

	Happy to hear that other people are thinking the same. 

	 

	I was into this subject on my own thoughts recently.

	 

	I would like to add in your list: Occupants' behavior actually which is not in the stage to be included into the prediction.

	I would add commissioning as well along with controls simulation and controls operation.

	 

	Apologies because I haven't worked with LEED projects but I think the above applied in general to "the performance gap".

	 

	Really looking forward to hearing more about this subject as I am not that experienced engineer yet, but really interested in "the performance" side of buildings.

	 

	Best,

	Maria-Lida Kounadi

	 

	 

	2015-05-21 15:04 GMT+01:00 Jacob Dunn <jdunn at eskewdumezripple.com>:

		Bldg-Sim Community – 

		 

		I’m trying to compile a list of why it might be inappropriate to compare Appendix G models to actual consumption data.  This comes about because I recently got into a debate with one of my co-workers when looking at the infamous NBI chart/study that shows little correlation to predicted and actual energy values of LEED buildings.  I was trying to explain that the Appendix G model’s intent is NOT to be compared to actual consumption, as it is a modeling protocol aimed at creating consistent relative comparisons for LEED points. 

		 

		Here are the reasons thus far that support this notion (that App G models shouldn’t be compared to actual data).  Does anyone know of any resources out there that expand upon this?  Or can you think other reasons?

		 

		-          Appendix G does not take into account external shading, which can be critical in urban environments for accurate energy predictions

		-          Schedules are typically not created with the intent of being predictive.  Overall building hours are adhered to, but detailed schedule creation is not usually in the scope of a LEED model (or is it, in your experience?).  For instance, typical plug load base values during unoccupied times are .3, this is a pretty big assumption.

		-          The App G model uses a TMY weather file, which can vary from the current weather year (I wonder on average by how much?)

		-          Infiltration values are assumed, unless blower door testing has been done (which is rare for commercial buildings).

		-          Thermostat values are modeled as consistent across the building, which is rarely the case in an actual operating building

		 

		Any additional insight is much appreciated!

		 

		 

		Jacob Dunn LEED AP BD+C

		ESKEW+DUMEZ+RIPPLE, APC 

		2014 AIA National Architecture Firm Award

		 

		365 Canal Street Suite 3150 

		New Orleans LA 70130

		504.561.8686

		eskewdumezripple.com <http://www.eskewdumezripple.com/> 

		 

		
		_______________________________________________
		Bldg-sim mailing list
		http://lists.onebuilding.org/listinfo.cgi/bldg-sim-onebuilding.org
		To unsubscribe from this mailing list send  a blank message to BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG <mailto:BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG> 

	 

	
	_______________________________________________
	Bldg-sim mailing list
	http://lists.onebuilding.org/listinfo.cgi/bldg-sim-onebuilding.org
	To unsubscribe from this mailing list send  a blank message to BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG <mailto:BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG> 

	
	
	

	 

	-- 

	James V Dirkes II, PE, BEMP, LEED AP
	CEO/President
	The Building Performance Team Inc.
	1631 Acacia Dr, GR, Mi 49504
	
	Direct: 616.450.8653
	jim at buildingperformanceteam.com
	
	Website <http://buildingperformanceteamcom> l  LinkedIn <https://www.linkedin.com/pub/jim-dirkes/7/444/413> 

	The truth is still the truth, even if nobody believes it.  A lie is still a lie, even if everyone believes it.

	
	_______________________________________________
	Bldg-sim mailing list
	http://lists.onebuilding.org/listinfo.cgi/bldg-sim-onebuilding.org
	To unsubscribe from this mailing list send  a blank message to BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG <mailto:BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG> 

 


Classification: UNCLASSIFIED
Caveats: NONE



Classification: UNCLASSIFIED
Caveats: NONE