RE: [EnergyPlus_Support] Proper use of R-squared evaluation for energy use

My thanks to all who responded! I think I understand it better now.

A summary of those responses and some personal research is shown below:

Predicted variable:	Vp
Measured Variable:	Vm
Error:	E = (Vp - Vm)
Number of measurements / predictions:	n

RMSE:	[(E1² + E2² + E3² + .... En²)/n]^.5

RMSE (%):	RSME
	(Vm1 + Vm2 + Vm3 + ...Vmn) / n

I don’t think R² is as important for energy model calibration purposes as RMSE (Root Mean Squared Error).

It appears that RMSE is normally calculated as a numerical value with the same units as the measured variable. I prefer to express it as percent of the measured variable (utility bills), so I added that value.

Complementary measures include the annual total error and monthly error, which can also be expressed as a percent. The chart below is borrowed from the IBPSA “Energy Modeling Best Practices �” seminar at the ASHRAE Winter Conference 2013.�� These, in turn, are cited as from FEMP M&V Guidelines v. 2.2 and ASHRAE Guideline 14-2002.

For all of these measures, a smaller value is better!

James V Dirkes II, PE, BEMP, LEED AP
www.buildingperformanceteam.com
Energy Analysis, Commissioning & Training Services
1631 Acacia Drive, Grand Rapids, MI 49504 USA
616 450 8653

From: EnergyPlus_Support@xxxxxxxxxxxxxxx [mailto:EnergyPlus_Support@xxxxxxxxxxxxxxx] On Behalf Of jeannieboef@xxxxxxxxx
Sent: Sunday, March 24, 2013 4:44 PM
To: EnergyPlus_Support@xxxxxxxxxxxxxxx
Cc: EnergyPlus_Support@xxxxxxxxxxxxxxx
Subject: Re: [EnergyPlus_Support] Proper use of R-squared evaluation for energy use

Isn't adding errors the same as adding tolerances...at any one point in time you could be "wrong" by error 1 + error 2.

Jean

Sent from my iPhone

On 24.03.2013, at 21:01, Niraj Poudel <nirajpdl@xxxxxxxxx> wrote:

Jim,

May I ask you:

a) Are you looking for 1 regression model that captures the energy use for both, when you use electricity and gas?

b) I guess the reason why I am asking this is because:

This question arises because I am looking at a model which, when using 24 data pairs, shows:
·         R² = 84% for electricity,
·         R² = 61% for natural gas,
·         and R² = 57% overall.

When you say that using 24 data points gives you a R^2 of 84% for electricity and R^2 of 61% for natural gas, I cannot understand how you get separate R^2 values for each parameter (electricity and natural gas). Are you setting up your regression model with energy consumption as a dummy variable (if electricity = 1 if not = 0) kind of deal?

Anyways, if I could see the data and understand it better, I could go ahead and run the regression model in SAS and see if I get anything different. Plus, it would be only on monday that I could access the computer in my office. If that would be no problem. I could take a stab at it.

Niraj

On Sun, Mar 24, 2013 at 3:42 PM, Jim Dirkes <jim@xxxxxxxxxxxxxxxxxxxxxxxxxxx> wrote:

Dear Forum,
I don’t recall if this has been a topic in the past � my apologies if it has! (Just point me to a past post)
When creating an energy model for an existing building that has historical (metered) consumption data, I typically calculate an R² value for the goodness of fit between actual and predicted energy. I do this for electricity use and also for natural gas / fossil fuel use.
That is fairly straightforward (or this statistics-challenged engineer wouldn’t calculate it at all!)
Here’s the question:
·         There are normally 12 data pairs for electricity and 12 for gas
·         Assuming that I have made the dates coincident for electricity and gas,
·         and also use consistent units (e.g., kWh for both)
·         If I want the overall R² value (composite of all energy), should I use 24 data pairs in the calculation or 12?

This question arises because I am looking at a model which, when using 24 data pairs, shows:
·         R² = 84% for electricity,
·         R² = 61% for natural gas,
·         and R² = 57% overall.

Note that the overall value is less than either of its components. If I use 12 data pairs, the overall R² = 69%. Intuitively, 69% makes more sense because it falls between the other two values, so I am inclined toward use of 12 data pairs. I’m even more inclined, however, to make the calculation correctly! So I’m reaching out for input, hoping that some of you are more knowledgeable in the area of statistics than I. Which is more correct � 12 or 24 data pairs? Thanks in advance.

James V Dirkes II, PE, BEMP, LEED AP
www.buildingperformanceteam.com
Energy Analysis, Commissioning & Training Services
1631 Acacia Drive, Grand Rapids, MI 49504 USA
616 450 8653

--
Niraj Poudel, Architectural Engineer.
PhD student, PDBE Program.
Clemson University, Clemson, SC.

__._,_.___

Primary EnergyPlus support is found at: http://energyplus.helpserve.com or send a message to energyplus-support@xxxxxxxx The primary EnergyPlus web site is found at: http://www.energyplus.gov The group web site is: http://groups.yahoo.com/group/EnergyPlus_Support/ Attachments are currently allowed but be mindful that not everyone has a high speed connection. Limit attachments to small files. EnergyPlus Documentation is searchable. Open EPlusMainMenu.pdf under the Documentation link and press the "search" button.

Your email settings: Individual Email|Traditional
Change settings via the Web (Yahoo! ID required)
Change settings via email: Switch delivery to Daily Digest | Switch to Fully Featured
Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe

__,_._,___