Re: [EnergyPlus_Support] WEATHER DATA: the TMY3 weather files for 1,020 US locations have been revised

I forgot to mention something that's puzzled me, i.e., the rumor that the TMY3s don't contain rain data. With the exception of very few stations (< 3%) that did not report any rainfall, the TMY3s have always contained rain or liquid precipitation data, just that the reported values seem to be irregular and not correlate with other known sources. I'm just finishing up a brief report on this topic that I will upload in a few hours at www.whiteboxtechnologies/download_TMY3_liqprec_rpt.htm

Joe

Joe Huang
White Box Technologies, Inc.
346 Rheem Blvd., Suite 205A
Moraga CA 94556
yjhuang@xxxxxxxxxxxxxxxxxxxxxxxx
http://weather.whiteboxtechnologies.com for simulation-ready weather data
(o) (925)388-0265
(c) (510)928-2683
"building energy simulations at your fingertips"

On 1/16/2015 9:24 AM, Joe Huang wrote:

Jean, others,

Yes, the liquid precipitation values in the TMY3 are not randomized values from an external source, but a better processing of the same raw data, i.e., standard weather station reports, used to create the TMY3s. Therefore, they should be synchronized with the other values, which is very important for simulations because it tends to be cool, humid, and very cloudy when it rains :-)

Although I've added a new field for PresentWeather (nota bene: this parameter is in the TMY3A.CSV but not in the EPW) that will allow you to check whether the station reports says it's raining or not (Present Weather = 60 through 68), I did not use it as a filter in my processing, so it's still conceivable for there to be liq precip when the sun is shining or the station reports no rain.

As for extending this rain data to international stations, that's already been done in the ASHRAE IWEC2 weather files. In fact, this revision of the TMY3 relied on the technique original developed to process the IWEC2 weather files.

I had a chuckle when I saw you wrote "co-inside" instead of "coincide", because when my older brother first taught me the word "coincide" when I was 12, I thought he was saying "go inside".

Joe
Joe Huang
White Box Technologies, Inc.
346 Rheem Blvd., Suite 205A
Moraga CA 94556
yjhuang@xxxxxxxxxxxxxxxxxxxxxxxx
http://weather.whiteboxtechnologies.com for simulation-ready weather data
(o) (925)388-0265
(c) (510)928-2683
"building energy simulations at your fingertips"
On 1/16/2015 2:17 AM, Jean Marais jeannieboef@xxxxxxxxx [EnergyPlus_Support] wrote:

Hi Joe,

That sounds awesome. The missing rain data has been a headache for a long time. Next, you can do the european locations :-).

I have one question:

Do the presipitation, i.e. rain flags, co-inside with the actual temps, humidities and cloudiness in the TMY year? It's an obvious quesion sorry, but I just want to make sure it's not going to rain when the sun is shining, if you know what I mean.

Kind regards,

Jean

2015-01-15 3:04 GMT+01:00 Joe Huang YJHuang@xxxxxxxxxxxxxxxxxxxxxxxx [EnergyPlus_Support] <EnergyPlus_Support@xxxxxxxxxxxxxxx>:

Revision of TMY3 Weather Files

White Box Technologies (WBT), in collaboration with the National Renewable Energy
Laboratory (NREL), has revised the TMY3 weather files to correct inconsistencies in units
for illuminances, improve the reporting of Liquid Precipitation, replace all missing
records with interpolated or filled values (chiefly for visibility, ceiling height,
aerosol optical depth, and albedo), and add a new variable for Present Weather.

Since their release in 2005, the TMY3 weather files covering 1,020 locations in the US and
dependencies has become the standard set of weather files for use in computer simulations
of solar energy conversion systems and buildings. For more information about the TMY3 data
set, please go to http://rredc.nrel.gov/solar/old_data/nsrdb/1991-2005/tmy3.

The intent of this revision is not to redo the fundamental analysis, but to correct some
known errors and make modest improvements in the reporting of subsidiary climate
variables. The only changes affecting building energy simulations are those for
Illuminance (used in daylighting programs such as Radiance), and Liquid Precipitation
(used in EnergyPlus to model green roofs, and other programs that model moisture migration
in building envelope). Filling in missing data for Visibility, Ceiling Height, etc.,
should have no effect on any known simulation program, but makes the TMY3 files more
usable for further analysis.

Explanation of changes in content:

1) Illuminance. The TMY3s contain values for Global Horizontal, Direct Normal, Diffuse
Horizontal, and Zenith Illuminance that are calculated based on the sun position and solar
radiation. Although the documentation states that the units are 100 lux for the first
three illuminances, and 10 candela/m2 for the Zenith Illuminance, all the TMY3 files used
these units only for the period Jan. 2-31, and for the rest of the year used units of lux
and candela/m2, respectively. NREL put out a notice in August 2008 (see the link listed
earlier) informing users of this discrepancy and promising a corrected version that didn't
make it out until now! In this revision, we've decided to keep the units in the original
TMY3 documentation, i.e., 100 lux or 10 candela/m2, which is the same as what was in the
TMY2, and avoids the superfluous five-digit precision.

2) Liquid Precipitation. The TMY3s are the first set of TMY files to contain Liquid
Precipitation (rainfall). In the original TMY3 files, rainfall was taken directly from
weather station reports with two numbers, one for the amount and the other for the
duration in hours of the rainfall. The difficulties moving this reporting to hourly
weather files are: (1) the rainfall needs to be apportioned to hourly values, and (2)
there can be multiple or overlapping records that were ignored. Furthermore, by changing
the precipitation units from 0.1mm to 1mm, the TMY3s contained significant round-off
errors. In this revision, WBT went back to the raw weather reports from the National
Climatic Data Center (NCDC) and applied a procedure developed for ASHRAE RP-1477 that
checks for multiple records as well as disaggregate the rainfall to hourly values. To
winnow out spurious reports, filters were added to limit the maximum amount of rainfall
per hour, and to omit it if the dewpoint temperature dropped below freezing. Overall, the
revised rainfall correlated much better to historical averages, although there probably
remains significant undercounting in a number of locations. For example, 23 locations out
of the 1,020 (2.3%) reported no rainfall throughout the year (see
www.whiteboxtechnologies/download_TMY3_liqprec_rpt.htm for a more thorough discussion).

3) Filling in missing values. One of the main reasons to use processed weather files like
TMY3s is the absence of missing records, as often found in raw weather data. Thus, it was
surprising that the original TMY3s contained a lot of missing data, albeit almost all for
secondary variables like Liquid Precipitation, Visibility, Ceiling Height, calculated
variables like Aerosol Optical Depth and Albedo, and on one file (Mountain View CA) Wind
Direction. Although none of these instances will likely cause problems for simulations, in
the spirit of completeness all of them have been filled using either linear interpolation
for continuous values (Visibility, Ceiling Height, etc.) or a step function for discrete
values (Wind Direction).

4) Present Weather. This variable is a qualitative description of weather conditions,
e.g., clear, rain, snow, etc., generally recorded using a numerical code. Although the
TMY3s do not contain this variable, the previous TMY2s did using a 10-digit numerical
code. Although Present Weather is not used directly in any calculations, it's a useful
variable to corroborate other variables for precipitation, cloud cover, etc. For example,
in revising the Liquid Precipitation described earlier, Present Weather was used to decide
whether or not reported heavy rainfall was bogus. In this revision, an extra field has
been added to the end of each line with the international METAR 2-digit code for Present
Weather (please see Appendix C of the ASHRAE RP-1477 final report for coding).

Explanation of changes in format

The canonical form of the TMY3 files are in the *.CSV format. Since the illuminance data
revert back to the units in the TMY3 documentation, there are only two minor changes in
the TMY3.CSV format: (1) unit for Liquid precipitation depth (Variable 65) is changed from
1mm to 0.1mm, (2) a 2-digit number for Present Weather has been added at the end of each
line as Variable 69. For those familiar with ASHRAE's IWEC2 files, with these changes the
data line format of the two types of files are now exactly the same, although the header
line formats remain slightly different.
Conversion of the revised TMY3.CSV files to TMY2 or EPW should not be a problem, as long
as the change in units for Liquid Precipitation has been noted, and the illuminance units
are those in the original TMY3 documentation. Conversion of the revised TMY3.CSV files to
BINM is unnecessary since the BINM files do not include any of the changed variables.

The updated TMY3 files are available on the NREL TMY3 web site at
http://rredc.nrel.gov/solar/old_data/nsrdb/1991-2005/tmy3. The revised files are labeled
as *TYA.CSV to distinguish them from the previous *TY.CSV files, which will remain on the
Web site for another month before being taken down.

The updated TMY3 files in EPW and BINM formats are available on the WBT Weather web site
at http://weather.whiteboxtechnologies.com/wd-TMY3 or by selecting File Type "TMY3" at
http://weather.whiteboxtechnologies.com/search. The zip file names are unchanged, but the
underlying EPW and DDY files are labelled as *TYA.epw, etc., to distinguish them from the
previous versions.

Supplemental data of interest to EnergyPlus users

In addition to the weather file, EnergyPlus also needs a *.DDY file that's typically taken
from a data base of design conditions in the ASHRAE Handbook of Fundamentals (HOF). With
the proliferating availability of weather files, there are times when a location is not
listed in the HOF and the attached DDY file is blank, as is the case for 57 of the 1,020
TMY3 locations. Since WBT has weather files for thousands of stations going back to 2001,
this allowed us to calculate the design conditions for all 1,020 TMY3 files following the
ASHRAE design day procedures. To avoid confusion with the DDY files based on the HOF,
these new design files are given the *.DDY2 extension. For the 57 locations, this will
provide DDY files needed for EnergyPlus sizing calculations. For the remaining 955
stations, the DDY2 files provide an alternate set of design conditions based on more
recent data.

If you find anything unusual in the TMY3A or TMY3, please let us know!

Such a large data set makes it impossible to visually inspect all 90 million elements
(8.935 million x 10 elements per hour). In most instances, anomalies or quirks become
apparent only upon use. If anyone finds anything that seems puzzling or unreasonable,
please bring it to our attention. We’re not promising we’ll fix everything, but it’s very
useful for identifying glitches which we can then avoid by adding software filters and
barriers.

Joe Huang Manajit Sengupta
White Box Technologies National Renewable Energy Laboratory
Moraga, CA Golden, CO

Jan 14, 2015

Joe Huang
White Box Technologies, Inc.
346 Rheem Blvd., Suite 205A
Moraga CA 94556
yjhuang@xxxxxxxxxxxxxxxxxxxxxxxx
http://weather.whiteboxtechnologies.com for simulation-ready weather data
(o) (925)388-0265
(c) (510)928-2683
"building energy simulations at your fingertips"