Sign inRegisterCheck connection

Daily Averages and Extremes from Weather Display Reports

How to extract Weather Display daily means and extremes into structured tables, including day boundaries, quality flags, and why HTML extractors break.

Back to Weather Display software guides

A daily climate row is a contract: a mean, a maximum, a minimum, a rain total, and the clock that defined the day. Weather Display can emit that contract as an HTML daily report. Historical TNET tooling then tried to lift the numbers out of the HTML and restyle them as tables. The recovered /tws-avgext-script.php page was a download-and-configure tutorial for that extractor. The unsupported PHP is not republished here.

What remains useful is the science. Mean is not extreme. Midnight-to-midnight is not the only day. A missing hour is not a zero. HTML is a brittle schema. A structured table is the point.

A worked example of how those extracted rows were presented—without treating the old script as a product—is the sample daily-report interpretation. Read it as a historical table layout, not as a live Mesa climate service.

What the HTML report was

Weather Display can generate a month-to-date averages and extremes page, typically named for the month (for example April2009.htm). The page is observed station history as WD computed it: daily means, daily extrema with times, rain totals, and short-period rain rates.

That file is not a National Weather Service Climate Report or a WMO CLIMAT message. It is the station software's own reduction of its archive. Official daily climate products use their own day boundaries and completeness rules; the neighboring NOAA-style daily page in this cluster is that other family. Do not merge the two tables without labeling the source.

Mean versus extreme

A daily extreme is the highest or lowest valid sample in the day window, with the time it occurred if the logger kept it. Maximum temperature and minimum temperature are extremes. Maximum gust is an extreme. They are not averages. If the window missed the true afternoon peak because the logger was down, the recorded maximum is the maximum of the available samples, not the meteorological maximum of the atmosphere.

A daily mean is a reduction of the same window. There is more than one honest mean:

  • The arithmetic mean of the archived samples (whatever the logger interval was).
  • The mean of the daily maximum and daily minimum, which WMO notes is a method in operational use for daily mean temperature in some national practices.
  • A mean of selected hours (for example three- or four-times-daily readings on a former manual schedule).

WMO Guidelines on the Calculation of Climate Normals (WMO-No. 1203) states that the definition of the observation day, and the way daily mean temperature is calculated, should follow national standards and be documented in metadata. Different methods are in operational use. A Weather Display "Average temperature" line is WD's method applied to WD's archive. It is comparable to another WD station only if both used the same archive interval, the same day boundary, and the same completeness. It is not automatically comparable to a national climate normal.

Rain is neither a mean nor a simple extreme. Daily rainfall is a sum of tips or of depth increments in the window. "Maximum rain per minute" is an extreme of a short-period rate. Month and year rainfall are running sums, not daily means. Mixing them in one visual column without a header is how a table starts lying.

Wind direction as an "average" is a further special case. A scalar mean of compass degrees will treat 350° and 10° as if the mean were south. A vector mean is the defensible average. If a report prints a single degree figure, say which method produced it.

Day boundaries

"Daily" is a closed time interval. Common choices:

  • Local midnight to local midnight. Typical for personal Weather Display reports when the PC clock is in local time.
  • UTC midnight to UTC midnight. Common in automated national networks and in some published gridded daily products.
  • A climatological day that is not midnight. Some national services credit maxima and minima to a 24-hour window that ends in the morning, so the night minimum is not split across two calendar dates.

If the station PC is in local time and the report is named December2007.htm, the usual reading is local civil days for that month. If the PC clock was wrong, or the operator switched time zones, the HTML still looks fine and the table is still wrong. Always carry the time zone and the window in the structured output, not only the day number.

Incomplete days need a flag. A day that starts at installation time, a day that ends at a logger outage, and a month-to-date row that is not yet a closed month are different objects. Plotting an incomplete day as a full daily mean, without a note, biases a month.

Quality flags the HTML usually forgot

A structured extract should add flags the pretty HTML omitted.

Missing. No valid samples, or too few to compute a mean. Do not emit 0 °F or 0 mm as a stand-in. WMO-No. 1203 and national practice (for example Canada's "3/5" rule for monthly temperature means) refuse a monthly mean when too many daily values are missing. A hobby table can use a simpler rule, but it must show it.

Suspect. A range or step failure (implausible temperature jump, humidity outside 0–100%). Keep the number if you keep it, but flag it.

Accumulated or estimated. Rain recovered after a clog, or a mean after gap-filling. Derived, not observed.

Extreme time missing. Keep the value; do not invent a clock.

Unit. Store in. versus mm, °F versus °C, inHg versus hPa as columns. The historical extractor had to hunt English phrases ("in. on day" versus "mm on day") because the HTML was the schema.

The archive sample page is a historical snapshot of one English WD report, not a current observation and not a universal unit system.

Why HTML extractors fail when WD output changes

The old approach searched the daily-report HTML for English phrases ("Average temperature", "Maximum gust speed") and then counted words to the number. That binds the table to:

  • language (a Spanish or German WD report will not match);
  • wording (a WD update that inserts a word, a unit, or a line break shifts the count);
  • layout (CSS or table markup that splits a value across tags);
  • the assumption that the first match is the daily value rather than a repeated month-to-date echo.

Those are schema breaks. They are the same family of breaks as web-tag tokens going stale, except HTML scraping is even less stable because chrome and fields share one document.

The durable alternatives are, in order of preference:

  1. Read the archive or ClientRaw daily files with a versioned field map, not the HTML.
  2. If HTML is all you have, parse with a versioned grammar: WD build, language, unit system, and a fixture file for that month.
  3. When the grammar fails, fail the row. Do not guess.

Do not "fix" a broken extractor by pasting the old PHP onto a modern PHP runtime. That code is unsupported, was written for a specific English HTML dialect, and is not provided here.

Building a structured table

A useful extract has one row per closed day plus optional running month/year rows. Carry station ID, time zone, day window, sample counts, completeness flag, means, extrema with times, rain sums and units, source file, and WD build if known.

That table is historical observed data when it comes from closed days in the archive. A month-to-date row that is still accumulating is a running summary, not a closed climate value. A forecast does not belong in this table at all.

Practical checklist

  1. Name the day boundary and time zone before you name the mean.
  2. Store units in columns; do not infer them from English fragments.
  3. Flag missing and incomplete days; never coerce missing to zero.
  4. Keep extreme times with extreme values.
  5. Version the extractor against a saved HTML fixture whenever WD or language changes.
  6. Prefer ClientRaw or archive files when they exist.
  7. Use the sample page above for the historical layout, not a resurrected script.

The Weather Display hub collects the publishing tools. Definitions, completeness, and provenance of measurements are the same issues TNET states publicly on data sources, quality controls, and methodology.

Sources