Karl Uppiano

1-Wire Weather Service for Java Tech Notes

Home
The Weather Station
Java Weather Software
Software Tech Notes
Reliable Wind Speed Data
Science vs. Religion
My Resume

Tech Notes for 1-Wire Weather Service for Java -- Advanced Users

Problems Detecting the Default 1-Wire Net
Dallas Semiconductor 1-Wire API 1.11_PreRelease_20071108 fails to pick up the setting stored by the Default 1-Wire Net application. WxService will not start, and you will see an entry similar to the following in the log file:

Jan 30, 2009 11:22:23 PM wxservice.Service initOneWire
INFO: Dallas Semiconductor 1-Wire API 1.11_PreRelease_20071108.
Jan 30, 2009 11:22:24 PM wxservice.WxWeb main
SEVERE: Specified adapter name "DS9097U" is not known

The solution is to place a file named onewire.properties in the WxService directory, and add the lines as shown in the following examples:
 
For the 1-Wire USB adapter on USB port 1:
onewire.adapter.default={DS9490}
onewire.port.default=USB1
For the 1-Wire COM adapter on COM port 1:
onewire.adapter.default={DS9097U}
onewire.port.default=COM1
For the 1-Wire COM adapter on COM port 1, etc, using the 64-bit 1-Wire drivers:
onewire.adapter.default={DS9097U_DS9480}
onewire.port.default=COM1
To determine your actual settings, you should note the settings in the Default 1-Wire Net application, and enter them into the onewire.properties file. Be sure to include the curly braces around the adapter name.
 
Dallas Semiconductor 1-Wire API 1.11_PreRelease_20071108 is required for 64-bit support.

Changing WxService Settings
You can edit configuration properties using the Configuration page on WxMonitor or by directly editing wxservice.properties in the wxservice directory.
 
If you make a mistake, you can delete an entry from the Configuration page, and it will revert to the default value when you click "Apply". Make a backup copy of wxservice.properties if you want to revert to previously saved settings.

Changing Units (Metric, English, etc.)
WxService and WxMonitor will switch automatically to the correct units for your locale. For example when you install Java in Canada, the installer will set your locale to en_CA or fr_CA. If you want to force metric in a locale that doesn't normally use it (primarily the USA), you need to change a few things.
 
The simplest way to do that is to add the following to the shortcut that starts WxMonitor to a country that uses metric. For example, right click the shortcut, and select Properties. Add
-Duser.country=CA
just after javaw.exe. This will tell WxMonitor that you are in Canada, a country that uses metric.
 
It is a little trickier for WxService, because you will need to edit the specific properties to switch from .miles. or .english. to .kilos. or .metric. Editing the properties gives you the most fine-grained control, because you can specify the units for each device. For the anemometer, metric might mean meters per second or kilometers per second. For example, to make the CSV formatter log in metric,
  • change csv.formatter.air.pressure.data from barometer.english.data to barometer.metric.data
  • change csv.formatter.dewpoint.data from hygrometer.english.data  to hygrometer.metric.data
  • change csv.formatter.temperature.data from thermometer.english.data to thermometer.metric.data 
  • change csv.formatter.wind.speed.data from anemometer.miles.data to anemometer.meters.data 
and so on.
 
For precipitation, change the multipliers, e.g.,
precipitation.rate.csv.data.scale=0.01
precipitation.total.csv.data.scale=0.01
to
precipitation.rate.csv.data.scale=254e-3
precipitation.total.csv.data.scale=254e-3
Advanced topic: note that you can create your own completely custom units, by creating additional types, such as anemometer.revs.data (which I use to determine the actual anemometer RPM), and putting in your own conversion constants.

Setting Event Logging Levels
The 1-Wire Weather Service for Java uses the extremely flexible java.util.logging.Logger package. The default logging verbosity is INFO. You modify logging features by editing logging.properties in the lib directory of your Java Runtime Environment (JRE). The following example sets up the logger to log only SEVERE errors for the entire service, except for the anemometer task, which overrides the general settings and logs ALL events.
wxservice.level = SEVERE
wxservice.AnemometerTask.level = ALL
You can set logging levels individually for any class in the 1-Wire Weather Service for Java; each class has its own logger name associated with it. Logging levels from highest to lowest, are SEVERE, WARNING, INFO, CONFIG, FINE, FINER, FINEST, ALL.

Error Messages Reference
This tech note describes the error messages that you may find in the WxService system log, and provides suggestions for fixing the problem. The values in curly braces are placeholders for replaceable parameters. They will be replaced with actual values in the system log.
  • SEVERE log entries usually indicate that the system or a sensor cannot continue.
  • WARNING log entries usually indicate a problem if it persists, but sporadic warnings are usually not serious.
  • INFO log entries and lower (even if they use the error messages below) are purely informational.
Invalid administrator key
User attempted to configure the server without setting the correct administrator key. See "Configure Authentication", below for instructions on how to configure authentication.

Counter reset to {0}
The precipitation sensor counter should always count up. This error indicates that the current value is less than the previous value. This may happen if the sensor loses power and the backup battery is dead.

Device not found: Address = {0}, file = {1}, family = {2}
A search for a 1-Wire device with the specified address, device file or device family failed. This may happen if the device address or the device file are in error, or the device is not connected, or if there is interference on the 1-Wire bus. The barometer task uses the device file name to identify the 1-Wire switches inside the AAG Pressure Sensor.
 
Device not found: Address = {0}, family = {1}
A search for a 1-Wire device with the specified address or device family failed. This may happen if the device address is in error, or the device is not connected, or if there is interference on the 1-Wire bus. All device tasks use the device family or device address to locate devices on the 1-Wire bus.

Abnormal operation completion status
The barometer task logs this error if the pressure sensor does not report the expected value at the end of a command. If this error persists, it indicates a device malfunction, or persistent interference on the 1-Wire bus.
 
Data conversion timeout
The barometer task logs this error if the A/D conversion does not complete within the expected time. If this error persists, it indicates a device malfunction, or persistent interference on the 1-Wire bus.

Could not connect to any host in the rotation list
The APRS formatter logs this error if it cannot connect to any of the hosts in the host rotation aprs.formatter.address configuration setting. If this error persists, check your network and Internet connection. If they're working properly, check CWOP News for server issues.
 
Timing tolerance error: {0} = {1, number}
The anemometer task logs this error when the time between samples exceeds plus or minus 20% of the anemometer.task.sampling.interval configuration setting. This may occur if the computer becomes extremely busy, and cannot read the device at the specified intervals, or if the system clock changes abruptly (syncs with a network time server, or user sets the system clock). See "Enable Incremental Time Sync" below.

Access tolerance error: {0} = {1, number}
The anemometer task logs this error when the time to access the anemometer spin counter exceeds 250 milliseconds. This may occur if the computer becomes extremely busy while attempting to access the device.
 
Undefined pattern
The wind vane task logs this error when the A/D converters return values that it cannot resolve into a directional heading. This occurs if the wind vane changes direction rapidly while the device is being read. This is not usually a serious error unless happens more than a few times a day. Gusty winds can aggravate the problem. Interference from the anemometer magnets can cause this. Refer to your weather hardware installation guide for adjustment instructions.

Data value error: {0} = {1, number}
Various sensor tasks log this error if a value is outside the expected range. The error format will be description = value to explain the error. If this error persists for a given device, it might be out of calibration, or the device might be malfunctioning.
 
Data value error: {0} = {1, number}, {2} = {3, number}
Various sensor tasks log this error if a value is outside the expected range. The error format will be description = value, description = value to explain the error. If this error persists for a given device, it might be out of calibration, or the device might be malfunctioning.

1-Wire errors and CRC errors
These errors indicate problems with the 1-Wire bus. This usually indicates interference on the 1-Wire bus, or wiring problems. Refer to the Dallas tech note Guidelines for Reliable 1-Wire Networks for detailed troubleshooting tips.

Configure Authentication
To configure WxService remotely, you need an administrator key. The default key is "default", which enables remote configuration out-of-the-box. If you expose WxService as a web service, you may wish to change the default. You cannot change the administrator key remotely. You will need to edit wxservice.properties, changing
# Administrator key for setting properties remotely.
wxservice.administrator.key=default
to the desired key. Note that your custom key will not be displayed in WxMonitor or published in the web service API. It will always show up as "default". You can configure each individual WxMonitor site to remember the key by adding the key locally to wxmonitor.properties. The key is not encrypted. The design goal was to prevent someone from inadvertently changing the settings. Someone sniffing the communication channel might discover the administrator key. You can discourage dictionary attacks by choosing a long random key. There is no specified key length limit.

Calibrating the Humidity Sensor
Note: A hygrometer auto-ranging feature was added in ow4j090107, which may reduce the need for manually calibrating the hygrometer. The TAI8540A 1-Wire Humidity Module frequently seems to indicate too high when calibrated using "typical" values for slope and zero offset for the Honeywell HIH-family of humidity sensors. The auto-ranging feature detects readings that exceed 100% and adjusts the slope to compensate. Similarly, readings less than zero cause an adjustment of the zero offset. The adjustements remain in effect until WxService is restarted. WxService doesn't modify the configuration, but it writes the auto-ranging corrections to the daily log, so that you can enter the values in the configuration manually if desired.

If the humidity sensor is reading out of range, first determine, if possible, how it compares to other sensors in the area. There will be three cases:

Case 1: Offset error -- the humidity range is acceptable, but the average humidity consistently reads above or below reference instruments. To calculate a new Vzero, enter the currently configured values for Slope and Vzero (hygrometer.task.sensor.slope and hygrometer.task.sensor.zero, respectively, in the WxMonitor Configure panel), into this formula:

Vzero = (Slope * SensorRH) + Vzero - (Slope * ReferenceRH)

where ReferenceRH is the actual humidity from the reference instruments. Enter the new calculated value for Vzero into hygrometer.task.sensor.zero and click Apply.

Case 2: Range error -- the humidity range is too wide or too narrow, compared to reference instruments. To calculate a new slope, enter the currently configured value for Slope (hygrometer.task.sensor.slope in the WxMonitor Configure panel). into this formula

Slope = Slope * SensorRH / ReferenceRH

where ReferenceRH is the actual humidity from the reference instruments. Enter the new calculated value for Slope into hygrometer.task.sensor.slope and click Apply.

Case 3: Range and offset error -- a combination of the other two cases. Fix range and offset separately, as described above. Correct the raw SensorRH data for offset and then plug the new values for SensorRH, and Vzero from Case 1, into the formulas for Case 2. There may be some interaction between Case 1 and Case 2, so this might require a few iterations to dial it in.

Note that you can use historical data from logs to perform the correction as long as SensorRH and ReferenceRH are taken at the same time.

Disambiguating Multiple Devices of the Same 1-Wire Family
If you have more than one sensor from the same 1-Wire family on your 1-Wire network (multiple temperature sensors for example, or more insidiously, an anemometer and a rain gauge -- they both use the same Dallas DS2423 IC), the device that gets associated with a given WxService task-name will be arbitrary unless you change the task-name.task.device.address configuration from "discover" to the 1-Wire address for each particular device from the same family. If you don't know the address (sometimes it is printed on the device), you can use the 1-Wire Viewer utility to find the device address.

Routing Devices on the 1-Wire Network
WxService can route alternate devices to various displays or formatters. For example, the TAI8540A Humidity Sensor Module is built around a Dallas DS2438 Smart Battery Monitor IC, which contains a temperature sensor that is normally used for hygrometer temperature compensation and dew point calculations. However, you can also use the hygrometer's temperature sensor instead of the one provided in the TAI8515 1-Wire Weather Instrument Kit (which is adversely affected by solar heating) by changing the settings

wxmonitor.temperature.task=thermometer.task
csv.formatter.temperature.task=thermometer.task
wunderground.formatter.temperature.task=thermometer.task

to

wxmonitor.temperature.task=hygrometer.task
csv.formatter.temperature.task=hygrometer.task
wunderground.formatter.temperature.task=hygrometer.task

This does not affect any of the hygrometer functionality, it simply makes WxMonitor and the formatters use the temperature information that the humidity module is transmitting. Similarly, you can route other devices as well.

Using Multiple Devices of the Same 1-Wire Family
You might have multiple devices from the same 1-Wire familiy on your 1-Wire network, e.g., indoor and outdoor thermometers. You can use these devices simultaneously, with WxService. This tech note describes how to use multiple thermometers, but the same principles apply to other devices as well.
You need to create a new task for your second (or third, or n-th) thermometer. You need to add the following properties to the WxService configuration:

thermometer1.task.classname=wxservice.sensor.task.Thermometer
thermometer1.task.device.address=discover
thermometer1.task.device.offset=0.0
thermometer1.task.device.scale=1.0
thermometer1.task.sampling.interval=60000

This creates a task called "thermometer1.task". The name isn't important. You could name it "indoor.task" or "outdoor.task" if you want. You can add the properties with the Add... button on the WxMonitor Configuration page, or you can paste them into wxservice.properties anywhere in the file. Don't forget to change thermometer1.task.device.address=discover to the actual device address if you need to disambiguate the devices.

Then, add thermometer1.task to the other tasks in the WxService task list:

wxservice.sensor.task.names=...;thermometer.task;thermometer1.task;

Notice that you now have a thermometer.task and a thermometer1.task, so you really have two thermometer tasks at this point.

Finally, change settings for the various formatters and WxMonitor to use the thermometer.task or the thermometer1.task as described above in the tech note "Routing Devices on the 1-Wire Network".

Resetting Your Rain Gauge
Sometimes you have to take your rain gauge offline for repairs or maintenance. This can cause the counter to advance by a large amount. Replacing the counter with one that has a much higher count can have a similar effect. To avoid seeing insanely high precipitation readings, you should reset your counter or clear WxService precipitation records before placing the rain gauge back in service. Either one is adequate; you don't need to do both.
 
To reset the counter: Rain gauge counters have battery backup, so the device does not reset during a power failure. Some counters have a jumper to disconnect the battery; others do not. If yours has a jumper, disconnect the jumper for several seconds to allow the device to reset, then reconnect the jumper. If yours does not have a jumper, you must clear WxService precipitation records.
 
To clear WxService precipitation records: Delete files (if present) named precipitation.total.csv.data, precipitation.total.aprs.data and precipitation.total.wunderground.data from the wxservice directory. Also delete precipitation.total.english.data or precipitation.total.metric.data from your properties and settings directory (user home directory on *nix systems) for each WxMonitor user.

Enable Incremental Time Sync
Please refer to my article Obtaining Reliable 1-Wire Wind Speed Data to understand the motivation for this tech note. If you are running a stand-alone version of Windows (without a domain server), Date and Time Properties will have an Internet Time tab, which allows you to "automatically synchronize with an internet time server". This configures the w32time to synchronize once a week with a time server you specify, and it makes stepwise corrections to the system time. This sudden and potentially large discontinuity in system time causes serious problems with critical timing applications such as the WxService anemometer.
 
There is a better way. From the command line, type:
net time /setsntp:server-name
where server-name is the sntp server you want to synchronize with, for example,
net time /setsntp:us.pool.ntp.org
This configuration method does two things: Unless the system time is seriously out of sync, it doesn't make a stepwise correction, but rather it simply adjusts the system clock speed slightly, to allow it to catch up with the time server, or vice-versa. And, it determines how frequently to synchronize based on the amount of correction needed. Thus, it can dial in the system clock rate, so as the system clock becomes more accurate, the need for synchronization become less frequent.

Why Doesn't the Log File Name Match the Date?
Sometimes the log file list bar in the WxMonitor Logs panel looks inconsistent, e.g., Fri Jun 04 08:51:41 PDT 2010 [2010-06-03.txt]. The log file name, e.g., 2010-06-03.txt reflects the date the log file was created. The log file date, e.g., Fri Jun 04 08:51:41 PDT 2010, is the latest file modification date as returned by the operating system. The logger flushes the file output stream every time it logs an event, but this does not guarantee that data in the operating system disk buffer is flushed to the disk. The modified date is updated when the file is actually modified on disk. It is up to the operating system to determine when to flush the data to disk. 

If errors are few and far between, several days may pass between logged errors, so event data may remain in the disk buffer until the file is closed, and a new one created for an event on a new day.
 
The CSV files don't exhibit this effect because CSV events are recorded every ten minutes (by default), and the file is opened and closed each time. The event logger uses java.util.logging, for which the normal behavior is to keep the log file open.

Using the WxService as a Web Service
WxService is implemented as a web service. This tech note documents the web service operations, and how to use them.

WxMonitor is a web service client of WxService, but there is nothing special about that. You can write other clients by importing the WxService Web Service Description Language (WSDL) and the schema into any software development tool that is capable of consuming web services. SoapUI is a good tool to use to experiment with the web service and explore the XML documents used to exchange data between WxService and its clients. See the WxService JavaDocs (included in the WxService zip file, available for download) for more information about how WxService works.

getSensorData -- Returns sensor data since the specified time (in milliseconds since January 1, 1970 UTC). WxService has a short history buffer (approximately 10 minutes, to cover for temporary network interruptions). Normally, you would specify the time of the last sensor data received from the last getSensorData invocation. When first starting up, specify a time of zero. This causes WxService to send the minimum and maximum data and time (since local midnight) for each sensor, plus all data in the history buffer.

Sample XML Request

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:xmlns:uppiano-karl:wxservice">
   <soapenv:Header/>
   <soapenv:Body>
      <urn:getSensorData>
         <time>1195791936109</time>
      </urn:getSensorData>
   </soapenv:Body>
</soapenv:Envelope>

Sample XML Response

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:ns1="urn:xmlns:uppiano-karl:wxservice">
   <soapenv:Body>
      <ns1:getSensorDataResponse>
         <sensorData>
            <name>windvane.task.direction</name>
            <value>14</value>
            <time>1195865915156</time>
         </sensorData>
         ...
         <sensorData>
            <name>anemometer.task.speed</name>
            <value>10</value>
            <time>1195866138109</time>
         </sensorData>
      </ns1:getSensorDataResponse>
   </soapenv:Body>
</soapenv:Envelope>

getFileNames -- Returns a comma-delimited list of files and file creation times (in milliseconds since January 1, 1970 UTC) of log files and CSV history files stored on the weather server. File names selected from this list can be used to get file data. Use only the file name portion of the response as an input parameter to getFileData.

Sample XML Request

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:xmlns:uppiano-karl:wxservice">
   <soapenv:Header/>
   <soapenv:Body>
      <urn:getFileNames/>
   </soapenv:Body>
</soapenv:Envelope>

Sample XML Response

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:ns1="urn:xmlns:uppiano-karl:wxservice">
   <soapenv:Body>
      <ns1:getFileNamesResponse>
         <fileName>2005-08-23.csv,1124866200031</fileName>
         ...
         <fileName>2007-11-23.txt,1195864819734</fileName>
      </ns1:getFileNamesResponse>
   </soapenv:Body>
</soapenv:Envelope>

getFileData -- Returns data from the specified file (log file or CSV history data). The file name is obtained from the method getFileNames.

Sample XML Request

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:xmlns:uppiano-karl:wxservice">
   <soapenv:Header/>
   <soapenv:Body>
      <urn:getFileData>
         <fileName>2007-11-22.csv</fileName>
      </urn:getFileData>
   </soapenv:Body>
</soapenv:Envelope>

Sample XML Response

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:ns1="urn:xmlns:uppiano-karl:wxservice">
   <soapenv:Body>
      <ns1:getFileDataResponse>
         <fileData>time, temperature, dew_point, humidity, wind_speed, wind_gust, wind_direction, air_pressure, precip_accum, precip_rate,</fileData>
         <fileData>00:00:00,031.3,024.3,075,002,004,022,30.46,0.00,0.00,</fileData>
         ...
         <fileData>00:10:00,031.8,023.9,072,002,005,044,30.46,0.00,0.00,</fileData>
      </ns1:getFileDataResponse>
   </soapenv:Body>
</soapenv:Envelope>

getConfiguration -- Returns a list of name/value pairs that represents the current WxService configuration.

Sample XML Request

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:xmlns:uppiano-karl:wxservice">
   <soapenv:Header/>
   <soapenv:Body>
      <urn:getConfiguration/>
   </soapenv:Body>
</soapenv:Envelope>

Sample XML Response

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:ns1="urn:xmlns:uppiano-karl:wxservice">
   <soapenv:Body>
      <ns1:getConfigurationResponse>
         <config>
            <key>wxmonitor.wind.direction.task</key>
            <value>windvane.task</value>
         </config>        
         ...         
         <config>
            <key>thermometer.task.sampling.interval</key>
            <value>60000</value>
         </config>
      </ns1:getConfigurationResponse>
   </soapenv:Body>
</soapenv:Envelope>

setConfiguration -- Sets zero or more specified name/value pairs on WxService and restarts the service.

Sample XML Request

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:xmlns:uppiano-karl:wxservice">
   <soapenv:Header/>
   <soapenv:Body>
      <urn:setConfiguration>
         <config>
            <key>wxmonitor.wind.direction.task</key>
            <value>windvane.task</value>
         </config>        
         ...         
         <config>
            <key>thermometer.task.sampling.interval</key>
            <value>60000</value>
         </config>
      </urn:setConfiguration>
   </soapenv:Body>
</soapenv:Envelope>

Sample XML Response

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:ns1="urn:xmlns:uppiano-karl:wxservice">
   <soapenv:Body>
      <ns1:setConfigurationResponse/>
   </soapenv:Body>
</soapenv:Envelope>