[Xastir] Rainfall greater than 10 inches

[Tom Russo]

I just reread the 1.01 spec, and it is pretty clear that the last hour,
24 hour and rain-since midnight fields are indeed supposed to be only 3 
digits, and it's right there on page 64 of the spec, where it shows that the 
rainfall fields are exactly 4 characters wide --- one character for the "p" or 
"P" and three for the rainfall in hundredths of an inch.

Looks like the spec does NOT allow for the possibility of more than 9.99
inches in 24 hours.

I misspoke --- in my state, it would be a good *year* if we got that much rain.

So there's nothing to fix here except the spec *and* every single program
that implements that spec (that is, you can't).

When you get more than 10 inches of rain in a day, APRS is not going to 
support reporting it unless the spec is amended.

That it worked when you posted to APRS-IS directly suggests that there are
some applications that support it even though the spec doesn't.  Those
extended packets might not be properly parsed by programs that receive it,
though.  Xastir, for example, would not --- if it receives a packet with a
weather report in it, it copies out only the first three characters following
the 'p', 'P', or 'r'.  If you sent one that had 4 characters, we'd misread it.


On Thu, Mar 18, 2021 at 07:10:05PM -0600, we recorded a bogon-computron collision of the <russo at bogodyn.org> flavor, containing:
> The reason for this is that for some reason there is code in wx.c specifically
> for the Davis (in the DAVISMETEO case of the huge "switch (type)" statement
> that blindly and unconditionally terminates the wx_prec_24 and wx_prec_00 
> strings at the third character without considering whether it's legitimately a 
> 4-character string.
> 
> This code needs to be reviewed and probably fixed to be less hamfisted about
> terminating the string like that.
> 
> The code looks like this:
>       if ((temp_conv=strchr((char *)data,'p')))   // Rain per 24 hrs/total
>       {
>         xastir_snprintf(weather->wx_prec_24,
>                         sizeof(weather->wx_prec_24),
>                         "%s",
>                         temp_conv+1);
>         weather->wx_prec_24[3] = '\0';
>       }
> 
>       if ((temp_conv=strchr((char *)data,'P')))   // Rain since midnight
>       {
>         xastir_snprintf(weather->wx_prec_00,
>                         sizeof(weather->wx_prec_00),
>                         "%s",
>                         temp_conv+1);
>         weather->wx_prec_00[3] = '\0';
>       }
> 
>       if ((temp_conv=strchr((char *)data,'T')))   // Total Rain since
>       {
>         // wx station reset
>         xastir_snprintf(weather->wx_rain_total,
>                         sizeof(weather->wx_rain_total),
>                         "%s",
>                         temp_conv+1);
>         weather->wx_rain_total[4] = '\0';
>       }
> 
> So after the string following 'p', 'P', or 'T' is copied into the appropriate 
> 10-character buffer in the WeatherRow structure, regardless of how long it 
> actually is it's truncated at 3 (for the 24-hour and rain-since-midnight) or 
> 4 (for total rain since station reset).  That is clearly not right.
> 
> Looking back over git history, it appears that the DAVISMETEO code has *never* 
> been correct for 4 characters rain output.  It has always either tried to copy 
> out only 3 characters *OR* copied out whole chunks of data then truncated to 3 
> characters.  The DAVISMETEO code was introduced in commit 885fad4 in 2004,
> and was storing only the total rain --- and only 3 digits of that.  It was
> modified in commit 2868e0b (also in 2004) to switch how it does the copy,
> but it was still wrong.  All subsequent "fixes" to the DAVISMETEO code 
> preserved this basic error even while fixing other problems (such as storing 
> stuff like the last 24 hour rain as total rain, or letting valid station 
> information be overwritten by subsequent calculation that assumes we didn't
> *get* valid station information).
> 
> The problem here is specific to the Davis/Lacrosse/wxnow code, which
> is parsing out pre-formatted APRS-style weather reports, as opposed to 
> actual numerical values of each type of field.
> 
> We're trying to copy out small bits of a larger string, and not checking 
> carefully for when the field really ends (that is, when we hit the end of the 
> string *or* a non-digit) --- we're simply assuming a field length that is
> incorrect.
> 
> It is not likely that I will have time to review this code anytime in the next
> week or so.  If one of the other developers cares to have a crack at it, please
> do.  What I envision is that instead of blindly clobbering the 4th character
> by writing a null into the array at position [3], we'd check that 4th character
> to see if it's already null (meaning we copied the end of the larger string)
> or if not if it's an ascii digit (in which case we copied a 4-digit number
> rather than a 3-digit, so should put the null at the FIFTH character).
> 
> I guess nobody has ever noticed this issue after such an enormous dump of 
> rain (I certainly never would have, as I'd rarely see that much rain in a 
> *month* much less 24 hours where I live).
> 
> On Fri, Mar 19, 2021 at 11:34:17AM +1100, we recorded a bogon-computron collision of the <aprs at exemail.com.au> flavor, containing:
> > I don't know if this question belongs here, or on the aprs sig. Please send
> > me away if I'm in the wrong place.
> > 
> > I use xastir as my aprs client and I have modified db.c so that 24 hour rain
> > is displayed on the map, along with the usual wx parameters. I've been doing
> > this for about 12 years and all has been fine, until this morning when 267mm
> > of rain displayed as 26.7mm. The problem happened overnight when the
> > rainfall figure exceeded 254mm, or 10 inches, when p000 became 4 digits. I
> > convert mm to hundredths of an inch in a script prior to using the xastir
> > udp client to upload the data. This is a test packet for more than 10" of
> > rain ..
> > 
> > /usr/local/bin/xastir_udp_client localhost 2023 vk2tv-1 xxxxx
> > "VK2TV-ZZ>APRS:=3102.12S/15252.72E_156/016g019t070r003p1752P123h96b10194
> > Davis VP2"
> > 
> > p1752 equates to 445mm which xastir displays on the localhost as 44.5mm
> > 
> > I then configured the script to directly access aunz.aprs2.net ....
> > 
> > /usr/local/bin/xastir_udp_client aunz.aprs2.net vk2tv-1 xxxxx -to_inet
> > "VK2TV-ZZ>APRS:=3102.12S/15252.72E_156/016g019t070r003p1752P123h96b10194
> > Davis VP2"
> > 
> > The data was received on my main xastir installation, with its own
> > connection to aunz.aprs2.net and displayed as 44.5mm, not 445mm. I seem to
> > have convinced myself that by sending via aprs-is the 4 digit rainfall
> > number is not an issue, or is my thinking flawed?
> > 
> > The aprs spec 1.01 provides for 4 bytes which allows for considerably more
> > than 10" of rain, or am I misunderstanding something, because a formatted wx
> > packet string for 24 hour rain is shown in various examples as p000?
> > 
> > Having seemingly convinced myself that the sent packet is correctly
> > formatted I started pointing my injured finger (sanding discs aren't
> > forgiving!) and questioned if xastir is truncating the result for the
> > display. This is how I amend db.c to display rainfall
> > 
> > if (strlen(weather->wx_prec_24) > 0) {
> > ???????????????????????????? xastir_snprintf(wx_tm, sizeof(wx_tm), "
> > R:%.1fmm", atof(weather->wx_prec_24)*0.254);
> > ???????????????????????????? strncat(temp_wx_temp,
> > ???????????????????????????????????? wx_tm,
> > ???????????????????????????????????? sizeof(temp_wx_temp) - 1 - strlen(temp_wx_temp));
> > ?????????????????????? }
> > 
> > I am NOT a programmer. I'm not afraid to try simple things, but I'm not a
> > programmer.
> > 
> > Would somebody please care to comment on my problem, and clarify if
> > misunderstandings on my part are responsible? We don't get 10" of rain very
> > often but I would like to resolve this for just in case!
> > 
> > Ray vk2tv
> > 
> > _______________________________________________
> > Xastir mailing list
> > Xastir at lists.xastir.org
> > http://xastir.org/mailman/listinfo/xastir
> 
> -- 
> Tom Russo    KM5VY
> Tijeras, NM  
> 
>  echo "prpv_a'rfg_cnf_har_cvcr" | sed -e 's/_/ /g' | tr [a-m][n-z] [n-z][a-m]
> 
> _______________________________________________
> Xastir mailing list
> Xastir at lists.xastir.org
> http://xastir.org/mailman/listinfo/xastir

-- 
Tom Russo    KM5VY
Tijeras, NM  

 echo "prpv_a'rfg_cnf_har_cvcr" | sed -e 's/_/ /g' | tr [a-m][n-z] [n-z][a-m]