[Xastir] RAC Database

[Tom Russo]

On Fri, May 18, 2012 at 04:07:18PM +0000, we recorded a bogon-computron collision of the <jbrower at meteorchaser.net> flavor, containing:
> Hi Tom and group,
> 
> Unfortunately I don't know perl Tom. I did check out the database format and it looks like Industry Canada added additional/auxiliary club information to the call signs data.  The new format is as below. I think if the script could read up to and including character 209 of the string then things should work with xastir's look up. Is there an easy parsing to clean up this new format?

The club stuff isn't the problem, because Xastir is already expecting that
data.  It looks like perhaps an extra license level field was added 
(Basic with Honors).  Here's what Xastir thinks the file
format is:

typedef struct {
    char callsign[7];
    char first_name[36];
    char last_name[36];
    char address[71];
    char city[36];
    char province[3];
    char postal_code[11];
    char qual_a[2];
    char qual_b[2];
    char qual_c[2];
    char qual_d[2];
    char club_name[142];
    char club_address[71];
    char club_city[36];
    char club_province[3];
    char club_postal_code[10];
    char crlf[2];
    char filler[8]; // To prevent overruns
} rac_record;


What Xastir does when it finds an RAC database is first create an index
file, where it records the call sign and file position for every 100th call.
This index building is conservative about how many characters it will allow
on a line, and reads the full line, ignoring everything but the call sign.
As far as I can tell, this probably works even with the new format (I do not
have a recent RAC database, so can't check --- but what you should find in
your .xastir/data/AMACALL.ndx file should be a list of 6-character call signs
and a number after it).

Once the index is made, a call sign lookup is done by finding the closest
call (by lexographic compare) in the index, retrieving the file offset of that
call from the index, seeking to that point in the main AMACALL file, and doing
a linear search from there, line by line.

The problem is, I think, that during the lookup Xastir is doing an "fgets"
with the number of characters in the rac_record structure as the maximum 
character count.  fgets will read either a line of data *OR* that maximum number
of characters, whichever is smaller. It does not say "read the entire line" 
and then just ignore the rest.  So if the record in the file is longer than the
rac_record (by more than 8 characters, because of that "filler" thing), then
subsequent reads after the first may be starting from somewhere other than
the beginning of a line.

I haven't had time to look in more detail than that, but it may be possible to
replace all the parts of the rac_record structure other than the bits we
really need to parse out with a larger-than-necessary filler block, and let
fgets get the entire line every single call, parsing out only those early parts
of the record that haven't changed.  Since the character count passed to
fgets means "read the entire line if possible, but no more than this many 
characters" that should be a safe thing to do. As far as I can tell, none
of the fields beyond the license level fields are even used by Xastir, so 
all those fields could be stuffed into "filler" instead, and filler expanded
a little to assure that no buffer overruns happen, and to assure
that the whole line is always read, no matter how long it is.

Simply changing the record to match today's AMACALL format might not be
the best solution, because then it will break everyone who hasn't updated
the directory in a while.  The ideal solution will be one that is both 
correct today *and* supports older format data, and that may very well not
break with tomorrow's data format if they just add more data to the end.

> Jeff VA7TZ
> 
>     Each record is in the following format:
> 
>       FIELD                        STARTING COLUMN   LENGTH
>         ~~~~~                        ~~~~~~~~~~~~~~~   ~~~~~~
>       Callsign                               1            6
>       Given Names                            8           35
>       Surname                               44           35
>       Street Address                        80           70
>       City                                 151           35
>       Province                             187            2
>       Postal/ZIP Code                      190           10
>       BASIC Qualification (A)              201            1
>       5WPM Qualification (B)               203            1
>       12WPM Qualification (C)              205            1
>       ADVANCED Qualification (D)           207            1
>       Basic with Honours (E)               209            1 <--- End here?
>       Club Name (field 1)                  211           70
>       Club Name (field 2)                  282           70
>       Club Address                         353           70
>       Club City                            424           35
>       Club Province                        460            2
>       Club Postal/ZIP Code                 463            7
-- 
Tom Russo    KM5VY   SAR502   DM64ux          http://www.swcp.com/~russo/
Tijeras, NM  QRPL#1592 K2#398  SOC#236        http://kevan.org/brain.cgi?DDTNM
"And, isn't sanity really just a one-trick pony anyway? I mean all you get is
 one trick, rational thinking, but when you're good and crazy, oooh, oooh,
 oooh, the sky is the limit!"  --- The Tick