[Xastir] Search FCC database not working?

[Tom Russo]

On Tue, Jan 25, 2005 at 05:19:41PM -0800, we recorded a bogon-computron collision of the <archer at eskimo.com> flavor, containing:
> On Mon, 24 Jan 2005, Tom Russo wrote:
> 
> > There are about 8 entries in the file with "^M^M^J" (that is, two CRs and a
> > newline) embedded in the address field.  This is confusing the sort procedure
> > and results in a few lines of the sorted data that don't fit the pattern
> > the fcc lookup feature requires.  By deleting the broken lines we get the
> > thing to work, but we're also getting broken entries for those 8 hams.
> >
> > A filter to strip out "^M^M^J" before sorting would fix this.  An appropriate
> > "sed" incantation might do the trick.  I would expect that this sort of junk
> > will only appear more often in the FCC data and we should figure out a way
> > to clean it out.
> 
> So basically those lines are broken into two no matter what OS
> you're running on, correct?  This causes 8 lines to turn into 16,
> and none of them are correct lines.

Nod.  The "^M^M^J" instances all appear what would normally be the middle of a 
line, usually in address fields.  I'm guessing that these might be from
blank lines inserted into input fields in online applications that aren't
being stripped out before saving data in the database.

> I was initially thinking that a short one-liner Perl script could
> work to fix up the line-ends, but in this case the input data file
> is just plain broken.  It should be reported to the FCC so that they
> can fix their process.

Yes, that's the ideal fix.  But I wouldn't count on them being very
responsive, so the quickie fix to remove all the "^M^M^J" instances would 
still be prudent.  Especially since there's already an "fcc-get" script
in xastir's script directory to massage their unsorted data into what we need 
for xastir.

-- 
Tom Russo    KM5VY     SAR502  DM64ux         http://www.swcp.com/~russo/
Tijeras, NM  QRPL#1592 K2#398  SOC#236 AHTB#1 
 "When life gives you lemons, find someone with a paper cut."