[Xastir] torrents and xastir

[Gerry Creager]

Curt, WE7U wrote:
> On Fri, 1 Aug 2008, Jason KG4WSV wrote:
> 
>> ...I'm trying to understand if and how we (xastir
>> and APRS users) can benefit from the distribution system in the
>> sharing of our sometimes large chunks of data, like maps and VM
>> images.
>>
>> The basic premise/assumption is that many people will want to download
>> and many of those will make the data once they have downloaded, right?
>> If no one peering, there is nothing to download?  I guess it degrades
>> to a traditional master-client download if there's only 1 "peer".
>>
>> I'm trying to figure out if it's worth the effort, given our
>> relatively small numbers, to make things like VM images, TIGER
>> shapefiles, DRGs, DOQQs, etc available via bit torrent.
> 
> In the recent past I said I didn't know anything about torrent.
> That was somewhat false, as I've been using it for a few years but
> only in the "leecher" mode.  When it got to "seeding" I always shut
> it down.  Now I see a reason to leave it in seeding mode and have
> done so.  I basically understand how torrent works and am getting
> used to some of the special lingo for it.
> 
> I don't yet know if the below scheme would work but I'll put it out
> there.  Someone more familiar with it can shoot it down:
> 
> a) Someone runs a web page that has multiple ".torrent" files on it,
>    representing all the things we have available to download.  This
>    could be a Wiki page, right?  We'd have to be careful who has
>    edit access to it (user/password).  We have one such page now
>    with three items on it, but it's not a Wiki.  This web page ends
>    up with very little usage as the bulk transfers happen
>    client-to-client, not from this web page.
> 
> b) People make various maps available for download.  For instance I
>    have three DVD's containing all of the USGS topos for WA that are
>    freely distributable.  I also have a good portion of OR and a few
>    for ID/MT/HI.  I'd love for these to be "out there" for other
>    people to use.
> 
> c) These same people create a torrent file out of this data, upload
>    it to the Wiki, and edit the Wiki to make that torrent link show
>    up.
> 
> d) The first download ends up being a server/client relationship as
>    you described, but if at least a few of the people doing the
>    downloads leave their torrent client up and running, they become
>    a shared resource for the same file.  More than likely if we put
>    maps up there we'll have non-Xastir people doing downloads as
>    well, maybe even becoming additional resources to download from.
> 
> e) The maps contributed in this manner must be freely distributable
>    to preserve our good project name.
> 
> Questions I have still:
> 
> 1) Would we want to create a torrent for each file?  For each CD or
>    DVD set?  In my case the DVD's were created from a LOT of CD's,
>    and each CD had a LOT of maps on it.  Of course each map is
>    megabytes of data as well.  I know I could create a torrent for
>    an entire directory, but in the case of the DATA and METADATA
>    directories for these CD's or especially DVD's the download would
>    be huge.  I guess once two people have it and are seeding it
>    changes things, but that initial first download would take forever.
> 
> 2) To make it easier on the map providers, one big download is the
>    way to go.  To make it easier on the "leechers" it might be
>    better to have smaller increments.  How does one decide this?
>    I'm sure I could write a script that would make a torrent file
>    out of each map file, but am not sure I could do the same to get
>    the multiple thousands of torrents uploaded to a Wiki.
> 
> 3) Is the idea of distributing maps via torrent workable at all?
>    I've been looking for a method to make USGS DRG's and DOQQ's
>    available for a number of years, and this seems the closest match
>    to date.

CAVEAT:  I can't even START to think about this 'til I can extricate 
myself from a particular project, but what I'd like to see would be some 
variant on this...

Imagine a site with a CONUS map (to start, and thinking small; we could 
scale up) where you can use a bounding box to identify your region of 
interest or cursor to select a particular point (map).  After that 
selection you see an inventory of different maps and types of maps 
available, and you use a check-list to identify the ones you want.  The 
site prepares a separate page/Torrent stream to provide these, and the 
page is lightly persistent (days before it ages out) and indexed on a 
page of recent selections.

You'd have the option of getting the data via download or Torrent at 
that point.  Simplified data delivery.

I'm not sure how making delivery of all those maps via Torrent can be 
done without some form of simplified selection, but then I'm not a big 
Torrent user.  I tend to get OS distro's and kernel updates that way, at 
home, because it "just happens" and I don't have to worry much about it. 
  This doesn't make me an expert, though, as I got it working once and 
locked in the format.
gc
-- 
Gerry Creager -- gerry.creager at tamu.edu
Texas Mesonet -- AATLT, Texas A&M University
Cell: 979.229.5301 Office: 979.862.3982 FAX: 979.862.3983
Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843