[Xastir] making big shapefiles

[Tom Russo]

On Fri, Mar 09, 2007 at 01:07:36PM -0600, we recorded a bogon-computron collision of the <jdw at eng.uah.edu> flavor, containing:
> I've been thinking about ways to simplify map configuration, and an  
> idea that just hit me was to combine TIGER shapefiles from multiple  
> counties to make one huge shapefile, reducing the set to three or  
> four per state.  I know I'm gonna take a hit the first time I read a  
> big shapefile, but rtree should save me after that first one.

I don't know if you'll really save that much.  Having smaller, tiled
shapefiles would probably save you more (they could be excluded in toto).
There is not a huge cost to searching an rtree, but there is  a cost.

I'd be interested to see your results.

> Can I configure rtree's cache timeout value?  

Not at the moment.  The cache purge frequency is hard coded as the 
"PURGE_PERIOD" preprocessor macro.  There's no reason that couldn't be changed.

> I know there's not a  
> slider on the timer menu and I would probably need to hack the  
> source.  Other than the obvious permanent memory utilization, is  
> there any reason I shouldn't make rtree's cache live forever?

That's the only reason it's not permanent --- my logic was that if an rtree
hasn't been used for over an hour, the benefit of having that memory used 
isn't there --- rtree is a memory/time trade-off, and if you're not zooming
in and out or panning a bunch, the memory is lost for no speed gain.  If you
*are* using it, the rtree isn't purged.

Having the cache time out user-configurable makes some sense, though.  It's 
just never really come up as a priority yet.  I'm all for it if you wanna
hack.  The relevant code is in shp_hash.c, all using the macro "PURGE_PERIOD".
The function purge_shp_hash() is called (from main.c) very often, but returns 
doing nothing if the period hasn't passed since the last time it purged.

-- 
Tom Russo    KM5VY   SAR502   DM64ux          http://www.swcp.com/~russo/
Tijeras, NM  QRPL#1592 K2#398  SOC#236 AHTB#1 http://kevan.org/brain.cgi?DDTNM
"And, isn't sanity really just a one-trick pony anyway? I mean all you get is
 one trick, rational thinking, but when you're good and crazy, oooh, oooh,
 oooh, the sky is the limit!"  --- The Tick