[Xastir-dev] Shared memory questions...

[Greg Eigsti]

Thanks for the reply!  I think I now have a perma-headache from trying to
understand how the pipe'ing works between all of the forked processes in
x_spider ;)

I have hacked the living daylights out of x_spider and have my web server
code working; I understand your concern with opening yet another port...
But darnit this is what I am interested in ;)  I'll consider your suggestion
re: writing out a file for apache (or other) to serve.

My basic take on the whole thing is...  TCP_Server starts my thread which
sits in an accept loop; when a new connection comes in my thread spawns
another thread which sends html back.  In my implementation str_echo2 has
gone away and most of its work is handled in pipe_check which is now the
work horse.  Kind of yucky but it is working.  I had to get everything
within the same process so that my http server thread could have access to
the latest pipe_object list (I added a bunch of stuff to that struct).

Understanding the pipes and reworking all of that was a giant headache, but
as I said it now works.  I'll ply you with a screenshot ;)

I am not sure how insecure my code is since I never read the socket on an
incoming "status http request".  I just blurt http bytes back after a
successful accept.  Naïve, maybe, but leaves hackers no room to send nasty
strings to your open port...

> You're _new_ to Unix programming and are alreading getting into

New to purist *nix programming; I have lots of experience on NT and OS X.
Both support fairly BSD like API's so the concepts are not new.

Greg


On 12/21/06 5:45 PM, "Curt Mills" <archer at eskimo.com> wrote:

> On Tue, 19 Dec 2006, Greg Eigsti wrote:
> 
>> Here comes a newbie question; I am fairly new to *nix programming and have a
>> question regarding forking and shared memory.
> 
> You're _new_ to Unix programming and are alreading getting into
> forking, threads, and shared memory?  :-O
> 
> 
>> I am working on adding an http status page to xastir which shows xastir
>> status including connected TCP port status.  Most of this work is in
>> x_spider.c where the pipe_object structure is defined.  I have extended the
>> pipe_object structure to contain additional per user connection data.
>> 
>> The problem that I am having is that since the TCP_Server is forked (and it
>> 'manages' the pipe_object list) any changes to the list are not reflected in
>> the other x_spider processes.
>> 
>> I investigated using shared memory to hold the pipe_object list but this
>> will turn into a huge hassle quickly if one considers the code that needs to
>> be written to manage the shared memory (and all the changes to the
>> implementation of the pipe_object list).
>> 
>> Another alternative is to change the forking of the TCP server (and my new
>> 'status server') to be threads.  This would make it much easier for the TCP
>> server and 'status server' to share the pipe_object list.
>> 
>> Suggestions, warnings, hints, tips, tricks?
> 
> First, we discussed putting a web server into Xastir some time back.
> I believe the general concensus was that we shouldn't, instead
> relying on the Unix strategy of smaller/simpler tools that one can
> add together to get the job done.
> 
> Doing our own web server also opens us up to cracking, on one of the
> favorite TCP ports for this sort of activity.  Programs like Apache
> already keep track of exploits and needed security, therefore
> they're better suited to write/patch code for http.
> 
> So...  I guess what I might suggest is that you add code which
> writes out html files to disk, perhaps with a configuration option
> for defining the top-level directory for this, and also perhaps SUID
> access to such directories.  Apache or some other web server could
> then serve up these pages.
> 
> w.r.t. the fork versus thread question, I tried both when I wrote
> the spider server.  The end result (what you see today) was due to
> the need for a smaller memory image for all the server processes.  I
> fork the x_spider server immediately upon starting Xastir (if the
> server ports are enabled), then each incoming socket gets a
> relatively small program to service it.  If I forked from the main
> Xastir process later I'd end up with very large memory images for
> these processes.  Same I think for threading.
> 
> I'm not aware of any major problems with the x_spider code, so I'd
> certainly rather leave it alone right now and concentrate on other
> parts of Xastir that need work.  Or enhancements.