[CQ-Contest] Competing in the Daylight - Making it Happen

[Lyndon Nerenberg]

>
> I was envisioning a system where there are hooks in logging  
> programs (N1MM,
> WriteLog, TR-Log, etc.) what would forward log entries to a central  
> web
> server cluster via HTTP POSTS's.

HTTP is unnecessary overhead. Updates via UDP will work just fine.   
In fact, this whole thing will run fine on UDP.

> The other half of this involves a socket push server cluster that  
> would
> support 100,000 or more live connections.

Where do you get this 100000 figure from? That seems awfully high.

 > I would need about 3-4 Intel servers
> running the Linux O/S.   I would also need a load balancer (F5, local
> director) or another server that can function as a load balancer.   By
> clustering 3-4 servers there would be redundancy in case one server  
> should
> fail.

You should be able to make this work on a single 1GHz P3 Celeron.   
You're trying to make this way more complicated than it needs to be.   
The two critical resources will be sufficient RAM and a reasonably  
fast network stack (for which I would lean towards running this on  
Solaris 10).

> 2)  I would have to write a detailed specification for the logging  
> software
> providers and get their concensus as to how this is implemented.

All you have to define is the UDP update protocol.  This should be  
very simple to do.

But before setting off to do all of this you need to address a more  
fundamental issue: how are you going to prevent people from sending  
in bogus log entries?  You need a security layer for the protocol.

--lyndon