Great Circle Associates List-Managers
(May 2001)
 

Indexed By Date: [Previous] [Next] Indexed By Thread: [Previous] [Next]

Subject: Re: test messages etc
From: Tim Pierce <twp @ rootsweb . com>
Date: Wed, 30 May 2001 23:51:29 -0400
To: Chuq Von Rospach <chuqui @ plaidworks . com>
Cc: J C Lawrence <claw @ kanga . nu>, Nick Simicich <njs @ scifi . squawk . com>, List-Managers @ GreatCircle . COM
In-reply-to: <200105302144 . f4ULiZ113596 @ lists . apple . com>; from chuqui @ plaidworks . com on Wed, May 30, 2001 at 02:46:26PM -0700
References: <25994 . 991258275 @ kanga . nu> <200105302144 . f4ULiZ113596 @ lists . apple . com>
User-agent: Mutt/1.2.5i

On Wed, May 30, 2001 at 02:46:26PM -0700, Chuq Von Rospach wrote:
> 
> On Wednesday, May 30, 2001, at 02:31 PM, J C Lawrence wrote:
> 
> > What I ___*REALLY*___ want is to hook up MHonArc to an SQL backend
> > for archive message storage with all the thread links stored in
> > under SQL.
> 
> Me -- I've sort of decided at some point to throw mhonarc out 
> completely, because from what I've seen, it'd be a lot cheaper 
> computationally to stuff it all in a database, and generate index pages 
> and data pages on the fly than it is to have mhonarc update the indexes 
> and everything every time we insert a message in case a user might want 
> to visit before the next insertion...

That is exactly what we found.  Our archive server with 7-8 million
messages took about 12 hours to reindex the Mhonarc archives every
night.  The archive required three times as much disk as the raw
messages occupied, in order to build an HTML version of everything.
And only a fraction of the messages in the archive ever got viewed
*at all*.

We're much happier now.  I keep DBM files to cache threading and
index metadata, and render each message on the fly.  If we decide
to give the messages a different background or layout, I spend ten
minutes tweaking code, not 48 hours rebuilding the whole archive.
The machine hums and we have enough disk to last us at least another
year.

> > I've toyed with rondomly varying the above pattern across a variety
> > of replacement characters/strings, but haven't yet.
> 
> my problem with munging addresses is that any attempt to mung in some 
> standardized way (or one of a few more or less standard variants) is 
> just as easy for the spambots to unmung.

A wise man once compared archive security to "The Club" -- you just
have to make it easier for the spammers to go steal from someone
else. :-)




Follow-Ups:
References:
Indexed By Date Previous: Re: test messages etc
From: Tim Pierce <twp @ rootsweb . com>
Next: Re: test messages etc
From: "John R Levine" <johnl @ iecc . com>
Indexed By Thread Previous: Re: test messages etc
From: J C Lawrence <claw @ kanga . nu>
Next: Re: test messages etc
From: "John R Levine" <johnl @ iecc . com>

Google
 
Search Internet Search www.greatcircle.com