Name: Jasper Wallace
Member since: 2000-05-24 00:24:25
Last Login: 2007-11-26 14:40:04



NetBSD user, 'full-time' sysadmin ;-)

Has done minor patches for a few things, and some major patches that he should really ship, currently the only thing i can think of that credits me is EtherApe.

I'm also working on a userland IrDA stack, but IrDA is wayyy too much like hard work ;-P


Recent blog entries by jasper

Oops, Jerzy Balamut, the solid pop3 author pointed out that the pop3 protocol dosn't count bytes stuffed line beginings in the size returned by the LIST and STAT commands. I checked the RFC's, and he's right.

It's not that bad actually, cos i've been thinking about writing a script that takes a mail on stdin and writes it into a maildir with the right line endings etc.

It's needed for MTA's that arn't as flexable as exim, and i can now encode the file size without byte stuffed line beginings in the filename (a bit of a hack, but not a problem).

got TOP working, and made exim 'byte-stuff' lines with a single '.' (which is how the pop server tells the client that it's finished sending the mail).

It all seems to work! (and, more importantly, is RFC complient).

Now it's supper time ;-)

I'm setting up a high capacity pop server to replace one thats thrashing it's self to death at the moment, so I've been trying to find the most efficent pop server for what I want to do.

At the moment we are running qpopper, but if a mailbox gets above 50Mb or so the server really slows down as qpopper copies the entire mailbox before and after a user transfers their mail.

We need a system that works well with:

  • Large (50Mb+) mailboxes
  • Many users accessing large mailboxes at the same time

To get around the problem of large mailboxes I want a pop server that uses maildir, and for vairous reasons I want one that runs standalone, i.e. not through inetd.

After some searching through vairous mailing list archives and freshmeat I decided upon Solid POP3.

The problem now is that the wire-format for POP3 has all line endings as \r\n, where as the normal unix line endings are just \n. This means you have to scan through each message to get a count of the lines so you can account for transmitting all the line endings as \r\n - you need to know the size of each message so you can give the right responce to the LIST command.

With the test mailbox I was using (an excessive 1Gb in size and 329 messages) this takes 2 minutes!, which is WAY to long as far as user interactivity goes!

Now, since the only processes that are going to be touching these mailboxes are the MTA i'm using (exim), and the pop server I could do whatever I wanted with the mailbox format.

I set up exim to deliver all the mails with \r\n line endings, and then created a new mailbox type in the solid pop3 source, crlf_maildir.

Since we don't need to munge the line endings any more all we have to do for tha vairous pop commands is:

  • Getting the data for LIST is just a question of looping through readdir and stat'ing each file for it's size.
  • RETR is just a mmap and write.
  • TOP is just mmap, loop through counting line endings, and then write. (fetchmail uses TOP <msg> 99999999, and so might other pop clients, so if we check to see if no. of lines >= size of message we can just chuck out the message as a whole...)
  • ... and everything else is as before

I've implemented most of this, and it now takes less than a second to authenticate a client, rather than 2 minute, which is a nice imporvement ;-)

Expect to see a new version of Solid POP3 soon!


jasper certified others as follows:

  • jasper certified alan as Master
  • jasper certified jasper as Apprentice
  • jasper certified miguel as Master
  • jasper certified Stab as Apprentice
  • jasper certified wsanchez as Master
  • jasper certified rmk as Master
  • jasper certified rakholh as Apprentice
  • jasper certified highgeek as Journeyer
  • jasper certified toledo as Journeyer

Others have certified jasper as follows:

  • jasper certified jasper as Apprentice
  • Stab certified jasper as Apprentice
  • sad certified jasper as Apprentice
  • jrf certified jasper as Journeyer
  • Zaitcev certified jasper as Apprentice

[ Certification disabled because you're not logged in. ]

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

Share this page