Fri, Jul 22, 2005 at 01:15:49PM +0200, Jerry L wrote:
Maildir/work.sieve
Maildir/private.sieve
>
>That would look less intrusive, opening up the possibility that
>other programs can edit them safely

That sounds like a good plan, I'll make it so.

[]

Well there isnt really any (un)written standard that says .sieve
either so.

[]

The only fear I have about extending sieve into bincimap is that the
draft managesieve-04.txt is LD.

Forgive my ignorance here guys, but how well is the IMAP client able
to control this filtering-on-delivery? From the name I guess that's
what said draft describes? Hmm, how would the filters be used? To
automatically sort incoming mail into different folders?

Again, sorry I don't know more about this. I'd appreciate a pointer
somewhere or just a few lines describing how it fits into IMAP.

//Peter

Friday 22 July 2005 09:30 am, Peter Stuge wrote:
Fri, Jul 22, 2005 at 01:15:49PM +0200, Jerry L wrote:
Maildir/work.sieve
Maildir/private.sieve
>
>That would look less intrusive, opening up the possibility that
>other programs can edit them safely
>
That sounds like a good plan, I'll make it so.

[]

Well there isnt really any (un)written standard that says .sieve
either so.

[]

The only fear I have about extending sieve into bincimap is that the
draft managesieve-04.txt is LD.

Forgive my ignorance here guys, but how well is the IMAP client able
to control this filtering-on-delivery?

clients that support sieve would connect to a sieve daemon and talk a protocol
to change their filter settings.

kmail is the only client I am aware of that supports sieve and I've never
used cyrus, so I'm not sure how WELL it supports it :)

From the name I guess that's what said draft describes?

sieve is server-side filtering which with imap is where all of your
filtering should go :)

Hmm, how would the filters be used? To automatically sort incoming
mail into different folders?

yes, based on rules you can configure with a sieve-speaking MUA or a special
configuration utility.

Again, sorry I don't know more about this. I'd appreciate a pointer
somewhere or just a few lines describing how it fits into IMAP.

http://www.cyrusoft.com/sieve/ (google is your friend! :)
-Jeremy

PGP SIGNATURE
Version: GnuPG v1.4.1 (GNU/Linux)

6UD8/CnUCo1JUyxaxk2GevQ=
=G
PGP SIGNATURE

Fri, Jul 22, 2005 at 12:10:14PM -0500, Gary wrote:
Jclients that support sieve would connect to a sieve daemon and
Jtalk a protocol to change their filter settings.

This is for draft-managesieve. I don't like that design too much, nor
having the scripts stored in the filesystem. They kind of go
together. The problem I see is that there's no way to build
(portable) tools around them since the sieve RFC is independent
of all mail architectures. That makes it a two-edged sword

I like Sieve itself a lot, but the best thing would be if we could
figure out a mail architecture agnostic way to store the scripts, to
go with the ditto filtering system.

I guess an implementation of managesieve will abstract it away, but
Mh, I don't know

Jkmail is the only client I am aware of that supports sieve and
JI've never used cyrus, so I'm not sure how WELL it supports it

Mulberry also supports it (in Linux/Unix, SX, or Windows
versions), in fact, this client will generate the script for you.
Cyrus supports it fully, as that is what Fastmail.fm uses as their
backend IMAP.

I would really have liked to see managesieve as an IMAP extension.

another note, this is kind of related to the folder hooks that I
care a lot for. :) Is the intention to implement Sieve in bincdrop
and only bincdrop and use it only for incoming mail? Would there be a
point in having Sieve filters on all folder? In that case, one could
use folder hooks to outsource all message delivery from bincimapd to
bincdrop.

Flame away! :)

//Peter

Sun, Jul 24, 2005 at 01:30:59PM +0200, Andreas Aardal Hanssen wrote:
Sat, 23 Jul 2005, Peter Stuge wrote:
>Would there be a point in having Sieve filters on all folder? In
>that case, one could use folder hooks to outsource all message
>delivery from bincimapd to bincdrop.
>Flame away! :)

Hahaha ;-). Folder hooks are a cool idea; I must say. Sieve
filtering should happen on delivery, though.

Agreed, and that's out of scope for bincimapd anyway. But I like
bincdrop and the fact that it can reuse so much of bincimapd.

But with messages added through APPEND, the client expects the
exact same message to be in the folder it APPENDed to when the
command has finished, and clients do go haywire if the contents
change or the message isn't there.

Fair enough, but clients can also be notified of changes (supposedly)
made by other clients accessing the same folder, right? If a sieve
and/or hook changes the "contents" of a folder in some way that might
as well have been by another client, wouldn't the client always
behave correctly as long as it is kept up to speed on folder changes?

For hooks, I think it's K to require the hook tool to keep the state
itself and push change notifications to binc, but if everyone prefers
binc to keep folder state internally and keep track of the hook tool
I won't mind since it'll likely make hook tools much simpler to
write.

Does anyone know what other servers do? (Cyrus?)

Not me, but Sieve is part of Cyrus in some way and since Cyrus isn't
a mail delivery system AFAIK I assume Sieve is involved in
inter-folder delivery somehow. Just speculation though.

//Peter

Mon, Jul 25, 2005 at 08:58:36AM +0200, Jerry L wrote:
Peter Stuge wrote:
>This is for draft-managesieve. I don't like that design too much,

I totally agree with you that the draft is horrible. It is some
what sad that they happen to make a email filtering script and not
a good way to interface with them on a imap server.

My point exactly.


>I like Sieve itself a lot, but the best thing would be if we could
>figure out a mail architecture agnostic way to store the scripts,
>to go with the ditto filtering system.

My idea was a filter directory in the root of the maildir folder
that contains mime-tagged files. But that is just an idea.

Sure, as long as someone is looking there, that will work. But what
about compatibility with regard to the Maildir spec? Admittedly, Binc
has already broken that with .binc* files, but I'd like to try and
fix that rather than adding more files in there.

Do you have any ideas btw?

Well, any number of database engines could be used to store the
scripts, LDAP, SQL, simple dbase, etc. But at least for some that
still means storing a file in the filesystem somewhere, which is what
I want to avoid, at the very least on a per-user level. Perhaps in
the service directory? in /etc/bincimapd?

My problem with extra files in the mailbox stems from the fact that I
mostly have virtual users and want to keep the mail directory
structure clean and uncluttered. For sites with mostly real user
accounts the opposite actually makes more sense since users would be
able to work on the files/scripts in any way they please.

I can imagine that virtual mail management packages get cranky if
there's too many unexpected files around too but I don't know of an
example at the moment.

Andreas, have you thought about generalizing metadata somehow,
perhaps even moving it completely into some DBMS that is easy to work
with? I'm using SQLite for a project now and it's small and quick and
public domain. :) But like I said, there are many options, and
something like the metadata backend that Jerry was thinking about for
bincdrop would be great to have in bincimapd as well.

//Peter

Mon, Jul 25, 2005 at 08:58:57AM +0200, Jerry L wrote:
Andreas Aardal Hanssen wrote:
>Hahaha ;-). Folder hooks are a cool idea; I must say. Sieve
>filtering should happen on delivery, though. But with messages
>added through APPEND, the client expects the exact same message to
>be in the folder it APPENDed to when the command has finished, and
>clients do go haywire if the contents change or the message isn't
>there.

Hmmm, maybe this should be clarified some, filtering should NLY be
applied on delivery. Never when APPENDing mail. Unless I've totally
missed something.

Hmm.

The managesieve draft is the extensions for manipulating the script
from an imap client.

Hmm, then I misunderstood. There was talk of a special daemon that
speaks the managesieve protocol and I didn't read the draft very
carefully. Hold on No, I remember right, "IANA registration is
pending. Current implementations generally use port number 2000." and
managesieve is described as a separate protocol, not an extension of
IMAP. It just is "a line oriented protocol much like [IMAP4rev1] or
[ACAP]."

The abstract also notes that "it is hoped that eventually Sieve
scripts will be stored on ACAP." but after doing some quick research
on ACAP it doesn't seem to be quite the hit that Sieve makers were
hoping for.

Cyrus ACAP has a "Note that we're not currently actively developing
this product, and it requries SASL 1.5.x." message by the download
link.
CMU's latest release of their acapd is from January 1998.

(Interesting note in the ACAP RFC: "an attempt was made to keep ACAP
as simple as possible. It is a traditional Internet text based
protocol which massively simplifies protocol debugging. It was
designed based on the successful IMAP [IMAP4] protocol framework,"

and here I was thinking that IMAP was just a little bit messy :)

So the changes would be adding bincdrop into the bincimapd suit and
adding the extention of managing the scripts.

Add bincdrop to bincimap suite or even package separately. I'd prefer
the latter actually since both could be used without each other. But
as far as I can tell, MANAGESIEVE isn't an IMAP extension. I would
like it a whole lot more if it was.

//Peter

Mon, Jul 25, 2005 at 01:27:05PM +0200, Jerry L wrote:
>>My idea was a filter directory in the root of the maildir folder
>>that contains mime-tagged files. But that is just an idea.
>
>Sure, as long as someone is looking there, that will work. But what
>about compatibility with regard to the Maildir spec? Admittedly, Binc
>has already broken that with .binc* files, but I'd like to try and
>fix that rather than adding more files in there.

Well, the .binc* is a nono and the Maildir spec says its ok to
store files in Maildir/. with alphanumeric (or atleast thats what
every one does) so I don't see why we can't store our filter there
also.

You are of course right and I'm busy taking the foot out of my mouth.

<-- cr.yp.to/proto/maildir.html:
Can a maildir contain more than tmp, new, cur?
Yes:

* .qmail: used to do direct deliveries with qmail-local.
* bulletintime: empty file, used by system-wide bulletin programs.
* bulletinlock: empty file, used by system-wide bulletin programs.
* seriallock: empty file, used to serialize AutoTURN.
8--

Maybe we should only have one file named "filter" and have the
scripts in MIME parts.

At least name it sievefilter.
Sure, MIME will work, except that it's hard to work with for everyone
else, who don't have a nice MIME parser at hand. :)


>>Do you have any ideas btw?
>
>Well, any number of database engines could be used to store the
>scripts, LDAP, SQL, simple dbase, etc. But at least for some that
>still means storing a file in the filesystem somewhere, which is what
>I want to avoid, at the very least on a per-user level. Perhaps in
>the service directory? in /etc/bincimapd?

Sure, having a database backend that can store data in a directional
structure is a fine idea

Doesn't have to be directory based - the username is a unique
identifier.

but hello? users filter files in /etc ? =)))
I don't know how many users you have but I'm dealing with around
50.000 and several machines.

Yeah, sorry, disregard that /etc thing. :) I wasn't thinking
straight.


>My problem with extra files in the mailbox stems from the fact that
>I mostly have virtual users and want to keep the mail directory
>structure clean and uncluttered. For sites with mostly real user
>accounts the opposite actually makes more sense since users would
>be able to work on the files/scripts in any way they please.

Erhm, I dont really get you here, what is the problem with virtual
users having their filters inside the Maildir. The point of sieve
is to have the scripts managed remotly via the IMAPd.

except that it isn't. I wish it was.

is there more than one "managesieve draft" ? Perhaps I've been
reading the wrong document? :) See my other message from a few
minutes ago.


>I can imagine that virtual mail management packages get cranky if
>there's too many unexpected files around too but I don't know of
>an example at the moment.

Not really, there is alot of program already today that adds
bunches of files directly into the Maildir/. .

And that is fine. Strike that complaint of mine.


>Andreas, have you thought about generalizing metadata somehow,
>perhaps even moving it completely into some DBMS that is easy to work
>with? I'm using SQLite for a project now and it's small and quick and
>public domain. :) But like I said, there are many options, and
>something like the metadata backend that Jerry was thinking about for
>bincdrop would be great to have in bincimapd as well.

No thanks, you are forgetting the idea behind Maildir and it is

Sorry, I should have been more clear. I didn't mean for storing mail
but for storing binc's internal metadata, to get rid of .binc* files.

The mail storage is already abstracted in bincimapd. :)

//Peter

Mon, Jul 25, 2005 at 02:08:18PM +0200, Jerry L wrote:
Peter Stuge wrote:
>Sure, MIME will work, except that it's hard to work with for
>everyone else, who don't have a nice MIME parser at hand. :)

And for ppl that want to edit it by hand.

True as well.

[]

>Sorry, I should have been more clear. I didn't mean for storing
>mail but for storing binc's internal metadata, to get rid of
>.binc* files.

I can't really see a better solution then plain files since dbm
files will break distributed filesystems.

dbm as in dbase or in database management? And why does contents
matter , you mean one single database for all users - yes, I
see Well, that rules out some backends, but not all. A central SQL
server could do the job and serve all bincimapds.

Let me clarify; I don't have any more problems with the way things
work right now - a couple of extra files in the maildir - they sort
of bother me, but the performance gain alone is motivation enough.
I'm just saying that if we start adding more than one or two files to
the maildir I think it would be nice to have the possibility to
switch to another backend for bincimapd's metadata (subscriptions,
cache and uidvalidity) as well as bincdrop's filters.

Mon, Jul 25, 2005 at 02:08:47PM +0200, Jerry L wrote:
Peter Stuge wrote:
Mon, Jul 25, 2005 at 08:58:57AM +0200, Jerry L wrote:
>
>>So the changes would be adding bincdrop into the bincimapd suit and
>>adding the extention of managing the scripts.
>
>Add bincdrop to bincimap suite or even package separately. I'd prefer
>the latter actually since both could be used without each other. But
>as far as I can tell, MANAGESIEVE isn't an IMAP extension. I would
>like it a whole lot more if it was.

Gah, you are so right, I got blinded by the sieve code in
cyrus-imapd/imap/ that I didn't see the timsieved. And its Tim
Martin that has written the draft and make the daemon. Well that
isn't an option.

Yes and no I agree it's a bad idea to add a managesieve daemon to
binc, but the problem still needs to be solved. Users' scripts have
to be accessible somehow, and the easier (for both man and machine)
the better.

But this doesnt stop bincdrop and I would really want to see
bincdrop included inside bincimapd because it's easier to reuse and
maintain the depot and mime code that bincdrop will use.

I'm all for bincdrop using the same codebase that bincimapd does, and
I don't mind bincdrop being included in the bincimap package, but I
can imagine other uses for bincdrop at times when one doesn't want
bincimap, so it would be a good idea to package bincdrop on it's own
as well. And then it may not be a good idea to have it included in
bincimap anymore. :) But I absolutely agree it should reuse binc
code, that's still possible, right?

perhaps break out lowlevel shared things into libbinc? Am I getting
carried away? :)

//Peter

Tue, Jul 26, 2005 at 08:58:02AM +0200, Jerry L wrote:
Peter Stuge wrote:
>I'm just saying that if we start adding more than one or two files
>to the maildir I think it would be nice to have the possibility to
>switch to another backend for bincimapd's metadata (subscriptions,
>cache and uidvalidity) as well as bincdrop's filters.

I only disagree to this because the most common setup of a imapd
server is to have a distributed/network filesystem (nfs afs samba)
and there is alot of problems having filebased databases such as
db, gdbm and sqlite on such filesystems.

Yes, the fs:es don't have locking. But for those who don't have a
separate storage server the DBMS:es you mention work great.

What I agree about is that implementing a generall storage backend
(and this is not only for the maildir class) that can store data in
a directory way,

Mh, I'm not sure I want it to be hierarchical, but ok.

I didn't want to use this for depot abstraction, since that's already
done in binc, it's possible to just write a new depot and have
whatever support desired.

But you have a point, if there's a (simple) metadata storage solution
with classes for a couple of DBMS solutions, it wouldn't be too hard
to make a Depot wrapper around it. :)

enable ppl to build mailsystems that doesnt need some kind of
advanced network filesystem since it can just talk to a mysql
cluster.

Storing mail in a DBMS has been discussed already, and Andreas shot
it down for scalability reasons - there's simply better performance
in a file system. But yes, if there's a clever storage solution in
the background somewhere, the IMAP server will have to know how to
use it to access the depots.

I think this should be desided by Andreas, either solutions is
possible, it just a matter of taste really.

Right. What do you think about it, Andreas?


>perhaps break out lowlevel shared things into libbinc? Am I
>getting carried away? :)

Hehe, yes =) I think it's easier just to distribute the same needed
code twice.

It uses more RAM and leads to slower deliveries.

//Peter