| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| |
|
|
|
|
|
| |
files and preserve the file. This occurred when dequeue
of a large entry threw MemoryError, the runner died and
restarted, and then unpickling the .bak threw another
MemoryError.
|
| |
|
|
|
| |
from the dequeue() method to the recover_backup_files() method
in order to minimize added i/o.
|
| |
|
|
|
|
|
| |
of times the entry has been dequeued in a recover backup
scenario. When the count reaches three, the entry is
preserved to the shunt queue and the fact logged. This
prevents looping on a bad entry that causes a runner crash.
|
| |
|
|
| |
or -admin address in the list's owner attribute (1834569).
|
| |
|
|
| |
encoded message part.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
previous commit.
- Normalized a couple more 'URL:'.
- Improved handling of None payloads.
- Cleaned up a few charset coercions.
OutgoingRunner.py - Made probe bounce processing and queuing of bounces
conditional on having some permanent failure(s).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The email address which forms a part of the various CGI data keys
in the admin membership list is now urllib.quote()ed. This allows
changing options for and unsubbing an address which contains a
double-quote character.
- CGI/admindb.py
Added additional test to not display "Database Updated ..." when
coming from the login page.
- CGI/roster.py, HTMLFormatter.py
Changed to show hidden members when authorization is site or list's
admin or moterator password. Patch 1587651.
- Defaults.py.in, Handlers/Cleanse_DKIM.py
Added a new REMOVE_DKIM_HEADERS Defaults.py/mm_cfg.py setting (default
= No) to control removing dkim/domainkey signatures from posts and
mail to -owner.
- Handlers/Decorate.py, Handlers/Scrubber.py
Changed to preserve format=flowed and delsp=yes in the Content-Type:
of the body when adding header/footer and when scrubbing attachments
and to remove trailing spaces from the header/footer lines so they
won't be flowed. Bug 1495122.
Fixed a scrubber issue where the i18n translated 'next part' separator
can be garbled if the list charset is different from the message.
- Queue/Runner.py. Queue/Switchboard.py
Now that we have .bak queue entries for recovery, it is no longer the
case that an unparseable message is lost. In this case, and in case
of other exceptions when dequeueing, I added a preservation feature
to move the .bak file to qfiles/shunt as a .psv file and write an
appropriate log entry. It is also possible for an attempt to shunt
a message to fail. One example that occurred in practice (bug 1656289)
was caused by a huge message that threw a MemoryError in processing and
then threw another MemoryError in the attempt to pickle the message for
the shunt queue. In this case as well, I log and attempt to preserve
the original queue entry by renaming.
|
| | |
|
| |
|
|
|
|
|
| |
yet ready to work from the trunk). Also, get rid of all the obsolete
.cvsignore directories, they're no longer needed.
Almost ready for 2.1.9rc1!
|
| |
|
|
| |
"while times.has_key(key):" for execution efficiency while maintaining
Python 2.1 compatibility.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Queue/Runner.py
- Queue/Switchboard.py
Backported the gfiles backup changes from the trunk. The following comments
are from the trunk checkin. Note that the test cases are not added.
Added robustness to Switchboards and Runners so that if a runner crashes
uncleanly (e.g. segfaults the Python interpreter), messages being processed
will not be lost.
The vulnerability, ideas, and patches are credited to Richard Barrett and Mark
Sapiro. Their original work was modified by Barry for this commit and any
bugs are his fault.
The basic idea is that instead of unlinking a .pck file in dequeue(), the file
is renamed to a .bak file. The Switchboard grows a finish() method which then
unlinks the .bak file. That class's constructor also grows a 'restore'
argument (defaulting to false), which when true moves all .bak files it finds
in its hash space to .pck, thereby restoring a file lost while "in flight".
This relies on the fact that even with multiple qrunners, exactly one process
will be responsible for one hash space slice, so it's never possible (under
normal operation) for a .bak file to be renamed to .pck by some other process.
Test cases for both the new Switchboard behavior and the use of that by Runner
subclasses has been added.
There are two things to watch out for, either of which may require some
additional changes. There is some small potential to duplicate messages in
various queues, if say 'mailmanctl' were improperly started more than once by
a site admin. This usually won't happen unless an admin is overly eager with
the mailmanctl -s switch, so we can chalk this one up to operator error. I'm
not sure what more we can do about that.
There's also a possibility that if we're processing a message that continually
causes the Python interpreter to crash, we could end up duplicating messages
endlessly. This is especially troublesome for the Outgoing runner which could
conceivably cause a mail flood. I consider this the more critical issue to
defend against, probably by adding a numbering scheme to the .bak file names
and refusing to restore a .bak file more than say 3 times without human
intervention.
|
| |
|
|
|
| |
slices that could result in unprocessable queue entries. Improved FIFO
processing when two queue entries have the same timestamp.
|
| | |
|
| |
|
|
|
|
| |
BounceRunner.py to discard the bounce when Stop returned. Changed
DSN.py to recognize Action: headers with comments. Changed Qmail.py
to recognize an observed different starting string.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
from the posts from mailling list for gatewaying to nntp.
|
| | |
|
| | |
|
| |
|
|
|
| |
more precautions to assert the command should be in 'us-ascii'
and more errors.
|
| |
|
|
|
| |
any more (they were error signals in the previous switchboard implementation),
so simplify the loop. We can't have 'lost data files'.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
queuing of bounce events out of the process's memory and into a 'bounce event'
file, essentially consisting of a stream of pickles. Every once in a while
the qrunner wakes up and processes the event file, which is named to be
specific to each qrunner instance. The actually processing is factored out
into a mixin class which is now subclassed by both the BounceRunner and
OutgoingRunner classes (since the latter has to handle SMTP rejects as
bounces).
Specific changes here include:
OutgoingRunner: Use BounceMixin as a base class. We can now get rid of the
whole _permfailures and _permfail_counter stuff since we'll just use the mixin
class as the basic logic for bounce handling.
_dispose(): Simplified since we can just use _queue_bounces() when a message
is rejected at SMTP time, although if that triggers a probe, the attached
message will be the one for which delivery was attempted since there really
isn't any bounce. Also, make this method handle probe bounces.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
queuing of bounce events out of the process's memory and into a 'bounce event'
file, essentially consisting of a stream of pickles. Every once in a while
the qrunner wakes up and processes the event file, which is named to be
specific to each qrunner instance. The actually processing is factored out
into a mixin class which is now subclassed by both the BounceRunner and
OutgoingRunner classes (since the latter has to handle SMTP rejects as
bounces).
Specific changes include:
REGISTER_BOUNCES_EVERY: Promote to a Defaults.py.in variable.
BounceMixin: Mixin class for handling the basics of bounce event queuing,
periodic processing, and registering, as well as cleanup. It also contains a
_probe_bounce() method for handling bounces of the probe message (which cause
an immediate disable).
BounceRunner: Add BounceMixin as a base class.
|
| |
|
|
|
|
|
|
|
|
|
| |
one-file-per-queued-message architecture. The message object and metadata
dictionary are now written to the same .pck file -- in that order -- instead
of to separate files which were more complicated to manage. dequeue()
especially simplifies considerably.
Also, get rid of the special _Switchboard hack, as well as the
MarshalSwitchboard, ASCIISwitchboard, and BSDDBSwitchboard implementations.
Also get rid of the DumperSwitchboard class.
|
| | |
|
| |
|
|
| |
SHIGENO Kazutaka.
|
| |
|
|
|
| |
HeaderParseError, so if that happens just ignore the Subject header for
command purposes.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
Promote SYNC_AFTER_WRITE to a Defaults.py/mm_cfg.py variable after
all.
|
| | |
|
| |
|
| |
'Release_2_1-maint'.
|
| | |
|
| | |
|
| |
|
|
|
|
| |
dequeue(): If we can't read a .db file, initialize data to the empty
dictionary, since the following test expects to do a has_key() on the
value. Closes SF bug #707608.
|
| | |
|
| |
|
|
| |
Ademar de Souza Reis Jr
|
| | |
|
|
|
'Release_2_1-maint'.
|
