path: root/UPGRADING

Mailman - The GNU Mailing List Management System
Copyright (C) 1998-2004 by the Free Software Foundation, Inc.
59 Temple Place - Suite 330, Boston, MA 02111-1307, USA


UPGRADING FROM PREVIOUS VERSIONS

    For the most part, upgrading Mailman entails installing the latest version
    over the existing version.  Usually, you can unpack the new release, run
    'configure' with the same options you used in your previous install, and
    then do a 'make install'.  However, there are some changes that may need
    to be taken care of manually.

    What you need to do depends on the version you are using and the version
    you are upgrading to.  In all cases, you should first turn off your mail
    and web access to your Mailman installation.  You're essentially upgrading
    a database, and it's usually a good idea to make sure the database cannot
    be modified in the middle of the upgrade.

    My recommendations are:

    - Turn off your incoming mail daemon.  Most remote smtp servers will
      simply queue up messages destined for your domain if port 25 is shut
      off.

    - Temporarily disable web access to Mailman.  You can do this by either
      turning off your web server temporarily, or by setting up a temporary
      redirect to a "service unavailable" page for the Mailman URLs.  Refer to
      your web server documentation for details.


UPGRADING FROM 2.1.4 to 2.1.5

    In Mailman 2.1.5, some significant changes have been made to the file
    formats for qfiles and the pendings database.  All care has been taken to
    make sure the upgrades happen automatically and smoothly, but you should
    double check and, for the ultra-paranoid, make backups of your Mailman
    site before you upgrade.  BE SURE TO TURN OFF MAILMAN AS DESCRIBED ABOVE
    BEFORE YOU UPGRADE.

    Specifically, in MM2.1.4 every message in the queues were represented by
    two files, a .msg or .pck file containing the email message, and a .db
    file containing metadata about the message.  In MM2.1.5 this has been made
    more efficient by using only one file (with a .pck extension) for both the
    message and metadata.  This should make MM2.1.5 half as hostile to the
    file system.

    The bin/upgrade script, which is run automatically when you upgrade,
    should convert all the old style qfiles to the new style qfiles.  Note
    that this could take a long time if you have a lot of files in your qfiles
    subdirectories.  Pay particular attention to files you might have in
    qfiles/shunt; these will get upgraded too, although files in qfiles/bad
    will not.

    In MM2.1.4, the database file containing pending actions (i.e
    subscriptions, unsubscriptions, message holds, etc.) was shared globally
    among all mailing lists.  In MM2.1.5, each list now has its own pending
    database file.  All care has been taken to properly split pending actions
    from the global to the list-specific files, but it's possible there are
    bugs here.  Best practice is to clear all pending actions before you
    upgrade, although this is not always possible.


UPGRADING FROM 2.0.x to 2.1

    When you upgrade from Mailman 2.0.x to Mailman 2.1, you should double
    check that your moderation and privacy options are still set the way you
    want them.  The Mailman options dealing with moderation and privacy have
    changed significantly, to make them easier to understand and control.
    Ever effort was taken to translate the old configuration variables to the
    new configuration variables, but because the old semantics were so
    complex, it is possible your settings may not have been correctly
    translated.

    Check especially the values for (in Privacy -> Sender Filters)
    default_member_moderation, generic_nonmember_action, and
    accept_these_nonmembers.  Check also the moderation flag on member
    accounts in the Membership Management screen.

    In Mailman 2.1, the qrunner subsystem has been completely
    rewritten.  You no longer start qrunner from cron!  Instead, there
    is a bin/mailmanctl script which is used to start, stop, and
    restart mail delivery.  This script is appropriate to use as a
    Unix init script.  Be sure to update your crontab with the new
    cron/crontab.in file.

    NOTE: It is very important that if you are upgrading from a
    pre-MM2.1alpha2 system to a post-MM2.1alpha2 system that you let
    the old qrunner process clear any and all messages sitting in the
    qfiles/ directory *BEFORE* you upgrade.  Otherwise after the
    upgrade, those messages will not get delivered, and I'm not
    exactly sure yet how to upgrade those pending messages.

    NOTE: When upgrading to Mailman 2.1, you will need to regenerate
    your aliases files.  There have been many changes to the alias
    names, the programs they map to, and the name of the wrapper
    script.  See README.<yourMTA> for details of making Mailman work
    with your mail server.

    To regenerate your aliases, use the bin/genaliases script.

    Mailman 2.1 introduces multilingual (a.k.a. internationalization
    or i18n) support.  Previously only one language per list was
    supported, and it was assumed that this language would be English.
    The upgrade script for Mailman 2.1 creates a subdirectory `en'
    inside each lists/<listname> directory.  It then copies all the
    .txt and .html files from lists/<listname> into
    lists/<listname>/en.

    If you have modified those templates to contain non-English text,
    you will have to manually rename the en subdirectories to the
    language code for the language of your templates.  Mailman's
    upgrade script should handle cleaning up any templates which are
    duplicates of the defaults, but you'll want to double check this
    manually.

    If you are running a MM2.0.x system with non-standard patches
    applied, you might have some other problems with your upgrade.
    Here are some instances we know about:

    - If you've applied patch #413752 (coerce to plaintext), then your
      upgraded will not go smoothly.  Take a look at patch #651406 for
      an unofficial solution.

      http://sf.net/tracker/?group_id=103&atid=300103&func=detail&aid=413752
      http://sf.net/tracker/?group_id=103&atid=300103&func=detail&aid=651406


UPGRADING INDIVIDUAL LISTS

    If you're nervous about upgrading all of your lists to 2.1 in one
    go, you can move them and upgrade them one at a time.  Start by
    doing a clean Mailman 2.1 installation in an empty directory --
    call it $MM21.  (I'll assume your Mailman 2.0 installation is in
    $MM20.)

    Doing this means you'll have co-habiting Mailman 2.0 and 2.1
    installations for a while, until you have moved all of your lists
    to Mailman 2.1.  Depending on your MTA and web server, this could
    be transparent and painless, or it could be an ongoing headache.

    If you use Apache with mod_rewrite, then it's fairly
    straightforward to set things up so that both Mailman 2.0 and 2.1
    inhabit the /mailman and /pipermail URL-space of your server; this
    makes the transition almost transparent to list admins and
    subscribers.  See below for details.

    Now, for each list that you want to move, you'll have to

      * Shut down your MTA.

        If you have a lot of outgoing list traffic, you might need to
        leave your MTA up but only let it accept connections from
        127.0.0.1 (localhost), so Mailman 2.0 can flush its queue.
        How to do this is MTA-dependent; for Exim, you can set
        "local_interfaces = 127.0.0.1" and "kill -HUP" the Exim
        daemon.

      * Shut down your web server.  For a more professional look, or
        if you want to allow people to keep accessing the rest of your
        web site, you could make your web server respond to all
        /mailman/ URLs with a "temporarily unavailable" message.

        How to do this is web server-dependent; with Apache and
        mod_rewrite, this does the trick:

          RewriteRule ^/mailman/.* /var/www/unavailable.html [L]

        (Of course, you'll have to supply your own
        /var/www/unavailable.html.)

      * Force Mailman 2.0 to process its queue:

          python -S $MM20/cron/qrunner

        (This is only necessary if there are any files in $MM20/qfiles;
        if you need to do this, make sure you left your MTA listening to
        127.0.0.1.)

      * Move the list:

          cd $MM20
          mv -i lists/foo-list $MM21/lists
          mv -i archives/private/foo-list $MM21/archives/private
          mv -i archives/private/foo-list.mbox $MM21/archives/private
          rm archives/public/foo-list
          rm archives/public/foo-list.mbox
          cd $MM21
          bin/withlist -l -r fix_url mylist

        (The fix_url step will not be necessary if your Mailman 2.0
        and 2.1 installations will be sharing the same URL-space.)

      * Edit your web server config so the list's URLs continue to
        work.  There are two possible approaches here; the simpler way
        is to setup a new slice of URL-space that will be used by your
        Mailman 2.1 installation, eg. /mailman-21:
        With Apache and mod_rewrite:

          RewriteRule /mailman/(.*)/(foo-list.*) /mailman-21/$1/$2 [R=temp]

        (The [R=temp] assumes that "/mailman-21/" is a temporary URL,
        and you'll move all your lists to "/mailman/" when the
        transition to Mailman 2.1 is complete.)

        If you don't want to expose ugly temporary URLs like
        "/mailman-21" to the world, it's only slightly more work to make
        Mailman 2.0 and 2.1 share the same slices of URL-space.  Here's
        how to do it with Apache and mod_rewrite:

          RewriteRule ^/mailman/(.*)/(foo-list.*) \
                      $MM21/cgi-bin/$1/$2 \
                      [T=application/x-httpd-cgi]

        Not only is this more aesthetically pleasing, it's faster -- no
        redirects.

        In either case, you'll want to rewrite the list's archive URLs
        to Mailman 2.1's archive:

          RewriteRule ^/pipermail/(foo-list.*) $MM21/archives/public/$1

      * Restart your web server (or disable the "temporarily
        unavailable" stuff).

      * Restart your MTA (or make it listen to more than just
        127.0.0.1).


UPGRADING FROM 2.0 to 2.0.x (where x >= 1)

    Nothing much more than running "make install" (after upgrading)
    should be necessary.


UPGRADING FROM 2.0 beta to 2.0 final

    You MUST re-run configure; running config.status is not sufficient
    due to some recent changes in the autoconf scripts.  You can do a
    head of config.status if you don't remember the options you
    originally ran configure with.

    The cron jobs for Mailman 2.0 final have changed considerably,
    including the frequency with which they run.  You should reload
    misc/crontab.in for the `mailman' user to get the right settings.
    See the INSTALL file for details.

    FAILURE TO DO THIS WILL RESULT IN A LESS THAN OPTIMALLY FUNCTIONAL
    MAILMAN INSTALLATION.


UPGRADING FROM 1.x to 2.x

    In addition to the instructions above, I highly recommend that you
    make sure your Mailman queue is cleared /before/ upgrading.

    Mailman version 1.x had a cron script called run_queue which was
    part of its bulk mailer.  With Mailman 2.x there is no default
    bulk mailer (it lets the MTA handle this), and it is currently
    unknown what the effects of upgrading are on the run_queue script,
    but I'll bet it's not good. :)

    The way to make sure that your Mailman queue is empty is to look
    in your $prefix/data directory.  If you see any files that start
    with "mm_q." you've still got messages waiting on the queue.  You
    can run $prefix/cron/run_queue by hand until the queue is cleared.
    Multiple invocations of this script won't help though; they lock
    each other out.  Also, be warned that clearing the queue can take
    a while and may cause a large load on your system (two reasons why
    all this stuff has been redesigned in 2.x :).

    You do not need to run "make update" if you are upgrading from
    version 1.0 or 1.1 to version 2.0, since this is now run
    automatically when you do a "make install".  However you should
    modify your crontab entries to execute cron/qrunner instead of
    cron/run_queue.  You can also safely remove the file
    $prefix/cron/run_queue.

    If you are upgrading from a pre-1.0 beta, you need to follow the
    instructions below.


UPGRADING FROM PRE-1.0 to 2.x

    You need to do a few extra things to make sure that the file
    system layout for the early 1.0 betas is upgraded to the 1.x
    configuration.  There are two ways to do this.

    First, from the source directory, after you've done a "make
    install" you can run "make update".  "make update" creates a file
    named "update.log" in the top level of the source distribution.
    If the script that updates the Mailman filesystem encounters
    something that is not resolvable, it will log info about this to
    "update.log".  This is worth checking after the upgrade completes.

    You can also just change to the installation directory (i.e. $prefix)
    and run bin/update.  This is the same as above except that the
    update.log file is not generated.

    Check your crontab entry.  Remove any runs of obsolete scripts, in
    particular cron/upvolumes_yearly, cron/upvolumes_monthly, or
    cron/archive.


WHAT "MAKE UPDATE" DOES

    Below is an annotated listing of the things that "make update"
    does.  Hopefully, this will help resolve any problems you are
    having.

    Note that it can't hurt to run "make update" each time you
    upgrade, but if you're running version 1.0 or newer, it won't help
    much either!

    - To upgrade to 1.0b10, you will need to copy
      templates/options.html to lists/<listname>/options.html for each
      mailing list you have.  However, if you have edited the
      options.html file, say from the Web interface, you will have to
      merge these changes in manually.

    - The upgrade to 1.0b7 included the removal of
      Mailman/smtplib.py{,c} since Mailman now uses the default Python
      1.5.2 version of smtplib.

    - Archiving files are moved around as part of integrating
      Pipermail into Mailman, as of 1.0b6.  In particular,

      1) if a list has only a private mbox archive
      $prefix/archives/private/<listname> is moved to
      $prefix/archives/private/<listname>.mbox/<listname>

      2) if a list has only a public mbox archive
      $prefix/archives/public/<listname> is moved to
      $prefix/archives/private/<listname>.mbox/<listname>

      and a symlink is made that points
      $prefix/archives/public/<listname>.mbox to
      $prefix/archives/private/<listname>.mbox/<listname>

      3) if a list has both private and public mbox archives,
      make update picks one of the above 2 configurations based on
      whether or not the list currently is archived publicly.  It then
      renames the other mbox to mbox.preb6.

      4) if a list used recent CVS sources, where archives were placed in
      $prefix/public_html/archives, then these are moved to
      $prefix/archives/private/<listname> and a symlink is made from
      $prefix/archives/public/<listname> to that spot if the list's
      archives are public.  Also, a permissions-related security
      problem is removed.

      To integrate mbox archives of old lists, log in as user `mailman'
      and run $prefix/bin/arch <listname> <path-to-mbox-archive>.

      Also, by default, beta6 does both mbox and html based archiving,
      but you can configure Mailman to do one, both, or neither.
      Please see $prefix/Mailman/Defaults.py for details.

      There was a short period of time when the CVS sources archiving
      code was not organized into its own package.  The pickled
      articles in the archives that were placed into archives during
      this period stored the path to the module HyperArch, but that
      module has moved.  You can quick fix this by running

      ln -s $prefix/Mailman/Archiver/HyperArch.py \
              $prefix/Mailman/HyperArch.py

    - If upgrading from version 1.0b4 or earlier, "make update" moves
      list-specific templates.  For each list,
      $prefix/templates/<listname>/* is moved to $prefix/lists/<listname>.
      Please reference the generic templates in $prefix/templates to see
      if any variables have changed (There shouldn't be many, only
      options.html was updated from b5 to b6).

      For really old versions of Mailman, you may not even have
      <listname> subdirectories in $prefix/templates!  In this case
      you will need to manually copy some files into your new list
      directories.  Here's an example shell command that will do the
      trick:

      cp templates/{archives,handle_opts,listinfo,roster,subscribe}.html lists/<listname>

    - Some modules that existed in previous versions, but that have
      been replaced with newer (differently named) modules, are
      removed.



Local Variables:
mode: indented-text
indent-tabs-mode: nil
End: