News::Gateway - Tools for gatewaying messages between news and mail.
use News::Gateway;
my $gateway = News::Gateway->new (1, 'user@example.com');
$gateway->modules ('headers', mailtonews => ['misc.test']);
$gateway->config_file ('/path/to/config/file');
$gateway->read (\*STDIN);
my $error = $gateway->apply ();
if ($error) { $gateway->error ($error) }
$error = $gateway->post ();
if ($error) { $gateway->error ($error) }
This module is a collection of tools to allow gatewaying of mail messages into newsgroups or vice versa. It's written to be extremely modular and extendable, so that any modification or check of incoming messages can be dropped in. It's intended to be usable for both a straight gateway or a robomoderator, or even as the front end and supporting tool set for human moderation.
The module is intended to take input from three different sources: an incoming message, a set of modules that contain checks or modifications to be performed to that message, and a configuration file that affects the behavior of the modules. Each module can list one or more configuration directives that it takes, and then when those directives are encountered in the configuration file, they're parsed and and handed off to the module. Then, after the message has been read in, its headers and body are given to each module in turn, so that they can perform whatever tests and make whatever modifications they were designed to do.
A message is represented internally as a News::Article object that each module has access to, allowing the headers or body to be read and manipulated.
This builds a new News::Gateway object. INTERACTIVE is a true/false
flag specifying whether the program is running interactively. If true,
the error() method will call die in the event of an error;
otherwise, it will attempt to send mail to the maintainer and the sender
of the message. MAINTAINER is the e-mail address of the maintainer of
the gateway, defaulting to the user the program is running as if not
supplied. SENDER is the envelope sender which should be used for all
mail sent through News::Gateway, defaulting to MAINTAINER if not
supplied.
First, the standard methods in the order in which they are normally
called. Some of these methods, like modules(), are normally called
only once, while others, like read(), may be called multiple times for
multiple incoming messages. You'll almost certainly also want to read
about the config(), config_file(), and post() directives below in
UTILITY MODULES.
Sets the list of modules that should be applied to each incoming message.
The list of modules is cumulative; in other words, modules() can be
called multiple times to add more modules to the list of ones which will
be applied. This method both adds each supplied module name to the list
of modules that will be run if apply() is called with no arguments and
sets up callbacks so that any configuration directives that MODULE is
interested in will be passed along to it when configuration directives are
processed.
Some modules may take arguments; if so, they can be passed in as an anonymous array associated with the module. For example:
$gateway->modules (mailtonews => ['misc.test']);
adds mailtonews to the list of modules and passes in misc.test as an
argument to it.
Modules will be run in the order they are given to modules().
modules() should always be called before any call to apply(),
config(), or config_file() that those modules are supposed to
affect.
Reads in a new message from the supplied source, which can be a file name,
a file handle, or even an anonymous sub which will be called repeatedly
for each line of the message until it returns undef. MAXSIZE, if
given, specifies the maximum size in bytes of the article body, and
MAXHEAD, if given, specifies the maximum size in bytes of the article
headers. MAXSIZE defaults to 256KB and MAXHEAD defaults to 8KB.
This method returns the total number of bytes read if successful, or undef in the case of an error.
This method constructs a new News::Article object and calls its method, so
it will always support essentially the same arguments as the corresponding
News::Article read() method.
This method hands the message off to each module given (if no modules are
given, the message is given to each module registered by modules() in
the order in which they were given to modules()) and lets them do
whatever they wish with the article. If a module returns undef, it's
presumed to be successful and the message is given to the next module. If
a module returns a string, it's considered to be a failure message and
apply() stops.
If all modules succeed, apply() returns undef or the empty list. If a
module fails, apply() returns a list consisting of the module which
failed and the message it failed with in a list context; or a string
consisting of the name of the module, a colon, a space, and the message it
failed with in a scalar context.
The calling script should then look at the failure message and take the appropriate action.
The following additional methods are also supplied, some of which are mostly intended for internal use.
Handles a fatal error. If the INTERACTIVE flag passed to new() was
true, this method just calls die with the error message. Otherwise, it
calls mail_error() (part of the standard mail module), which
attempts to mail the maintainer and the sender of the message. If that
fails, for whatever reason, News::Gateway calls exit with the right
status code for a mail bounce.
This method should be used only for fatal, unrecoverable errors,
particularly when INTERACTIVE isn't true, since it could bounce mail or
notify the wrong person of problems. It's a last resort that goes to
great lengths to try to keep the message from being lost.
An accessor method that returns the underlying News::Article object.
Takes a News::Article object and makes that the underlying message in that instance of News::Gateway, discarding the old one if any.
The following modules to perform checks or rewrites of messages are part
of the standard distribution. To use them with a News::Gateway object,
pass them (along with anonymous arrays for their options, if any) to the
modules() method.
This module requires that every incoming message have at least one keyword on the subject line. For the purposes of this check, a keyword is defined as anything that matches:
/^(?:Re:\s+)?\[\S+\]/
(in other words, something without spaces enclosed in square brackets at the beginning of the subject line modulo any Re:). If no keyword is found in the incoming message, this module returns the message:
No keyword found
This module takes no arguments or configuration directives.
Extracts headers from the beginning of a message body and lifts them up into the message headers. This is to assist posters whose software makes it difficult for them to edit the headers of their posts or mail.
This module looks for recognized headers at the beginning of the article body. As long as it's seen only blank lines (possibly containing whitespace) and recognized headers, it adds all of those headers to the main message headers. As soon as it finds a line that isn't a recognized header, it stops, and if it found any headers it removes everything up to the first non-header line from the body.
This module takes one configuration directive:
A list of headers to look for in the beginning of the body. The headers are not case sensitive, and any amount of whitespace (including none) is allowed after the colon.
For example, with the directive:
bodyheaders x-no-archive
and a message beginning with:
x-no-archive:yes
Comment: hello
an X-No-Archive header with content "yes" will be added to the message headers and the first line of the message body after this module runs will be the line starting with "Comment:". Had "comment" also been listed in the configuration directive, that header would have been lifted into the article headers as well and this module would have continued looking.
Whitespace on the line before the header is not allowed, and such a line would not be recognized as a header to be lifted to the message headers.
This module takes no arguments and cannot fail.
This module checks the body of an incoming article and cleans it up, attempting to enforce standard ISO 8859-1 text. If it's encoded in quoted-printable, that encoding is undone. Microsoft Word SmartQuotes are converted into their ASCII equivalents, any Ctrl-Ms are removed, and the module checks to make sure that the message contains no invalid characters (outside of the standard ISO 8859-1 range) and that no lines are longer than 79 characters.
The messages returned by the checks in the event of failure are:
The body contains characters outside the range [\s!-~\xa0-\xff].
One or more lines in the body are over 79 characters long.
This module takes no arguments or configuration directives. If the original message was in quoted-printable, it will change the Content-Transfer-Encoding header to be "8bit". It recognizes quoted-printable from the MIME encoding headers.
Limits crossposting and followups. This module takes, in some ways, the opposite approach from the newsgroups module; rather than listing newsgroups which are allowed, it allows one to set a limit on the number of groups crossposted, on the number of groups to which followups would go, and on which groups can be crossposted to or to which followups can be directed.
A group can be excluded from the Newsgroups header or from the Followup-To header in one of two ways; either messages posted to that group or messages that direct followups to a set of newsgroups including that group can be rejected, or that group can be silently removed from the Newsgroups and/or Followup-To headers. The latter behavior is generally not recommended for robomoderators, since moderators are generally discouraged to make decisions about where posts should be posted for the poster. Returning the post and asking the poster to choose a more appropriate set of groups is generally a better approach.
This module will remove a Followup-To header (and not add one) if its content is/would be identical to the Newsgroups header.
This module takes the following configuration directives:
Reject any message crossposted to more than COUNT newsgroups.
Either remove GROUP silently from the Newsgroups header if present or
reject all articles crossposted to GROUP.
Reject any messages that would direct followups to more than COUNT
newsgroups, bearing in mind that if no Followup-To header is present,
followups would go to all groups in the Newsgroups header.
Either remove GROUP silently from the followups if present or reject
all articles crossposted to GROUP.
This module may return one of the following failure messages:
The message was crossposted to a group mentioned in a crosspost reject
directive.
Followups to the message would go to a group mentioned in a followup
reject directive.
The message was crossposted to more groups than allowed by a crosspost
max directive.
Followups to the message would go to more groups than allowed by a
followup max directive.
This module works with a Newsgroups header, so if other modules need to be applied to construct the Newsgroups header, they should run before this one. This module takes no arguments.
This module implements some general header rewriting functions, including adding, dropping, and renaming headers and replacing header contents. It takes one configuration directive:
Specify an action on the header HEADER. FUNCTION can be any of the
following: drop deletes a header, rename renames the original header to
X-Original-HEADER retaining the same value, ifempty adds a header with
content CONTENT if and only if the message doesn't already contain a
header HEADER, replace replaces all existing HEADER headers with one
containing CONTENT, prepend adds CONTENT to the beginning of the
first header HEADER or creates a new header HEADER with content
CONTENT if none already exists, and reject returns an error if
HEADER is present in the incoming message.
CONTENT can contain various special variables: $n will be replaced
with the name of the running program, $v will be replaced with the
version of News::Gateway, and $i will be replaced with a unique
identifier formed from the current time and the process ID. $$ will be
replaced with $, so to put a literal dollar sign in a header, you
should use $$.
For example, suppose you have a configuration file with the following directives:
header organization add SNAP
header message-id rename
header sender drop
header comment replace $n $v
header subject ifempty no subject (thread id $i)
and suppose you have an incoming message with the headers:
Organization: Restaurant Reviews
Message-ID: <123142@bar.org>
Sender: foo@bar.org
Comment: Hello
Comment: Hello again
After the headers module runs, the message will have a header of:
Organization: Restaurant Reviews
Organization: SNAP
X-Original-Message-ID: <123142@bar.org>
Comment: PROGRAM VERSION
Subject: no subject (thread id ID)
where PROGRAM is the name of the running program (ie, $0),
VERSION is the version of News::Gateway, and ID is a unique
identifier as described above.
This module may fail and call error() with the following message while
reading the configuration directives:
A rewrite action was specified that isn't among those that are supported. This probably indicates a typo.
This module may fail in one way:
A header that was associated with a reject action in a configuration directive was present in the incoming message. Note that the header will be given in all lowercase.
As a side note, if you're constructing a robomoderator for a newsgroup, dropping or renaming the Path header in incoming messages is highly recommended. It turns out that some news servers will add a Path header with their hostname before remailing the message to a moderator, and if you keep that Path header when you post, the article will never propagate back to the site of the original poster.
This module checks the Subject header of a message and ensures that it starts (modulo an initial Re:) with a valid keyword. By default, the following keyword formats are supported:
KEYWORD:
KEYWORD/KEYWORD:
[KEYWORD]
[KEYWORD/KEYWORD]
[KEYWORD][KEYWORD]
If multiple keywords are given, they all have to be valid. Keywords are checked against a list given in a file and are case-insensitive.
This module takes one optional argument, a reference to a sub which, if given a subject line, returns a list of all the keywords present. This can be used to override the default patterns listed above.
There is one required configuration file directive:
Specifies the file from which the list of valid keywords will be read. The file should list all valid keywords, one per line.
Two failure messages are possible:
There were no keywords in the subject header of the message.
A keyword was found that wasn't in the list of valid keywords.
Generates an X-Mail-Path header from the Received headers of a mail message. It attempts to figure out the real name of each host that the mail passed through based on the standard comment syntax used by most mailers, but this problem is inherently impossible to solve completely.
X-Mail-Path is set to a !-separated list of hosts that the mail has passed through. If from the Received headers the name of a host appears to be trusted, it's given; otherwise the name followed by the IP address in brackets is given. If the IP address isn't available, "[UNTRUSTED]" is appended.
The envelope sender is added to the end of the X-Mail-Path if known; if not, "UNKNOWN" is added instead.
mailpath takes no arguments or configuration directives and cannot fail.
Performs a variety of rewrites, changes, and checks necessary to do the basic conversion of a mail message into a news message. Most of this involves deleting or renaming headers that a news server will not accept or would overwrite and making sure that the required news headers are present.
This module does the following: Checks to ensure there is a From header; compacts the From header down to one line if it contained any continuation lines since INN can't deal with continuation lines in a From header; adds a Newsgroups header with the default group if none is present, or checks the Newsgroups header for validity and removes duplicate groups if one is present; drop the headers Lines, Received, Relay-Version, and Xref which will be rejected or overwritten by the news server; renames the headers NNTP-Posting-Host and Sender to be prefixed with X-Original- since new ones will be generated; adds a new Sender header pointing to the maintainer (set in the News::Gateway constructor); renames the Message-ID header to X-Original-Message-ID if the message ID is syntactically invalid for a news message (this will cause a new message ID to be generated by the server); checks the In-Reply-To header if present to see if it contains something that looks like a message ID and if so adds it to the end of the References header if and only if it doesn't duplicate the message ID on the end of the References header; and sets the subject of the article to the value "(none)" if there is no Subject header.
This module takes one argument, the default newsgroup to post to. A Newsgroups header containing this group will be added if no Newsgroups header is present in the incoming message. This argument is optional, but if you're gating mail messages to news messages you'll pretty much always want to supply it. While it's tempting to assume that all submissions to a moderated group will already have a Newsgroups header added by the news server, you'll find in practice that people will often mail submissions directly to the submission address, or that news servers won't handle things in the way that you would expect.
This module (possibly combined with the headers module) may be sufficient for generating the Newsgroups header if the program only handles one newsgroup, but any more complicated situation will probably require that a module (such as the newsgroups module) be run before this one to generate the Newsgroups header.
There are three possible failure messages which may be returned;
The body of the message was empty. Most news servers will reject messages with an empty body.
The incoming message doesn't have a From header. A From header is required for all messages.
The incoming message didn't have a Newsgroups header and no argument was supplied to the module. If you don't supply an argument specifying the default newsgroup, all incoming messages must have a Newsgroups header before this module is run, since a Newsgroups header is mandatory in a news article.
This module takes no configuration directives.
Does simple PGPMoose signing of messages. This module is not a full implementation of PGPMoose; in particular, it doesn't support all of the crosspost handling. It looks through the groups in the Newsgroups header and adds a PGPMoose X-Auth header for every group in the Newsgroups header for which a PGP key has been given.
This module takes one configuration directive, which associates a PGP key ID and passphrase with a newsgroup:
Associates the given KEYID and PASSPHRASE with NEWSGROUP.
KEYID is optional and will default to the name of the newsgroup
surrounded by spaces if not given. (This default will allow PGP to find
the key correctly if the key ID is something like "Moderator for
group.name <address>" while avoiding confusing group.name and
group.name.other.)
Note that using this module means putting the PGP passphrases for the moderation keys in cleartext in the configuration file or in the script.
This module takes no arguments and cannot fail.
Programmatically munges the message IDs in the Message-ID and References headers by adding a slash-separated list of newsgroups to which the article is going to be posted, followed by a slash, to the beginning of them. We also strip off any prefix that looks like a prefix we'd generate (any number of strings that could be a newsgroup name, separated by slashes) to prevent threads with replies from readers of multiple mungeid'd groups from generating constantly-growing message IDs and to hopefully still get threading right. For the purposes of this check, we assume newsgroup names will contain at least one period; this module may not work correctly for newsgroup names that do not.
The purpose of this is for use with mailing list to newsgroup gateways. Since messages to multiple mailing lists may all have the same message ID, and since the same mailing list may be gated to Usenet in multiple places, the message IDs of incoming mailing list messages need to be munged in some fashion before being passed on to Usenet. Ideally, however, threading should be preserved, and replies to mailing list messages from other people on the mailing list won't refer to the munged message ID. This module therefore applies the same munge to all of the message IDs in the References header as well, in the hope of maintaining threading and still getting message ID uniqueness.
If the message doesn't already have a Message-ID header, this module will generate one.
Since an accurate Newsgroups header is needed to correctly munge the IDs, this module should run after any modules responsible for generating that. It also needs the final Message-ID and References header, so it should run after the mailtonews module since that module may promote message IDs from In-Reply-To into References and discard existing Message-ID headers.
This module takes one optional configuration directive:
REGEX is a regex matching newsgroups which should be used to make up
the prefix that we'll be adding to message IDs. If this configuration
directive is supplied, then only newsgroups matching one of the supplied
regexes will be used to construct the prefix. By default, all newsgroups
in the Newsgroups header will be used in the prefix.
mungeids takes no arguments and cannot fail.
Construct the Newsgroups header for a message. This is the general module for this purpose, and is considerably more flexible than the simple handling built into the mailtonews module.
The guiding purpose of this module is to attempt to deal with crossposting correctly in the context of a generic mail to news gateway (where incoming mail messages may not, unlike submissions for moderated groups, already have Newsgroups headers in them). We're ambitious, though, and try to also handle the case where we do have an incoming Newsgroups header.
All of the gory details of what this module does in every circumstance are documented in the comment block at the beginning of the code. If you're really curious, I'd recommend reading that. It's far more information than most people are going to want to know. Here's the briefer summary:
The program tells this module what the primary newsgroup is. This should be the group to which we should post by default. In other words, suppose you have several different mail to news gateway addresses for several different groups which all feed into the same program. To use this module, configure such a system so that the program knows what address a given message arrived via (either by having access to the envelope recipient through using qmail, using procmail as a delivery agent, or some other method, or by having each mail to news gateway alias give the program the name of its primary newsgroup on the command line). The program tells this module what group that is by providing it as an argument to the module.
We also have a list of associations between addresses and newsgroups given to us in configuration directives. An address can either be a literal address (which will be matched exactly, albeit without regard to case) or a regex. Note that although this module wants to know about as many of those associations as possible, it can deal with not knowing about them all and will generally do the right thing. It will just tend to multipost when it could have crossposted unless it knows which addresses correspond to which newsgroups.
If the incoming message doesn't have a Newsgroups header, this module will construct one by inspecting the To and Cc headers and seeing where the message was sent. If the primary group corresponds to one of the addresses in the To and Cc headers, then it will crosspost between the groups corresponding to the recognized addresses in the To and Cc headers (with the caveat given below). Otherwise, it assumes that the address via which we received the message either is one we don't know about or was Bcc'd and posts only to the primary group.
There is one problem with this. If a message is addressed to a number of different addresses, all of which eventually gate to a newsgroup, if all of those gateways do this analysis and crosspost, the article ends up multiposted (one copy for each group crossposted to). The solution to this problem is for one of the instances of the gateway program decide to post and all of the other ones exit quietly. We support this by returning the error "Not primary instance" if the primary group is in the list formed from the To and Cc headers but isn't the first group in that list. We assume that we'll also get the mail going to the address corresponding to the first group in that list, and that instance will do the crossposting. A program using this module should probably exit silently if we return "Not primary instance".
If the message does contain a Newsgroups header, then what we do depends on whether our primary group is among the groups in that header, and whether any of the addresses in the To or Cc headers correspond to groups in that header. The exact rules are very complicated (see the source comments), but we ignore the Newsgroups header entirely if neither the primary group nor the groups corresponding to addresses in the To and Cc headers occur in it, honor it if the primary group is in it, and post only to the groups corresponding to addreses in the To and Cc headers that aren't in the Newsgroups header if some addresses in the To and Cc headers correspond to groups in the Newsgroups header but our primary group isn't in that header. We also return "Not primary instance" where appropriate to ensure that a post to a given set of newsgroups is only done once.
(Yes, this is a simplification. See the source comments for the real complexity.)
We also rename Message-ID to X-Original-Message-ID if we have some expectation that otherwise we'll clash with another post (either made by another instance of the same program or posted elsewhere). We also rename Newsgroups to X-Original-Newsgroups if we don't honor the supplied header. Note that this module goes to some length to avoid renaming Message-ID unless necessary; if you want to drop all incoming message IDs and generate new ones, you should do that using the headers module.
We take an argument specifying the primary group, and we take one configuration directive in one of three forms:
Adds NEWSGROUP to the list of valid newsgroups and optionally
associates it with either ADDRESS or PATTERN. ADDRESS is a
literal string that (case-insensitively) exactly matches the address
associated with NEWSGROUP. PATTERN is a Perl regex that matches
addresses associated with NEWSGROUP.
Tells this module to consider any newsgroup matching PATTERN to be
valid to crosspost to. Note that this directive doesn't set up any
address to group mappings, just changes what groups are allowed in a
pre-existing Newsgroups header.
Tells this module to add all newsgroups matching PATTERN in the file
FILE to the list of valid newsgroups for crossposts. Note that this
directive doesn't set up any address to group mappings, just changes what
groups are allowed in a pre-existing Newsgroups header. FILE must
be an absolute path (i.e., it must begin with /).
In all of the above, the /s around PATTERN arguments are required, as
they allow unambiguous parsing of the configuration file directives.
There are three possible failure messages returned by this module:
The incoming message already had a Newsgroups header which included a group that wasn't on the list of allowable newsgroups for crossposting.
Although the primary group is among the groups we would normally post to, it isn't the "first" such group. This probably means that multiple copies of the message were received by different gateways and this instance should exit silently since another instance will be doing the posting.
In addition to that, there are two possible fatal errors that can occur
during the parsing of the configuration file and a third at the time of
parsing the incoming message. These errors are passed to the
News::Gateway error() method.
An error occurred while compiling the given regex from a PATTERN
argument to a configuration directive.
An error occurred while attempting to open a file from a FILE argument
to a configuration directive.
The newsgroups module requires an argument giving the primary newsgroup, or all of the tricks we use to figure out crossposting and where to post won't work correctly.
Rewrites a news article into a mail message, using a set of newsgroup to e-mail address mappings to determine which addresses to mail the resulting message to.
This module does the following: Checks to make sure there's a Newsgroups header; drops Bcc and Resent-Bcc headers in the message; renames To, Cc, Apparently-To, Resent-To, Resent-Cc, Return-Path, and Sender headers (if present) to the same name prefixed by X-Original- since they may be misinterpreted by e-mail software, and sets a new To header consisting of the e-mail addresses corresponding to all the groups in the Newsgroups header that have defined mappings. Duplicate addresses are stripped out.
This module takes one configuration file directive:
Specifies the file from which the list of newsgroup to address mappings should be read, in the form newsgroup, whitespace, address. If the filename ends in .db, then the file will be assumed to be a Berkeley database instead, with newsgroups as the keys and addresses as the values.
Two failure messages are possible:
The incoming message doesn't have a Newsgroups header. This module only handles posts and needs the Newsgroups header to determine what addresses to which to send the message.
None of the newsgroups in the Newsgroups header had a mapping to an e-mail address, so there's nowhere to send the message.
This module takes no arguments.
Checks the message to see if it is or contains a binary and rejects it if it is. The following checks are performed: Ensure that no Content-Type header in the headers or the body contains the strings "application", "image", "audio", or "video"; ensure that no Content-Transfer-Encoding header in the headers or the body is equal to "base64", and ensure that encoded lines do not exceed 50% of the number of lines in the body for any article with at least 40 lines.
An encoded line is defined as a line beginning with an M (with optional leading whitespace or quoting characters) and exactly 60 or 61 characters in length, or a line containing no spaces, not starting with M or ~, and between 59 and 80 characters in length.
Lines meeting those criteria are counted separately (the first as potential uuencoding, the second as potential base64 encoding), and if the count of lines in any one category is over half of the total body lines, the message is rejected.
The rejection messages are:
The message contains a Content-Transfer-Encoding header with a value of "base64".
The message contains a Content-Type header that contains one of the strings "application", "image", "audio", or "video".
The message body is at least 40 lines and more than 50% of those lines start with M (modulo whitespace and quoting) and are exactly 60 or 61 characters long.
The message body is at least 40 lines and more than 50% of those lines contain no spaces, do not begin with M or ~, and are between 59 and 80 characters long.
This article takes no arguments or configuration directives.
Adds the previous host through which a mail message passed to the Path header of the message. It does this by looking through the raw Received headers, finding the first Received header that specifies a host from which the message was received, and prepending it to the Path header (or creating a new Path header if none already exists).
If no Received headers that contain a "from" clause are found (which may be the case for mail originating locally), nothing is done.
The reason for this module is to support news to mail and mail to news gateways for the same unmoderated newsgroup. If the mail to news address is subscribed to a mailing list and posts to the newsgroup are sent back to the mailing list, a loop will be created unless something is done to stop it. One method of stopping such loops is to alias the hostname of the mailing list machine out of the feed to the news to mail gateway in the news server and then use this module to put the hostname of the mailing list machine in the Path of all articles posted through the mail to news gateway.
This module takes no arguments or configuration directives and cannot fail.
Checks to make sure that the incoming message is from a poster in a file of valid posters. The most common use of this module is in conjunction with human moderation; all messages from pre-approved posters are sent straight through via the robomoderator and the rest are relayed to a human moderator for hand checking.
This module takes one configuration directive:
FILE is a list of poster addresses, one per line. Only the address
should be included in this file, not the name or other comments. Files
ending in .db are reserved for future implementations (eventually,
these will automatically be recognized as Berkeley db files for handling
large lists of posters).
There is one possible failure message:
The message is from a poster address which isn't in the whitelist of known addresses.
In addition, during parsing of the configuration directives, one fatal
error, which would be passed to error(), is possible:
An error occurred while trying to open a file given as a FILE argument
to a configuration directive.
This module takes no arguments.
The following modules are also part of the standard distribution. Rather than rewrite or check messages, they provide additional methods that can be used by programs, other modules, even or the News::Gateway core code. Methods supplied by utility modules can be called like any other News::Gateway method.
This module contains the code for reading and parsing configuration directives and calling the appropriate module callback hooks. Nearly all News::Gateway programs will make use of this module.
WARNING: Information in configuration directives is trusted by
modules, and modules may go so far as to eval things from configuration
directives as Perl code. This means that anyone who has access to the
configuration directives read by this module has as much control over what
a program does as someone who can edit the source. Any necessary sanity
or security checking on the content of configuration directives must be
done before they are passed off to modules, and config_file() does
no such checking.
Calls the registered module callbacks for all modules who have expressed
an interest in DIRECTIVE. This bypasses any parsing, and the arguments
are given to the callbacks exactly as passed in. DIRECTIVE is
case-insensitive.
Reads in configuration directives from the supplied file name or file handle (which can be either an object reference or a reference to a typeglob) until end of file is reached.
Blank lines and lines beginning with a # are ignored. All other lines
are parsed into a directive (the first whitespace-separated word) and some
number of arguments, and then the parse results are passed along to the
module which has registered interest in that directive. If an argument
contains embedded whitespace, it can be enclosed in double quotes. A
backslash will escape the next character, whatever it is. A line is
considered to be continued on the next line if it ends in a backslash, and
if the line continuation occurs inside double quotes, a literal newline
will be part of the string.
For example, the following configuration directive tells the headers
module to add a new header named X-Comment with the content "This is a
long comment about this newsgroup":
header x-comment add \
"This is a long comment about this newsgroup"
Multiple configuration files can be read by calling config_file()
multiple times. One handy trick for simple scripts using News::Gateway is
to put the configuration lines at the end of the script after an __END__
and then pass them to News::Gateway with:
$gateway->config_file (\*DATA);
config_file() should be called before any apply() it is supposed to
affect.
Parses and handles a single configuration line, just as if it were read
from a file (in fact, config_file() calls this method for each
non-blank, non-comment line). This can be used to feed individual
configuration lines to the News::Gateway object without having to give it
a file or file handle.
Parses a configuration file line, returning the results as an array. This
is the method used by config_file() and config_line() to parse
configuration files. This probably isn't generally useful.
These methods adds two additional fatal error messages, which may be
passed to error().
A configuration line was unable to be parsed. The most likely cause is unbalanced double quotes.
News::Gateway encountered a configuration directive that no module had
expressed interest in. Chances are you made a typo in your configuration
file or forgot to register a module with modules() that you were
planning on using.
This module separates out various functions related to sending mail into their own module so that it can be autoloaded on demand rather than always compiled. The following methods are included:
The message in its current form will be mailed to ADDRESS, and the
method will return true on success, false on failure. If ADDRESS is
not supplied, the message is mailed to the addresses in the standard mail
headers (To, Cc, Bcc, and the Resent-* varients depending on your MTA), so
this method should also be used after news to mail rewrites have taken
place.
NOTE: The old mail_forward() method is DEPRECATED.
Bounces the message by printing ERROR to stderr and then exiting with
the permanent failure status code (as defined in the beginning of the
News::Gateway source). This should only be used as a last resort,
particularly for robomoderators, since the bounces will go to the envelope
sender address which is often not the same as the poster.
Attempts to send an error message to the sender of the original message, cc'd to the maintainer address. Replies will be set to the maintainer address, the subject will be "failure notice," and a Precedence header of "junk" will be added, which can be used to avoid loops.
If no From or Reply-To header has been seen, or if sending the mail fails,
this method will call mail_bounce() with the same error message instead.
The message sent will be:
Hi. I'm afraid that I was unable to post your message. This
is a fatal error; I've given up. A copy of this report is
being sent to my maintainer. The error message I received was:
ERROR
--- Below this line is a copy of the message.
followed by the original article headers and the body, with ERROR replaced
by ERROR. Note that if ERROR should be word-wrapped in the message,
this must be done by the caller.
Mails a News::FormReply-style message to the author of the incoming
message. See the News::FormReply manpage for more information. FILENAME is the
path to the file to use to construct the reply, and SOURCES for
News::FormReply variables can optionally be provided. The following
variables are exposed to the form message by default:
@BODY Message body, possibly munged by previous modules.
@HEADERS Current message headers.
@OLDHEADERS Original message headers.
$SUBJECT Original subject line.
$MAINTAINER Maintainer of this gateway.
Returns true if sending the mail succeeded, false if it failed.
Implements all methods related to posting articles. The following methods are included:
Posts the article using the NNTP POST command. Under most circumstances, this is how you want to post an article. SERVER, if specified, may be either the name or IP address of a news server to use for posting or a reference to an open Net::NNTP connection to be used. undef is returned on success; the error message is returned on failure.
Before this method is called, the message headers should be rewritten into ones appropriate for a news message; this is one of the purposes of the modules.
Posts the article using the NNTP IHAVE command. This connects to a news server as a transfer server rather than as a client; under most circumstances this is not what you want. Unless you already know what IHAVE is, you may not want to use this. In order to use this posting method, the article already has to have all necessary headers to be a properly formatted news article, including Path and Message-ID; unlike POST, IHAVE will not automatically fill in missing headers.
As above, SERVER if specified may be either the name or IP address of a news server to use for posting or a reference to an open Net::NNTP connection. undef is returned on success; the error message is returned on failure.
Posts the article by running a program and passing the article to it on
stdin. This can be used to post via inews or rnews if you so wish,
although using the post() method is recommended under most circumstances.
The arguments to the command should be a list of strings; this command
does not fork a shell so shell metacharacters will not be interpreted as
such.
If the program exits with status 0, success is assumed and this method returns undef. If the program exits with non-zero status or some other error occurs, this method returns the output of the program (both stdout and stderr) and an error message. The output and error message will be newline-terminated.
In addition to the failure messages generated by modules (listed above),
the following messages may be generated by the News::Gateway core code and
would be passed to the error() method.
A method was called that doesn't exist in the core code and was unable to be loaded from a module. Chances are you made a typo when referring to a News::Gateway method.
News::Gateway attempted to open a file (probably the configuration file) and failed for the given reason.
One of the modules (either rewrite or utility) you used requires an external Perl module and was unable to load it.
You told modules() to load a module that has no hooks registered.
News::Gateway has to know about every module that it can use; you can't
just copy in a new module with the others and have it work. Instead, put
the module in the modules directory before the build process.
Many. This is an alpha release, with all that's implied by that in terms of bugs, interface instability, and other such concerns. Use at your own risk, but if you encounter a bug please let me know. Patches in particular are gratefully accepted.
If you are interested in being notified of new releases, helping with development, developing, testing, or using new modules, or even just getting tips on how to use this module, there is a mailing list available. To subscribe, send mail to majordomo@eyrie.org with:
subscribe gateway-users
in the body.
New patches and modules are always welcome, whether from people on the list or not, and can be mailed to the address listed below.
Russ Allbery <rra@stanford.edu>.
News::Gateway started as a simple mail to news Perl script, written right after the cs.utexas.edu mail to news gateway closed and just a quick hack for the use of people who had previously used that gateway to post to rec.arts.comics.creative and alt.comics.lnh. Over time, it became a general mail to news gateway with a fairly broad set of features.
Eventually, it became obvious that it really should have been a module rather than a script, and it also became clear that it could be used as the core of a robomoderator. Since there was no real general-purpose robomoderation software around that was nearly as flexible as I thought a package should be, I decided to be ambitious and try to solve the general problem.
Then I took a complete detour to rewrite the entire thing again to use Andrew Gierth's News::Article class, which is what I should have been doing from the beginning. And now, finally, it's getting closer to what I'd envisioned.