Submit By Mail

—- plugin —-

description: Allows messages to be submitted to phpList by email for distribution to particular mailing lists, even with attachments. No longer supported.
author : Arnold Lesikar
email : alesikar@mac.com
type :
lastupdate : 2015-11-29
compatible : 3.x
depends :
conflicts :
similar :
tags : mail_message_submission

downloadurl: https://github.com/arnoldle/phplist-plugin-submitByMailPlugin/archive/master.zip
bugtracker : https://github.com/arnoldle/phplist-plugin-submitByMailPlugin/issues
sourcerepo : https://github.com/arnoldle/phplist-plugin-submitByMailPlugin/
donationurl:

screenshot_img :

* You can give each/any of your lists in phpList it’s own email address for the submission of messages.

* As a security measure, you have the option of requiring senders to confirm each of their submissions.

* Messages may contain attachments, if your phpList installation is configured to allow attachments.

* Messages and attachments are checked for acceptability before they are processed.

* A list can be configured to save acceptable messages as a draft or to queue them immediately.

* The plugin generates scripts for mailbox pipes and cron jobs. No more complicated shell commands.

2015-5-28: 1.0b1 - The first release, not really ready for prime time
2015-6-15: 1.0b2 - The first real release, much better tested and more polished with easier installation. This release should be easily useable.
2015-6-16: 1.0b2.1 - Removed extraneous spaces in posts to ajax that seemed to interfere with email validation
2015-6-16: 1.0b2.2 - Now keep the display name of the sender instead of discarding it.
2015-6-16: 1.0b2.3 - Removed extraneous backslashes appearing in footer.
2015-6-16: 1.0b2.4 - Now indicate an error when an embedded image is included in the submitted message.
2015-6-22: 1.0b2.5 - Now make ajax calls through phpList. The plugin directory no longer needs to be inside the web root.
2015-6-23: 1.0b2.5a - Removed a warning about the missing .htaccess file in editlist.php. A big oops that prevented configuring lists.
2015-6-24: 1.0b2.5b - Now use a phpList function instead of a Mysql function to test for the existence of the 'escrow' table
2015-6-24: 1.0b2.6 - No longer use a system call in manual message collection. Can now run on Windows.
2015-6-26: 1.0b2.7 - Made sure of graceful failure if the browser has Javascript turned off. Warn the user.

2015-6-26: 1.0b2.7a - Had incorrect URL in Ajax call inside generateScripts.php. Also revised this page to work better without Javascript.
2015-6-27: 1.0b2.8 - Fixed protocol in link for confirmation of escrowed messages, when email to the sender is sent from a command line page.
2015-6-28: 1.0b2.9 - Creating a setting to define the HTTP protocol for plugin public pages. The settings are now in the 'General' category.
2015-6-30: 1.0b2.9a - Added cross-site scripting prevention to edit_list.php
2015-7-1: 1.0b2.10 - “Collect Messages Submitted by Email” is no longer a top menu option if manual message collection is set to 'off'
2015-10-7 1.0b2.11 - No longer disable plugin when the prerequisites are absent. Instead disable the plugin pages.
2015-10-8 1.0b2.11.1 - Added code to prevent the plugin from disabling phpList when the PHP iMap extension is lacking
2015-11-4 1.0b2.12 - Updated the plugin for compatibility with Phplist 3.2.x
2015-11-29 1.0b2.13 - Can now sign into POP server with either user name or submission
address depending upon what the server requires.

if you are running a later version of Phplist (3.06+), you should be able to install the plugin using the plugin installer provided by Phplist.

The plugin requires the PHP IMAP extension. Hosting services usually provide a PHP with the IMAP extension; so this prerequisite should offer most people little difficulty. You must also be running phpList with a secure connection (https, not http).

If your PHP binary does not contain the IMAP extension, you may disable phpList by installing the plugin. You can use the phpInfo() function to verify the presence of the IMAP extension. If you do install the plugin without the IMAP extension in your PHP binary, you may get an empty screen when you attempt to look at a phpList page. The plugin contains code that should prevent this, but the action of the plugin has not been tested without the IMAP extension.

If you are not running phplist on a secure connection, you can still install the plugin, but you will not be able to use it. Without a secure connection (https) all the plugin web pages will be disabled. You will not be able to configure any lists for email submission.

The plugin can be installed on Windows and should work there, but the plugin only generates shell scripts for Linux or Unix systems.

The PEAR mimedecoder is also required with supporting files, but these files are included with the plugin. It should not be necessary to find and edit a php.ini file.

You can install the plugin manually by downloading submitByMailPlugin.php and the submitByMailPlugin subdirectory from the “download” link above, and then uploading the files to the plugin directory of your phpList installation.

To use this plugin, you should have Javascript enabled in the browser. Without Javascript a list configuration cannot be validated, nor can messages be collected through the browser. Otherwise, the plugin will continue to function. Without Javascript you can still generate scripts for pipes or command line message collection.

A Very Important Note: if you do not install ALL the files in the submitByMailPlugin subdirectory, you may disable phpList. If the PEAR directory and its contents are not present in the submitByMailPlugin subdirectory, phpList will not function, and you will probably see just a blank screen when you attempt to view a phpList page.

After installing the plugin, check that the settings are appropriate for use. After selecting Settings in the phpList config menu, you will find the settings for this plugin at the bottom of the ‘General’ category, as shown in the image accompanying image.

The first setting has to do the location of the PHP command line binary. You do not have to enter anything here. This setting is relevant to the generation of command line scripts.That will be discussed later.

The second setting defines the HTTP protocol used in links to the public page where messages can be confirmed. Probably you can leave this as 'No' so that the page is accessed using 'https', Setting 'Yes' will set the HTTP protocol to be 'http' for this page.

The third setting is the number of days that you want messages to be held held for confirmation. The default is one day, but you can set any time up to 7 days

If the fourth option is set to ‘Yes’, you can use your browser to collect submitted messages similarly to the way the browser can be used to process the queue.

The fifth option is the time out for the internet connection to a mailbox in seconds. You can set any time up to 120 seconds. Set zero to accept the time out specified in the php.ini file for your setup. Normally you should not have to change this setting.

The next three settings specify the MIME types accepted by the plugin. These three settings are not relevant unless you have configured phpList to accept attachments.

Go to the dashboard. You should find the Plugins menu on the dashboard beneath the Campaign functions menu as seen in the accompanying image.

The selection Configure a List for Submission by Email will lead you to a page from which you can configure any of the lists that have been set up in your installation. On this page you will find a table of list ID’s, list names, and submission addresses.

Of course, the submission addresses all read None at the start, because no list has been configured yet. Click on a list name or a list ID to select the list you want to configure. That will bring you to the page shown in the next image.

Suppose you choose to configure the “newsletter” list. You be brought to the page shown in the image to the right. By Submission by mail allowed, click the radio button marked Yes. By Collection method, accept the POP3 with SSL/TLS default. Then into the box by Submission Address enter the email address to which you messages want to be submitted, for example, ournewsletter@example.com. Then enter into the password box, the password required to access the account corresponding to the submission address.

Into the next box labelled Mail Submission POP3 Server enter the mail server serving the submission account. That server might be perhaps “example.com” or “mail.example.com,” or something else. Consult the documentation for your hosting service. Please do not enter a port number; the plugin sets the port to 995, the standard port for secure POP3 service.

Many or most POP3 servers seem to use this address as the user ID for login. Nevertheless, there are some that use the user name, that is, the part of the submission address before the '@', as the login ID. This plugin therefore tries first to use the entire submission address as the login ID. If that doesn't work it uses the user name instead. The plugin will remember which ID works and will then use that ID for collecting messages.

If neither login works, you will be notified, so that you can correct the information before leaving the page.

What do you want to do with messages submitted to ournewsletter@example.com? Save is the default; messages submitted will be saved as a draft. Click the button marked Queue to have the messages queued immediately for distribution to the list. The next pair of buttons are labelled Confirm submission; Yes is the default — submitted messages will be held (“escrowed”) for confirmation by the sender who must be the list owner or a superuser.. Click No if you don’t think such confirmation is necessary.

Note: in terms of security it is a bad idea to queue messages directly without confirmation. You can do this if you want, but you had better be very confident that the submission address will not somehow become public, and that the list owner or a superuser cannot have their addresses spoofed. To repeat, if you allow messages to be queued without confirmation, any message sent to the submission address, spoofing the list owner or a superuser, will be distributed to your subscribers without review.

The last two selections on this page allow you to define a footer and select a template for messages sent through the submission address.

Click the Save button you save the information that you have entered. You will have to wait a short length of time while the plugin verifies the mail account that you have entered for the list. If everything is OK you will be returned to the previous page, where you can configure another list if you want. If the mail account does not verify, you will be notified and will remain on the same page, so that you can correct the information.

If you want messages to be submitted to your list through a pipe, then click the Pipe button by Collection Method. When you do this, the Password and Server boxes will disappear. The image shows the appearance of the page after clicking on the Pipe button.

The entries following the submission address are the same then as discussed before. When you click the Save button, the plugin will validate the form of the email address. You will be notified if something is wrong. You will remain on the page so that you can correct the information. Otherwise you will be returned to the previous page. Note: it is only the form of the address that is checked. We cannot easily verify that the specified address actually corresponds to an active email account served by a legitimate email provider. So please check the email address carefully; the plugin will not be able to catch every typo.

You can collect messages with a browser by selecting Collect Messages Submitted by Email in the Plugins menu on the dashboard. Note: this item will appear in the menu only if you have answered Yes in the setting Use browser to collect messages submitted by POP.

This will bring you to the page shown in the image. Click the button labelled Collect Messages. This will start the message collection. The button will disappear, and the table on the page will be continually updated as messages are collected from the different mailboxes by POP. When message collection is done, the Collect Messages button will be replaced by a View Event Log button so that you can view the results of message collection in the Event Log. If one or more connections are broken or time out, you will see an entry in red at the bottom of the table giving the number of these events that have occurred.

Collecting mail at scheduled intervals requires setting up a cron job with a terminal command. Similarly downloading messages from a mailbox as they are received requires messages to be piped to a terminal command from the mailbox. This plugin will create shell scripts that can be used for this purpose.

It is recommended that you let the plugin generate scripts; the commands required for a cron job or a pipe are quite complicated.

Begin by selecting Generate Scripts for Mailbox Pipes and Cron on the plugin menu available on the dashboard. This will bring you to the page shown below.

The top part of the page gives you detailed instructions for using the scripts generated by this page. In the form at the bottom of the page radio buttons allow you to select the script that you want to generate. You must also select the complete directory path where you want the script to be stored.

Note: The directory you specify must actually exist and it must be writable. If the directory is unsuitable, you will not be allowed to generate the script.

After the script is generated, you will be returned to this page, but with a message at the top of the page confirming creation of the script. You may also receive the following advice like the following (PHP version 5.2 is merely an example here!):

Usually you can safely ignore this message. If no path to a command line binary is specified in the plugin settings (the first setting), the plugin searches for the binary and takes the first binary that it finds, which may not be the binary used for the command line when you access your service through ssh on a terminal.

The scripts have been tested with PHP v5.2. You can expect them to work fine with that version or any later version, unless another plugin creates a conflict on account of the PHP version.

If there is a problem, go to the first plugin setting and enter the path to a modern PHP version.

NOTE: on this page you can also generate a script allowing you to automate queue processing with a cron job.

NOTE: message collection with a command line shell script will not work unless you have the setting Use browser to collect messages submitted by POP set to No.

When messages are escrowed, the sender, that is, the list owner or a super user — whoever sent the message, is sent an email containing a link to a web page. If the sender then clicks on the link, the message will be processed. That is, it will either be queued or saved as a draft, depending on the configuration of the list to which the message has been submitted.

The web page to which the sender is sent will give the outcome of that processing. If the sender does not respond by clicking on the link, the message will be held for the number of days specified in the hold time setting. After the expiration of the hold time, the message will be permanently deleted.

Any message targeting more than one list, will always be saved as a draft, regardless of the configuration of the lists targeted.

Whether arriving messages are to be escrowed, saved, or queued, they are checked for acceptability before they are processed. Messages must come from an administrator. An ordinary list owner is not allowed to send messages to a list he/she does not own. Superusers, of course, can send messages to any list. Messages from non-administrators are discarded without notifying the sender.

This plugin will not work without a secure https connection. This is not an arbitrary requirement. I had strongly considered incorporating the ability to configure the plugin for an unsecured connection. Then the realization dawned that you are doing your subscribers no favours to run phpList over an insecure connection – phpList should only be run with https!

In general you should require confirmation for all messages submitted by email, to avoid a hacker submitting messages under the guise of an administrator. As an additional level of security, the plugin requires that all message sent to multiple lists be held for confirmation regardless of how the individual lists are configured.

Unless the message is escrowed, a sender will be notified of the disposition of the submitted message. As discussed before, when a message is escrowed, the sender is sent an email containing a link that confirms the message.

All actions of the plugin are logged in the event log. When there is a problem with a submitted message, an error message will be logged in the event log. The error messages described here are entered into the Event Log, and if possible, a version of the error message will be returned to the sender. These error messages in general result in the message being discarded.

Some of the error conditions described here seem unlikely, but I’ve tried to address every problem that might conceivably occur while the plugin is trying to deal with a message.

1. Msg discarded: pipe not allowed for this list. This occurs when you attempt to pipe messages for a list not configured to allow it.

2. Msg discarded: cannot decode. The plugin cannot make sense of the message or the attached MIME.

3, Msg discarded: bad mailbox. The mailbox specified in the -e option does not match the submission address of the list addressed.

4. Msg discarded: no lists addressed. The address in the To: field of the message does not match the submission address of a list.

5. List <list name>: Msg discarded; unauthorized sender. The sender is not an administrator

6. List <list name>: Msg discarded: sent to list(s) sender does not own. The sender is an administrator but not a superuser, and he(she) has addressed a list other than the lists he or she owns.

7. List <list name>: Msg discarded; bad type for main message. The main message is allowed to be only text/plain or text/html. You cannot send a PDF except as an attachment.

8. List <list name>: Msg discarded; mime type not allowed. The mime type of an attachment does not correspond to an acceptable mime type as configured in the settings. The only multipart types allowed are multipart/mixed, multipart/alternative, and multipart/related. This is hard coded and cannot be changed.

9. List <list name>: Msg discarded; attachments not permitted. The message contains one or more attachments, but the phpList installation is configured to allow NO attachments.

10. List <list name>: Msg discarded; mime nesting too deep. An attachment may be multipart including its own multipart attachment, which in turn includes a multipart attachment…. This plugin allows no mime nesting more than three levels deep.

11. List <list name>: Msg discarded; inline type not allowed. An attachment that is not text/plain or text/html, has been found without a file name and with “inline” as the specified disposition.

This plugin cannot process an embedded image in a submitted message. When a message containing an embedded image is queued upon receipt, the following error message results: Message cannot be sent with unprocessed embedded image.

When a message containing an embedded image is saved as a draft, the following line is appended to the top of the message in red: Embedded images not allowed in email submissions. The image below cannot be displayed.

When an issue arises, please post a message on phpList-developers if you are a member of that list. Otherwise you can post a message to me through the “Report bugs” button the top of this page. The third option is to send a message to alesikar at my personal email provider, mac.com .