—- plugin —-

description: Send campaigns that automatically use content from RSS or Atom feeds.
author : Duncan Cameron
type :
compatible : phplist 3
depends :
conflicts :
similar :
tags :

sourcerepo : https://github.com/bramley/phplist-plugin-rssfeed
donationurl:

This plugin lets you create campaigns that include content from RSS and Atom feeds and are automatically repeated at regular intervals.
So, for example, you can have a campaign that is sent every day at 09:00 which contains items from an RSS feed that were published in the previous day.

See the README file on the GitHub page https://github.com/bramley/phplist-plugin-rssfeed for how to install and configure the plugin.

The plugin adds a tab to the Send a campaign page on which you specify the URL of the RSS feed.
You should validate the URL first, by opening it in a browser, to ensure that it is valid and returns what you expect.
Additionally you can choose whether feed items should be displayed in ascending order of publication date (oldest first) or in descending order (latest first).
Optionally, you can also enter a custom RSS template specifically for this campaign. This will be useful it you want to include placeholders for custom tags.

You must include the [RSS] placeholder in the message area on the Content tab. You can include other markup if necessary. Alternatively The [RSS] placeholder can be included in the message template instead of the message content.

The plugin uses the standard phplist repeat processing to automatically create and schedule a new campaign.
The repeat interval that you select is also used by the plugin in selecting the feed items to include in a campaign. For a repeat interval of one day the plugin selects feed items that have been published in the past day. Similarly for the other repeat intervals.

To enable repeat processing add this line to your config.php file

define('USE_REPETITION',1);

phplist has a “stop sending after” date for each campaign. You need to ensure that this period is longer than the repeat interval by adding a line to your config.php file, otherwise phplist might consider that a repeated campaign has already reached the “stop sending” date. This value sets the interval to 6 months, see config_extended.php for other values.

define('DEFAULT_MESSAGEAGE', 15768000);

When composing a campaign, on the Scheduling tab you need to enter

• the embargo date/time
• the repeat interval
• the repeat until date/time

Together these provide the delivery schedule for the RSS campaign. The embargo is when the first campaign will be sent. After then it will be repeated at the repeat interval keeping the same time of day/day of week. phplist will continue repeating the campaign up to the repeat until date.
Each campaign will include recent feed items in the content. If there are no recent items then the campaign will not be sent.

The repeat frequency can be hourly, daily, weekly, fortnightly, or four-weekly. Which interval is suitable depends on the content that you are using and how often it changes. Probably daily or weekly are the most suitable.

When the campaign is being sent, the plugin selects feed items based on the repeat interval:

• 'day' - items whose published date is within the past 24 hours
• 'week' - items whose published date is within the past 7 days
• 'fortnightly' - items whose published date is within the past 14 days
• 'four-weekly' - items whose published date is within the past 28 days

The plugin selects items to be included using the items' publication date, the embargo date/time of the campaign, and the repeat interval. phplist treats the embargo date/time as being in the database server's local time zone. When fetching items from a feed the plugin converts the published date/time to the server's local time zone so that it can be compared directly with the embargo.

As an example, if a campaign repeats at 09:00 each day, then it will include items published from 09:00 the previous day, with the embargo time and items published times being treated as being in the server's local time zone. Feeds do not always provide items in order of publication date, so this will not necessarily select the items most recently provided.

If there are insufficient items to be included then the plugin will modify the campaign's embargo date/time by adding the repeat interval instead of allowing phplist to send a campaign with no items. For example, a daily campaign at 09:00 will be changed to 09:00 on the next day, with the campaign status remaining at “submitted”. The plugin adds an entry to the phplist event log when it does this.

The subject of the message can automatically include the title of the latest feed item using a placeholder

• [RSSITEM:TITLE]

So, for example, when creating the campaign the subject is entered as:

but the emails are sent with the title of the latest feed item

Additionally, when there is more than one item in the generated email, some optional text can be appended.
For example, specifying the additional text , and more the emails are sent with a subject

The additional text is entered within the RSS group on the Settings page.

Further placeholders can be used within the subject or the optional text

• [RSS:N] is replaced by the number of feed items.

• [RSS:N-1] is replaced by the number of feed items less 1. Because the result can be 0 when there is only one feed item, this is of use only in the additional text, e.g. and [RSS:N-1] more articles

When phplist is building the email to be sent the plugin replaces the [RSS] placeholder in the message content by the available feed items, each of which is formatted using an RSS template. The default RSS template is specfied on the Settings page, but each campaign can have a custom RSS template, which is entered on the RSS tab.

The template consists of html containing placeholders that will be replaced by fields from the item:

• [CONTENT]
• [TITLE]
• [URL]
• [PUBLISHED]

The source of the content depends on the type of feed. For an RSS feed the description field is used if present, otherwise the content field. For an Atom feed the summary field is used if present, otherwise the content field.
The published date is formatted using a configuration setting. The default is to display as dd/mm/yyyy hh:mm.

Additionally, placeholders for custom tags can be included. The placeholders must match exactly the custom tags entered on the Settings page.

Using the default template, a formatted item has the item title as a link, with the published date beneath, followed by the item content and a horizontal line. As an example

As the formatted feed items are within the message content, which itself may be within a message template, the RSS item template can use styles defined in the message or the message template.

These placeholders are available only in the RSS item HTML template, not generally within the message or message template.

The template is on the Settings page in the RSS Settings group.

To include items from the feed in a test message you will need to have retrieved some feed items. After entering the feed URL you can save the campaign, retrieve feed items, and then switch back to editing the campaign to send a test message.

Depending on when you create the campaign, the embargo, and the repeat interval, feed items that are retrieved might or might not be included in that first campaign. This depends on whether an item's published date is within the window for the campaign (later than the embargo - repeat interval).

For example, if you create a campaign with a daily repeat several days in advance of the embargo then any feed items retrieved will not be included in the first campaign because their published date is too early.

But if you create a campaign with a weekly repeat two days in advance of the embargo, then any feed items can be included in the first campaign, because their published date is within one week of the embargo.

To avoid confusion when sending a test message, the plugin will not select both items that can be included in the first campaign and items that will not included in the first campaign.

It will select only items that can be included in the first campaign, if there are any. If there are no such items then it will select items that will not be included in the first campaign. If there are none of those then some sample items will be used.

It will also display a warning message on the RSS tab when a test message would contain items that will not appear in the first campaign.

The plugin treats items as being in HTML format and will generate the plain text version of the RSS content from the HTML version using a phplist function HTML2Text().

The plugin needs to periodically get new items from the feed. This can be done manually, using menu Campaigns > Fetch RSS items, or it can be automated using a cron job. The command will be similar to this (but adjust for the directory in which phplist is installed and the command to run php)

php /home/xxx/public_html/lists/admin/index.php -p get -m RssFeedPlugin -c /home/xxx/public_html/lists/config/config.php

You can also run the command as a remote page if your cron job is limited to fetching web pages and cannot run linux commands. The URL for the remote page is

https://mysite.com/lists/admin/?page=get&pi=RssFeedPlugin&secret=xxxxxxxx

where xxxxxxxx is the “Secret for remote processing” value from the phplist Settings page.

The page can be fetched using a linux command such as wget or curl.

wget -O rssget.out 'https://mysite.com/lists/admin/?page=get&pi=RssFeedPlugin&secret=xxxxxxxx'

Alternatively, some third-party cron job services can be used.

You do not need to run the command very often, maybe once or twice a day should be sufficient. However to ensure that a campaign is sent with the most recent items it should be run just before processing the message queue.

You can view feeds and their items using menu Campaigns > RSS feeds

A feed is shown as active if it is being used by a campaign that has “draft” or “submitted” status.

The number of items is a link to a page that displays the items' content.

Once feed items have been included in a campaign there is no need to keep them in the phplist database. To delete items older than a number of days use the menu Campaigns > Delete outdated RSS items.

The command can also be automated using a cron job. The command will be similar to this (but adjust for the directory in which phplist is installed). The -d parameter is the number of days for which to keep feed items.

php /home/xxx/public_html/lists/admin/index.php -p delete -m RssFeedPlugin -d 99 -c /home/xxx/public_html/lists/config/config.php

You can also run the command as a remote page if you have a limited cron capability. This approach fetches the page using a linux command such as wget or curl. The URL requires the Secret for remote processing value from the phplist Settings page. The days parameter is the number of days for which to keep items. This method requires phpList 3.3.7 or later.

wget -O rssdelete.out 'https://mysite.com/lists/admin/?page=delete&pi=RssFeedPlugin&days=99&secret=xxxxxxxx'

The plugin can be used with the View in Browser plugin to provide a link that displays the campaign in a browser. You should use the placeholders provided by the View in Browser plugin [VIEWBROWSER] and [VIEWBROWSERURL].

To avoid any problem with incompatible plugin versions you should upgrade to the latest version of this plugin and of the View in Browser plugin.

In addition to the standard tags in an RSS feed, the plugin can capture the values of custom tags, and also attributes of custom tags. You can then use the custom tag in the RSS template.
The set of custom tags needs to be entered on the Settings page, one tag per line. Each custom tag should be specified exactly as it appears in the RSS feed.

In an RSS template you can include a placeholder for the custom tag

[cppopvault:img]

To capture an attribute of a custom tag append “@attribute” to the tag. If the feed includes

<media:thumbnail url="https://zak.io/assets/images/posts/2020-02-22-share-unpublished-scratch-projects-with-scratch-project-viewer/teaser1.png"/>

then use the placeholder

[media:thumbnail@url]

We will create an RSS campaign for a feed that updates often, so that a repeat interval of one hour can be used.

http://feeds.bbci.co.uk/news/uk/rss.xml?edition=uk

To test the sending of the RSS campaign, create a list that contains a few of your own email addresses.

Then create a campaign.
On the Content tab, enter [RSS] in the message area. You can also include other layout in the message.
On the Lists tab select the new list that you have created
On the Scheduling tab, choose “hour” for the repeat interval, and “repeat until” the end of the current day.
Leave the embargo as the default (the start of the current hour).

On the RSS tab enter the feed URL
http://feeds.bbci.co.uk/news/uk/rss.xml?edition=uk

On the Finish tab submit the campaign.

You can now manually fetch items from the feed using the menu System > Fetch RSS items and then view the items using menu Campaigns > RSS feeds. Confirm that some items have been received whose published date is within the past hour. If not, then periodically repeat this step until there are some items.

If all looks ok then process the queue through the browser. Because the campaign's embargo is in the past it will be sent immediately and will include some of the feed items.
When the message queue has been processed you can check that a new campaign has been created. The Active view of the Campaigns page should show a new campaign whose embargo is one hour after that of the original campaign.

You can then repeat the Fetch RSS items and process queue periodically. So long as the feed supplies new items then each campaign will be sent with recent items and a new campaign created.

Alternatively you can create cron jobs to automate the RSS item retrieval and the phplist queue processing. Create a cron job that runs processqueue once an hour on the hour, and then shortly after, say at 3 minutes past the hour.

php /home/xxx/public_html/lists/admin/index.php -p processqueue -c /home/xxx/public_html/lists/config/config.php

Create another cron job that runs the plugin's get page once an hour just before process queue, say at 58 minutes past the hour

php /home/xxx/public_html/lists/admin/index.php -p get -m RssFeedPlugin -c /home/xxx/public_html/lists/config/config.php

The plugin adds an RSS group to the Settings page where you can enter:

• The minimum number of items to be included in a campaign. The default is 1.
• The maximum number of items to be included in a campaign. The default is 30.
• The item HTML template.
• Text to append when the title of the latest item is used in the subject.
• Additional feed elements that should be included in each item's data.
• The format to be used to display the published date.
• Whether to filter item content for whitelisted tags only. The default is to filter.
• Whether to generate special content when an item's URL field is to YouTube or to a pdf file. The default is not to generate the special content.
• The maximum size of the HTTP body of the feed response.
• Whether to use the summary content or full content for the [CONTENT] placeholder.

The set of whitelisted tags are

'audio', 'video', 'source', 'dt', 'dd', 'dl', 'table', 'caption', 'tr', 'th', 'td', 'tbody', 'thead', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'strong', 'em', 'code', 'pre', 'blockquote', 'p', 'ul', 'li', 'ol', 'br', 'del', 'a', 'img', 'figure', 'figcaption', 'cite', 'time', 'abbr', 'iframe', 'q', 'sup', 'sub'

When an item's URL field (taken from the link element) is a URL to a YouTube video or to a pdf file then the plugin can generate special content instead of using the summary, description or content elements.

<link href="https://www.youtube.com/watch?v=99999999"/>
<link href="https://example.com/document.pdf"/>

For a YouTube video the content will be an iframe element that refers to the video. This is not really useful because many email clients will not display an iframe element.

<iframe width="560" height="315" src="//www.youtube.com/embed/99999999" frameborder="0"></iframe>

For a pdf file the content will be a link to the file.

<a href="https://example.com/document.pdf" target="_blank">https://example.com/document.pdf</a>

See the GitHub page https://github.com/bramley/phplist-plugin-rssfeed