Friday, November 9, 2012

Dada Mail v6.0.0 Alpha 1 Released

This is the 6th MAJOR release of this app I've made since December of 1999. I think that's pretty wild. Github tells me that, compared to the last stable version of Dada Mail (v5.2.1) there has been: 353 changed files with 18,740 additions and 16,697 deletions.  Goodness.

One of the major focuses on this release has really nothing to do with Perl, but  JavaScript - I managed, somehow to port all the Prototype/Scriptaculous code to jQuery. Sadly, the Prototype project seems to be dead. But gladly, the jQuery code that's now part of the app is pretty easy to read.

To those interested, here's a rundown of the differences between the v5 and v6, as well as the direct link to install,






Dada Mail 5 to Dada Mail 6 Upgrade Guide


Focus

The focus of this major release of Dada Mail is to make room for growth of the app, by creating a better foundation for future features.

JavaScript UI/UX Breathing/Growing Room

Dada Mail's UI/UX has been migrated to jQuery, from Prototype, where development has all but ceased.
The vast majority of Dada Mail's extra functionality through JavaScript is done in an unobtrusive way. A great example of new features allowed with this JS library move is the rewritten and redesigned Tracker plugin.

Towards a new web framework

Dada Mail is written with the thinnest of veneer of a web framework, and the next major version of Dada Mail may be a migration to a more robust web framework such as CGI::Application, Catalyst, or Mojolicious. v6 of Dada Mail moves the various components that make up Dada Mail (Perl code, JavaScript code, templates, images, stylesheets) into consolidated bundles of files and directories that are more sensible than previous versions.

Database Schema

There are no changes in any of the database schemas between Dada Mail 5 and Dada Mail 6.
Upgrading is fairly straightforward, with no need for any Dada Mail 5 to Dada Mail 6 SQL upgrade scripts.
When using the Dada Mail Installer (which you should use, when installing or upgrading), you will definitely want to create a new .dada_config file, when asked.
There is one new config variable, $SUPPORT_FILES that needs to be set up correctly. If you do not make a new .dada_config file, or install manually, without putting in this new variable, the upgrade will not work.

Perl Version Requirements

This is currently no change in the Perl Version requirement of v5.8.1. This may change, before a stable release of v6 is released, most likely to v5.10.x of Perl. We're still working on that.

.dada_config file

The .dada_config file that's generated by the included installer has undergone a few changes.

Removed variables: $FILES, $TEMPLATES, $TMP, $BACKUPS, $ARCHIVES, $LOGS

These variables were added to the .dada_config file, but are almost kept the same as the default, ala:
        $FILES                    = $DIR . '/.lists';
        $TEMPLATES                = $DIR . '/.templates';
        $TMP                      = $DIR . '/.tmp';
        $BACKUPS                  = $DIR . '/.backups';
        $ARCHIVES                 = $DIR . '/.archives';
        $LOGS                     = $DIR . '/.logs';
Instead of taking up extra space and line noise, these files are simply added to the dada/DADA/Config.pm file as the default. You may still add these variables in the .dada_config file to change their default, but it's not really recommended. The, $DIR variable has been added to the dada/DADA/Config.pm as a true config variable, so you do not want to have the first line read,
        my $DIR         = '/home/account/.dada_files';
Simply,
        $DIR         = '/home/account/.dada_files';
Will be fine. Keeping, my will probably break things.

Added variable: $BACKEND_DB_TYPE; Removed variables: $SUBSCRIBER_DB_TYPE, $ARCHIVE_DB_TYPE, $SETTINGS_DB_TYPE, $SESSION_DB_TYPE, $BOUNCE_SCORECARD_DB_TYPE, $CLICKTHROUGH_DB_TYPE

For the SQL backend, all these variables are already set the same, ala:
        $SUBSCRIBER_DB_TYPE       = 'SQL'; 
        $ARCHIVE_DB_TYPE          = 'SQL'; 
        $SETTINGS_DB_TYPE         = 'SQL'; 
        $SESSION_DB_TYPE          = 'SQL'; 
        $BOUNCE_SCORECARD_DB_TYPE = 'SQL';
        $CLICKTHROUGH_DB_TYPE     = 'SQL';
For the Default backend, they're set like this:
        $SUBSCRIBER_DB_TYPE       = 'PlainText'; 
        $ARCHIVE_DB_TYPE          = 'Db'; 
        $SETTINGS_DB_TYPE         = 'Db'; 
        $SESSION_DB_TYPE          = 'Db';
        $BOUNCE_SCORECARD_DB_TYPE = 'Db';
        $CLICKTHROUGH_DB_TYPE     = 'Db';
To reduce line noise, $BACKEND_DB_TYPE is being introduced. If set to, SQL, it'll set the other variables accordingly. Same with setting this variable to, Default.

%SQL_PARAMS: removed table names

Remove to further lesson the line noise.

Installer

Database Connection, Bounce Handler POP3 Connection Testers

The installer now has inline testers for both your Database Connection, and the Bounce Handler's POP3 Connection, to help you make sure these work, before committing to an installation.

Installer can now grab information from the previous installation

When using the installer to upgrade a current Dada Mail, it'll able to grab some of the configuration information from your previous installation: Program URL, Support Files Directory, Dada Mail Root Password, Backend Options, Plugin/Extensions that were previously installed, and your WYSIWYG options.
(If you added additional configuration variables after an install has completed, you will still have to re-add these manually)
This can help save time during an upgrade and ensure that everything goes smoothly. You can also re-enable the installer, after a successful initial installation, to change your Dada Mail configuration - for example: add/remove a plugin.

Template Changes

"templates" directory move

In v5, template files lived in the directory:
dada/DADA/Template/templates
They are now located in:
dada/templates
It's hoped that this directory move will make it easier for people to find the templates directory. This directory is still in the, dada directory, instead of, say, the, static directory since template files are themselves little, simply programs, that need to be interpreted, and aren't themselves actually static.

Plugins: Templated out

There was a voluminous amount of inline HTML in the plugins/extensions that are shipped with Dada Mail. This HTML is now saved in separate template files, which you can find in,
dada/templates/plugins
Each plugin has its own directory for its templates, eg: dada/templates/plugins/tracker, etc.
The Scheduled Mailings plugin still has program generated and inline HTML. It's a big mess.

JavaScript, images and css files removed from templates directory, relocated in, "static" directory

Many static files that used to reside in the, templates directory have been moved to the, static directory, to be served directly from the webserver. These include JavaScript, image and cascading stylesheet files.
This change should help performance of the app. These files are also not cached in Dada Mail's Screen Cache, as they're going to be served faster by the webserver simply as static files.

Inline JavaScript removed from template files

The vast majority of JavaScript that may have been found inline in the template files has been removed. You will now find that code in the, static/javascripts/dada_mail.js file.

JavaScript

Migration from Prototype/Scriptaculous to jQuery/jQueryUI

Most of the JavaScript that relies on the Prototype/Scriptaculous libraries has been migrated to jQuery/jQueryUI. Some remains, and jQuery has been configured in, NoConflict mode. We hope to move all Prototype/Scriptaculous code to jQuery eventually.

Static Files directory location, $SUPPORT_FILES config variable, Installation and Configuration

Dada Mail now separates out its static files needed for the app, from the dynamic template files. These static files ship with the app under the directory,
dada/static
When you install Dada Mail using the included Installer, this entire directory will be copied to the publicly accessable directory that you specify, so that they may be served directly by the webserver. JavaScript, images and cascading style sheets will be served from this directory.
In previous versions of Dada Mail, these files were served via Dada Mail, creating a new running instance of the app per request, which is slow, awkward and confusing.
Both the server path, and URL to this directory will be saved in your .dada_config file, under the varible, $SUPPORT_FILES. This variable is required to be correctly configured, for Dada Mail to work.
This does create an additional thing that needs to be configured for Dada Mail to be successfully installed, but we feel this extra step is worth it, for performance and for future features of the app.

Clickthrough/Message Opens, etc tracking support dropped for Default Backend

The Tracker plugin and all of its functionality now requires the use of one of the SQL backends.

List Control Panel Screens

Mass Mailing >> Send a Message

Options for Mass Mailing, including File Attachments, Archive Options, etc have been moved to a tabbed interface.

Membership >> View

Breakdown by Domain Graph

The Breakdown by Domain graph, that used to be in the Tracker plugin has been moved to the Membership - View screen. Clicking on any of the pie slices will perform a search on the domain you have selected.

Search Autocomplete

The Subscriber search form has autocomplete capabilities, which searches and matches on email addresses

Membership >> Recent Activity

Subscription Trends graph

The Recent Activity screen now has a graph labeled, Subscription Trends. This graph shows daily, as well as cumulative data on Subscriptions and Unsubscriptions on your mailing list.

WYSIWYG editors and Template Tags

In version < 6 of Dada Mail, when template tags like,
        <!-- tmpl_var list_settings.list_name -->
are entered in the rich text (rather than source view) of the WYSIWYG editor, the "<", and, ">" characters are converted into their HTML entities, so the source of your message now looks like this:
        &lt;!-- tmpl_var list_settings.list_name --&gt;
Which breaks the tag, and you will see the original template tag, which will not be converted to its value.
In v6, Dada Mail will look for these types of conversions, and convert them back, fixing the template tag, to be parsed correctly.

Data Cache

Dada Mail now has a data cache, for things like generated JSON files. This data cache can be managed with the new, DADA::App::DataCache module and can be enabled/disabled using the $DADA_CACHE config variable. Old cached data will be removed periodically (1 hour), automatically. The data cache is located in the,
.dada_files/.tmp/_dada_cache
directory. The files, as well as the directory itself, may be safely removed at any time.

Plugins

Tracker

The Tracker plugin has been essentially rewritten, and its features have been expanded. All charts, graphs and maps generated are now interactive and mousing over various parts of the chart/graph/map will reveal additional information. The UI itself has been redesigned, as well.
The new charts/graphs/maps are powered by the Google Visualization API. Data, in the JSON format is created for Dada Mail using the Perl CPAN module, JSON (http://search.cpan.org/~makamaka/JSON/). This module will need to be installed on your hosting platform for these charts/graphs/maps to be created.
Although not part of the standard Perl Library, the module proves to be so useful, that it's usually already installed on many hosting platforms (like LWP, CGI, etc).

Expanded location-based information

As well as country-specific information, Tracker now supports showing information on a city-specific level.

City Reports by IP Address

Reports are now generated per country, breaking down then by city, and then by individual IP address that may have caused an event, (a message open, clickthrough, forward, archive view) and listed in chronological order.

Included City Geo IP Data

Dada Mail now comes with both a country Geo IP database, as well as a city-level Geo IP database. The city-level database is fairly large - ~20 megs., but it's utility is worth its size. Geo IP data is provided by http://maxmind.com, who also provide more accurate country/city data, on a subscription.
The distro of Dada Mail is much larger - around 20 megs. compressed, rather than ~7.5 megs for v5 of Dada Mail. This additional size is mostly from the bundled city-level Geo IP database.
Both the city, and country-level Geo IP databases can be found in the,
dada/data
directory.

Bounce Handler

Filename change: dada_bounce_handler.pl to bounce_handler.cgi

Make sure to update any cronjobs, links, etc.

Rules

The Bounce Handler Rules, used to match the type of bounced message that gets sent to the bounce handler, has been moved out of the code and into its own file, which you may find at,
dada/data/bounce_handler_rules.pl
You may also copy the file, and place it in your,
.dada_files/.configs
directory, make your own changes/customizations, and the Bounce Handler will use your copy, instead of what ships with Dada Mail.

Bridge

Filename change: dada_bridge.pl to bridge.cgi

Make sure to update any cronjobs, links, etc.

List Email Mail Forwarding Support

You may now set up a List Email as a Mail Forward, that pipes a message directory to Dada Bridge, rather than as a POP3 email account, that is checked on a schedule via a cronjob. This should greatly help any lag between when the message is sent to the List Email address, and when the message is checked by Bridge, and sent out to the entire mailing list.

Clean Up Replies Filter - Disabled

This experimental feature doesn't work very well and has been disabled in v6.

Scheduled Mailings - DEPRECATED

The current scheduled_mailings.pl plugin, as it currently stands, needs a rewrite and isn't currently benefitting from the advances of code style, best practices and techniques offered by the rest of Dada Mail.
Unless rewritten, it will be removed in a future version of Dada Mail.

Extensions

dada_digest.pl - REMOVED


Cheers,