Friday, November 9, 2012

Dada Mail v6.0.0 Alpha 1 Released

This is the 6th MAJOR release of this app I've made since December of 1999. I think that's pretty wild. Github tells me that, compared to the last stable version of Dada Mail (v5.2.1) there has been: 353 changed files with 18,740 additions and 16,697 deletions.  Goodness.

One of the major focuses on this release has really nothing to do with Perl, but  JavaScript - I managed, somehow to port all the Prototype/Scriptaculous code to jQuery. Sadly, the Prototype project seems to be dead. But gladly, the jQuery code that's now part of the app is pretty easy to read.

To those interested, here's a rundown of the differences between the v5 and v6, as well as the direct link to install,






Dada Mail 5 to Dada Mail 6 Upgrade Guide


Focus

The focus of this major release of Dada Mail is to make room for growth of the app, by creating a better foundation for future features.

JavaScript UI/UX Breathing/Growing Room

Dada Mail's UI/UX has been migrated to jQuery, from Prototype, where development has all but ceased.
The vast majority of Dada Mail's extra functionality through JavaScript is done in an unobtrusive way. A great example of new features allowed with this JS library move is the rewritten and redesigned Tracker plugin.

Towards a new web framework

Dada Mail is written with the thinnest of veneer of a web framework, and the next major version of Dada Mail may be a migration to a more robust web framework such as CGI::Application, Catalyst, or Mojolicious. v6 of Dada Mail moves the various components that make up Dada Mail (Perl code, JavaScript code, templates, images, stylesheets) into consolidated bundles of files and directories that are more sensible than previous versions.

Database Schema

There are no changes in any of the database schemas between Dada Mail 5 and Dada Mail 6.
Upgrading is fairly straightforward, with no need for any Dada Mail 5 to Dada Mail 6 SQL upgrade scripts.
When using the Dada Mail Installer (which you should use, when installing or upgrading), you will definitely want to create a new .dada_config file, when asked.
There is one new config variable, $SUPPORT_FILES that needs to be set up correctly. If you do not make a new .dada_config file, or install manually, without putting in this new variable, the upgrade will not work.

Perl Version Requirements

This is currently no change in the Perl Version requirement of v5.8.1. This may change, before a stable release of v6 is released, most likely to v5.10.x of Perl. We're still working on that.

.dada_config file

The .dada_config file that's generated by the included installer has undergone a few changes.

Removed variables: $FILES, $TEMPLATES, $TMP, $BACKUPS, $ARCHIVES, $LOGS

These variables were added to the .dada_config file, but are almost kept the same as the default, ala:
        $FILES                    = $DIR . '/.lists';
        $TEMPLATES                = $DIR . '/.templates';
        $TMP                      = $DIR . '/.tmp';
        $BACKUPS                  = $DIR . '/.backups';
        $ARCHIVES                 = $DIR . '/.archives';
        $LOGS                     = $DIR . '/.logs';
Instead of taking up extra space and line noise, these files are simply added to the dada/DADA/Config.pm file as the default. You may still add these variables in the .dada_config file to change their default, but it's not really recommended. The, $DIR variable has been added to the dada/DADA/Config.pm as a true config variable, so you do not want to have the first line read,
        my $DIR         = '/home/account/.dada_files';
Simply,
        $DIR         = '/home/account/.dada_files';
Will be fine. Keeping, my will probably break things.

Added variable: $BACKEND_DB_TYPE; Removed variables: $SUBSCRIBER_DB_TYPE, $ARCHIVE_DB_TYPE, $SETTINGS_DB_TYPE, $SESSION_DB_TYPE, $BOUNCE_SCORECARD_DB_TYPE, $CLICKTHROUGH_DB_TYPE

For the SQL backend, all these variables are already set the same, ala:
        $SUBSCRIBER_DB_TYPE       = 'SQL'; 
        $ARCHIVE_DB_TYPE          = 'SQL'; 
        $SETTINGS_DB_TYPE         = 'SQL'; 
        $SESSION_DB_TYPE          = 'SQL'; 
        $BOUNCE_SCORECARD_DB_TYPE = 'SQL';
        $CLICKTHROUGH_DB_TYPE     = 'SQL';
For the Default backend, they're set like this:
        $SUBSCRIBER_DB_TYPE       = 'PlainText'; 
        $ARCHIVE_DB_TYPE          = 'Db'; 
        $SETTINGS_DB_TYPE         = 'Db'; 
        $SESSION_DB_TYPE          = 'Db';
        $BOUNCE_SCORECARD_DB_TYPE = 'Db';
        $CLICKTHROUGH_DB_TYPE     = 'Db';
To reduce line noise, $BACKEND_DB_TYPE is being introduced. If set to, SQL, it'll set the other variables accordingly. Same with setting this variable to, Default.

%SQL_PARAMS: removed table names

Remove to further lesson the line noise.

Installer

Database Connection, Bounce Handler POP3 Connection Testers

The installer now has inline testers for both your Database Connection, and the Bounce Handler's POP3 Connection, to help you make sure these work, before committing to an installation.

Installer can now grab information from the previous installation

When using the installer to upgrade a current Dada Mail, it'll able to grab some of the configuration information from your previous installation: Program URL, Support Files Directory, Dada Mail Root Password, Backend Options, Plugin/Extensions that were previously installed, and your WYSIWYG options.
(If you added additional configuration variables after an install has completed, you will still have to re-add these manually)
This can help save time during an upgrade and ensure that everything goes smoothly. You can also re-enable the installer, after a successful initial installation, to change your Dada Mail configuration - for example: add/remove a plugin.

Template Changes

"templates" directory move

In v5, template files lived in the directory:
dada/DADA/Template/templates
They are now located in:
dada/templates
It's hoped that this directory move will make it easier for people to find the templates directory. This directory is still in the, dada directory, instead of, say, the, static directory since template files are themselves little, simply programs, that need to be interpreted, and aren't themselves actually static.

Plugins: Templated out

There was a voluminous amount of inline HTML in the plugins/extensions that are shipped with Dada Mail. This HTML is now saved in separate template files, which you can find in,
dada/templates/plugins
Each plugin has its own directory for its templates, eg: dada/templates/plugins/tracker, etc.
The Scheduled Mailings plugin still has program generated and inline HTML. It's a big mess.

JavaScript, images and css files removed from templates directory, relocated in, "static" directory

Many static files that used to reside in the, templates directory have been moved to the, static directory, to be served directly from the webserver. These include JavaScript, image and cascading stylesheet files.
This change should help performance of the app. These files are also not cached in Dada Mail's Screen Cache, as they're going to be served faster by the webserver simply as static files.

Inline JavaScript removed from template files

The vast majority of JavaScript that may have been found inline in the template files has been removed. You will now find that code in the, static/javascripts/dada_mail.js file.

JavaScript

Migration from Prototype/Scriptaculous to jQuery/jQueryUI

Most of the JavaScript that relies on the Prototype/Scriptaculous libraries has been migrated to jQuery/jQueryUI. Some remains, and jQuery has been configured in, NoConflict mode. We hope to move all Prototype/Scriptaculous code to jQuery eventually.

Static Files directory location, $SUPPORT_FILES config variable, Installation and Configuration

Dada Mail now separates out its static files needed for the app, from the dynamic template files. These static files ship with the app under the directory,
dada/static
When you install Dada Mail using the included Installer, this entire directory will be copied to the publicly accessable directory that you specify, so that they may be served directly by the webserver. JavaScript, images and cascading style sheets will be served from this directory.
In previous versions of Dada Mail, these files were served via Dada Mail, creating a new running instance of the app per request, which is slow, awkward and confusing.
Both the server path, and URL to this directory will be saved in your .dada_config file, under the varible, $SUPPORT_FILES. This variable is required to be correctly configured, for Dada Mail to work.
This does create an additional thing that needs to be configured for Dada Mail to be successfully installed, but we feel this extra step is worth it, for performance and for future features of the app.

Clickthrough/Message Opens, etc tracking support dropped for Default Backend

The Tracker plugin and all of its functionality now requires the use of one of the SQL backends.

List Control Panel Screens

Mass Mailing >> Send a Message

Options for Mass Mailing, including File Attachments, Archive Options, etc have been moved to a tabbed interface.

Membership >> View

Breakdown by Domain Graph

The Breakdown by Domain graph, that used to be in the Tracker plugin has been moved to the Membership - View screen. Clicking on any of the pie slices will perform a search on the domain you have selected.

Search Autocomplete

The Subscriber search form has autocomplete capabilities, which searches and matches on email addresses

Membership >> Recent Activity

Subscription Trends graph

The Recent Activity screen now has a graph labeled, Subscription Trends. This graph shows daily, as well as cumulative data on Subscriptions and Unsubscriptions on your mailing list.

WYSIWYG editors and Template Tags

In version < 6 of Dada Mail, when template tags like,
        <!-- tmpl_var list_settings.list_name -->
are entered in the rich text (rather than source view) of the WYSIWYG editor, the "<", and, ">" characters are converted into their HTML entities, so the source of your message now looks like this:
        &lt;!-- tmpl_var list_settings.list_name --&gt;
Which breaks the tag, and you will see the original template tag, which will not be converted to its value.
In v6, Dada Mail will look for these types of conversions, and convert them back, fixing the template tag, to be parsed correctly.

Data Cache

Dada Mail now has a data cache, for things like generated JSON files. This data cache can be managed with the new, DADA::App::DataCache module and can be enabled/disabled using the $DADA_CACHE config variable. Old cached data will be removed periodically (1 hour), automatically. The data cache is located in the,
.dada_files/.tmp/_dada_cache
directory. The files, as well as the directory itself, may be safely removed at any time.

Plugins

Tracker

The Tracker plugin has been essentially rewritten, and its features have been expanded. All charts, graphs and maps generated are now interactive and mousing over various parts of the chart/graph/map will reveal additional information. The UI itself has been redesigned, as well.
The new charts/graphs/maps are powered by the Google Visualization API. Data, in the JSON format is created for Dada Mail using the Perl CPAN module, JSON (http://search.cpan.org/~makamaka/JSON/). This module will need to be installed on your hosting platform for these charts/graphs/maps to be created.
Although not part of the standard Perl Library, the module proves to be so useful, that it's usually already installed on many hosting platforms (like LWP, CGI, etc).

Expanded location-based information

As well as country-specific information, Tracker now supports showing information on a city-specific level.

City Reports by IP Address

Reports are now generated per country, breaking down then by city, and then by individual IP address that may have caused an event, (a message open, clickthrough, forward, archive view) and listed in chronological order.

Included City Geo IP Data

Dada Mail now comes with both a country Geo IP database, as well as a city-level Geo IP database. The city-level database is fairly large - ~20 megs., but it's utility is worth its size. Geo IP data is provided by http://maxmind.com, who also provide more accurate country/city data, on a subscription.
The distro of Dada Mail is much larger - around 20 megs. compressed, rather than ~7.5 megs for v5 of Dada Mail. This additional size is mostly from the bundled city-level Geo IP database.
Both the city, and country-level Geo IP databases can be found in the,
dada/data
directory.

Bounce Handler

Filename change: dada_bounce_handler.pl to bounce_handler.cgi

Make sure to update any cronjobs, links, etc.

Rules

The Bounce Handler Rules, used to match the type of bounced message that gets sent to the bounce handler, has been moved out of the code and into its own file, which you may find at,
dada/data/bounce_handler_rules.pl
You may also copy the file, and place it in your,
.dada_files/.configs
directory, make your own changes/customizations, and the Bounce Handler will use your copy, instead of what ships with Dada Mail.

Bridge

Filename change: dada_bridge.pl to bridge.cgi

Make sure to update any cronjobs, links, etc.

List Email Mail Forwarding Support

You may now set up a List Email as a Mail Forward, that pipes a message directory to Dada Bridge, rather than as a POP3 email account, that is checked on a schedule via a cronjob. This should greatly help any lag between when the message is sent to the List Email address, and when the message is checked by Bridge, and sent out to the entire mailing list.

Clean Up Replies Filter - Disabled

This experimental feature doesn't work very well and has been disabled in v6.

Scheduled Mailings - DEPRECATED

The current scheduled_mailings.pl plugin, as it currently stands, needs a rewrite and isn't currently benefitting from the advances of code style, best practices and techniques offered by the rest of Dada Mail.
Unless rewritten, it will be removed in a future version of Dada Mail.

Extensions

dada_digest.pl - REMOVED


Cheers, 

Monday, July 30, 2012

Dada Mail v5.2.0 Released



Hello Everyone,

Dada Mail v5.2.0 has been released - install/upgrade/download it from:

http://dadamailproject.com/support/documentation-5_2_0/install_dada_mail.pod.html

The changelog is below:



Summary 5.2.0

Dada Mail v5.2.0 release focuses on WYSIWYG editors, which allow you to author your HTML email messages in rich text, right from within the List Control Panel.

Dada Mail now comes bundled with three different WYSIWYG editors: CKEditor, Tiny MCE and FCKeditor. KCfinder, an image/file browser/uploader is also bundled. Dragging and Dropping images to be used in your email messages is also supported in Dada Mail. All these utilities can be installed easily using the included Dada Mail Installer.

We've streamlined the Send a Message and Send a Webpage screens to better utilize and more cleanly present these editors, while still allowing you full flexibility on how you author your messages: We've moved to a tabbed interface for the mailing list message, so that you can easily toggle between your HTML Version and your PlainText version, without unnecessary scrolling.

If upgrading from any previous version of Dada Mail, we highly suggest Creating a new .dada_config file, when using the included Dada Mail installer, as many items in the starter .dada_config file have now changed.



Features 5.2.0

WYSIWYG/File Browsers


Dada Mail now supports and comes bundled with the following WYSIWYG editors:


  • CKEditor 
  • Tiny MCE 
  • FCKEditor 


Dada Mail also comes bundled with the file browser/file upload utility called, KCFinder. This utility allows file file browsing and file (image) uploading for all three included WYSIWYG editor.

Dada Mail Installer Support

The Dada Mail Installer now has the ability to configure and install all three WYSIWYG editors, as well as the file browser. Installing all these utilities is optional, but much easier than doing it manually. The editors and browser are bundled with Dada Mail in the, dada/extras/packages directory.

Added Support for Tiny MCE

Along with Support for CKeditor and FCKeditor, Dada Mail now supports the Tiny MCE WYSIWYG HTML editor in its Send a Message and Send a Webpage screens.


Support for Drag and Drop images (in some browsers)

Instead of having to go through the File Browser to upload an image, you can simply drag an image into one of the WYSIWYG editors. The image will then be saved in your file browser's file uploads directory and will be able to be used again for a future mass mailing.

This doesn't work for all browsers, but is confirmed to be support in Firefox 14+.



Discussion Lists: Experimental Support for Cleaning up long quoted replies of mailing list openings/signatures

Look for the option labeled, Attempt to clean up replies in Dada Bridge.



Tabbed Interface for PlainText/HTML Mailing List Messages

Dada Mail now allows you to toggle between the HTML, and PlainText versions of your mailing list message with a tabbed interface, instead of showing/hiding each type.



Subscription/Unsubscription Admin notices may now go to the entire mailing list

It's now an option to send the Subscription and Unsubscription Notices that usually go to the List Owner, to your entire list. This could be useful for a discussion list.

This feature was commissioned by David Smith for the Adytum Builders discussion list http://adytumbuilders.net/



Bugfixes 5.2.0

First Archive message shows incorrect link

https://github.com/justingit/dada-mail/issues/303



Lack of "message_body" tag in Mailing List Message Template leads to completely blank mailing list message

https://github.com/justingit/dada-mail/issues/304



Sending Preferences: Amazon SES has strange, blank blue box with nothing in it.

https://github.com/justingit/dada-mail/issues/305



Dada Bridge does not run the mailing monitor, when done checking messages

https://github.com/justingit/dada-mail/issues/306



Impossible to delete invalid-in-form email address in "Membership >> View" screen

https://github.com/justingit/dada-mail/issues/307

Changes 5.2.0

WYSIWYG Editors


Dada Bridge: Mailing List Message Template Tags for Discussion Lists Will be, "de-Personalized"

By default, the Mailing List Messages that ship with Dada Mail have template tags and links that point to various resources for subscribers of a mailing list. This includes changing the status of a subscription and logging into a subscriber's Profile. These links are personalized for the subscriber, so to avoid any additional entering of information/clicks.

These links cause havoc, though, on a discussion list, where messages are replied to and the original message is often quoted, and copied with the reply, as well as forwarded to people, off the list. The once-personal links now can be clicked by a third party, initializing such things as unsubscriptions. You may edit your Mailing List Message Templates to removed this personalization, but it is another step to set up a successful mailing list.

Starting with v5.2.0 of Dada Mail, these links will be, "de-Personalized" automatically for discussion lists. For example, a link that goes goes to a specific addresses' profile (or a form to log into a profile), that has their email address already embedded within the link, to pass automatically to Dada Mail:

ttp://example.com/cgi-bin/dada/mail.cgi/profile_login/user/example.com

will have the embedded email address removed:

http://example.com/cgi-bin/dada/mail.cgi/profile_login/

Similarily, Subscription and Unsubscription links:

http://example.com/cgi-bin/dada/mail.cgi/u/listname/user/example.com

will be changed to,

http://example.com/cgi-bin/dada/mail.cgi/u/listname/

To avoid problems.



FCKeditor listed as "Deprecated"


FCKeditor has now been deprecated in Dada Mail for many reasons: Internet Explorer 9 does not work well with FCKeditor, the included File Browser in FCKeditor is very buggy and has unfixed security issues and CKeditor is its natural replacement. We do not suggest using FCKeditor, but it is bundled with Dada Mail currently for backwards compatibility. We do bundle CKEditor and Tiny MCE, either of which will work better the FCKeditor.



$FCKEDITOR_URL and $CKEDITOR_URL Config variables - removed


Both the $FCKEDITOR_URL and $CKEDITOR_URL variables will not function as before - use the new $WYSIWYG_OPTIONS variable to configure these editors. Having these variables in your .dada_config file will not cause Dada Mail to error, yet.



"By default, reveal: PlainText Version|HTML Version" option removed


The Send a Message/Webpage screens now have a tabbed interface for HTML/PlainText messages, so this option has been removed. In Beatitude, both versions will be shown, by default.