Experimental Dada Mail w/unicode ¡Support! Released

(this is a repost from here, 'cause I'm pretty stoked on it)


This is the first step in the localization project, since we can't very well translate Dada Mail if Dada Mail can't use the translations available.

I have to let this project rest for a little bit (and collect my wits - it was a very difficult step!) but any and all feedback is welcome, if you'd like to give this a spin - bug reports/problems of any kind are very much appreciated.

This version of Dada Mail should basically be able to support any language that can in the unicode characters set and UTF-8 encoding. Which, should be, well, a lot of them. It doesn't (Dada Mail), but where it fails? I don't know - but it's a good time to test and see where it's wrong.

For simple Euro-centric stuff, like this:

Je peux manger du verre, ça ne me fait pas mal.

It should be fine. For something a little more wild:

أنا قادر على أكل الزجاج و هذا لا يؤلمني.

(which should be Arabic)

Well, I can only go on if something visually looks correct :) Even this email is sort of a test - I don't know if it's going to work, or not - so, fingers crossed! If it does - we're on a good track, since Dada Bridge taking a random email, having it go through the system that's mostly tested using a very specific way of creating emails and coming out readable on the other side is a great big step - not even talking about the online archive, rss/atom feeds, twitter thingie, etc, etc, etc.

Here's the download to the version I'm now running at the Dada Mail support site:

http://github.com/downloads/justingit/dada-mail/dada-4_0_2-unicode.zip

http://github.com/downloads/justingit/dada-mail/dada-4_0_2-unicode.tar.gz

If you want to check it out via github, the branch is at:

http://github.com/justingit/dada-mail/tree/charset_work

To grab it with git, you have to do this:


git clone git://github.com/justingit/dada-mail.git
cd dada-mail
git fetch
git checkout --track -b your_local_branch_name origin/charset_work


Here's the explanation of all that:

http://groups.google.com/group/github/browse_thread/thread/71f944b925467ab6

There's a guide of what to expect with Dada Mail and unicode/UTF-8 you can read here:

http://dadamailproject.com/support/documentation-4_0_2-unicode/features-UTF-8.pod.html

Which I'll paste the contents of at the end of this message - but you may also want to compare it to the version of this doc for 4.0.2 STABLE:

http://dadamailproject.com/support/documentation-4_0_2/features-UTF-8.pod.html

(Long story short: 4.0.2 UTF-8/Unicode Support: "uhh...")

And, that's about it. This was a hard part of the project, since this is a 10+ y/o codebase - it very much pre-dates even unicode/UTF-8 support in Perl itself, so there's a reason, I guess, why the program was in such bad shape when it came to support it. Many,

many many bugs showed themselves, once this feature was asked for. I think a great majority of them have been solved.

Give it a spin if this interests you and if I can help out with anything, let me know,


--
Introduction

Dada Mail can speak UTF-8 and almost expects that everything else around it does, too.

That means:

• It treats everything it handles as UTF-8
• Everything it returns is in UTF-8
How To Have a Pleasant Experience

If you're installing Dada Mail for the first time, there's nothing you'll need to do, but below are some great guidelines on how to keep your lists configured, so you continue to have a good experience.

If you're upgrading, make sure your configuration reflects the advice below.

It's heavily advised to keep everything in Dada Mail speaking UTF-8 without any real exceptions.

Config Variable: $HTML_CHARSET

By default, the config variable, $HTML_CHARSET is set to, UTF-8

Keep it that way, same case (UTF-8) - same everything.

Dada Mail is only tested with the charset set this way.

Advanced Sending Preferences

Default Character Set

Set this as, UTF-8 UTF-8

Default Plain Text/HTML Message Encoding

There's really only a few choices recommended for Dada Mail.

• 8bit
Should work.

• quoted-printable

If you have any trouble with 8bit, try quoted-printable. Because of the amount of time that Dada Mail creates, tweaks, formats and templates out email messages, the encoding can potentially get mucked up.

This potential mucking-up is mitigated when Dada Mail uses quoted-printable encoding internally. This should be the default for email messages.

Encode Message Headers

Have this option checked.

SQL Backends

Database

PostgreSQL

Encoding for PostgreSQL databases is done when the database is created - make sure to create your database with a, UTF-8 encoding, like so:

CREATE DATABASE dadamail WITH ENCODING 'UTF-8'
MySQL

Nothing you'll have to do.

SQLite

Nothing you'll have to do.

DBM Files

DBM Files have no encoding support, but Dada Mail knows this and compensates.

Schema

MySQL

The MySQL schemas are set to create tables with an encoding of, UTF-8

PostgreSQL

Nothing has changed.

SQLite

Nothing has changed.

Drivers

The current support SQL backends, mysql (MySQL), Pg (PostgreSQL) and SQLite all have different ways to somewhat, "enable" their UTF-8 support.

• MySQL
add,

mysql_enable_utf8 => 1,
has been added to the $DBI_PARAMS hashref.

• PostgreSQL
add,

pg_enable_utf8 => 1,
has been added to the $DBI_PARAMS hashref.

• SQLite
add,

sqlite_unicode => 1
has been added to the $DBI_PARAMS hashref.

No explicit encoding/decoding is done in Dada Mail when saving/retrieving data. Hopefully, the drivers are UTF-8-aware enough.

Plugins/Extensions

The Plugins and Extensions that come with Dada Mail have not been as thoroughly tested as the main program. There's still warts.

Dada Bridge

Dada Bridge has a unique position needing to handle a lot of different stuff thown at it and deal with it gracefully. Dada Mail does, in fact, handle, any realistic character set/encoding you throw at it, but Dada Mail will convert messages it receives to its internal format, before it resends it out to your list.

This means the encoding of your choice (8bit or quoted-printable) and the charset of your choice (as long as your charset is, UTF-8)

Upgrading

You are potentially going to have problems.

Its possible that, since List Settings were never decoded/encoded correctly in past versions, they'll show up the program (once you've upgrade) incorrectly. The easiest thing to do is to edit the mistakes and resave the information. For most of the program, you're going to have to manually export the information and re-import it with the correct encoding, sadly. Dada Mail will probably fail gracefully with old information, but it's possible that you'll see squiggly charaters, instead of what you want to see. There's nothing in Dada Mail that will stop this from happening. If you experience it (from old information), we're not going to count it as a bug, but rather a known issue.

Problems?

Please let us know via the Support Boards:

http://dadamailproject.com/support/boards/

Or the developer mailing list:

http://dadamailproject.com/cgi-bin/dada/mail.cgi/list/dadadev/

Thanks!

See Also:

• The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)

http://www.joelonsoftware.com/articles/Unicode.html

• perlunitut - Perl Unicode Tutorial

http://perldoc.perl.org/perlunitut.html

• perlunifaq - Perl Unicode FAQ

http://perldoc.perl.org/perlunifaq.html



--

Post:
mailto:dadadev@dadamailproject.com

Unsubscribe:
http://dadamailproject.com/cgi-bin/dada/mail.cgi/u/dadadev/

List Information:
http://dadamailproject.com/cgi-bin/dada/mail.cgi/list/dadadev

Archive:
http://dadamailproject.com/cgi-bin/dada/mail.cgi/archive/dadadev

Developer Info:
http://dev.dadamailproject.com

Perl HTML::Template and UTF-8 Unicode

HTML::Template does not support file encoding:

#!/usr/bin/perl -w
use strict;
use Encode;
use HTML::Template;
my $template = HTML::Template->new(
filehandle => *DATA,
);
print Encode::encode('UTF-8', $template->output);
__DATA__
¡™£¢∞§¶•ªº

prints, ¡™£¢∞§¶•ªº (or something like that!)

In the example above, this makes sense, since we're printing on an open filehandle (even if it's only to our magical, DATA) that we didn't put a file layer filter thingy to. That's easy to fix:

#!/usr/bin/perl -w
use strict;
use Encode;

binmode DATA, ':encoding(UTF-8)';

use HTML::Template;
my $template = HTML::Template->new(
     filehandle => *DATA,
);
print Encode::encode('UTF-8', $template->output);
__DATA__
¡™£¢∞§¶•ªº

prints, ¡™£¢∞§¶•ªº, yay!


This also works if we want to just pass a reference to a scalar to HTML::Template:

#!/usr/bin/perl -w
use strict;
use Encode;
my $content    = "\x{a1}\x{2122}\x{a3}\x{a2}\x{221e}\x{a7}\x{b6}\x{2022}\x{aa}\x{ba}";
use HTML::Template;
my $template = HTML::Template->new(
     scalarref => \$content,
);
print Encode::encode('UTF-8', $template->output);

prints, ¡™£¢∞§¶•ªº, yay!

This doesn't work, if we want to just give it a name of a template file. This is really useful, since HTML::Template has a feature to allow you to search through a file structure (or at least an array of directories, looking for the file).

And this is where encoding madness begins.

Cause I know what you're thinking, just treat HTML::Template's output like information that's coming from outside your program (since, if you're using a template *file*, it kinda is).

So, all you need to do is decode (this is the WRONG WAY to solve the problem, but let's just make that mistake...) the return value of ->output, like this:

#!/usr/bin/perl -w
use strict;
use Encode;
my $content = "\x{a1}\x{2122}\x{a3}\x{a2}\x{221e}\x{a7}\x{b6}\x{2022}\x{aa}\x{ba}";

my $filename = 'utf8string.tmpl';

open my $fh, '>:encoding(UTF-8)', $filename or die $!;
print $fh $content;
close $fh;

use HTML::Template;
my $template = HTML::Template->new(
     filename => $filename,
);

my $output = $template->output;
$output = Encode::decode('UTF-8', $output);

print Encode::encode('UTF-8', $output);

prints, ¡™£¢∞§¶•ªº. Yes.

But... what if you have a variable (it is a templating system) and the variable in the param() you pass has UTF-8 strings? MUAHAHA!

#!/usr/bin/perl -w
use strict;
use Encode;
my $content = "
<!-- tmpl_var one -->
\x{a1}\x{2122}\x{a3}\x{a2}\x{221e}\x{a7}\x{b6}\x{2022}\x{aa}\x{ba}
";

my $filename = 'utf8string.tmpl';

open my $fh, '>:encoding(UTF-8)', $filename or die $!;
print $fh $content;
close $fh;


use HTML::Template;
my $template = HTML::Template->new(
     filename => $filename,
);
$template->param(
one => "\x{a1}\x{2122}\x{a3}\x{a2}\x{221e}\x{a7}\x{b6}\x{2022}\x{aa}\x{ba}",
);

my $output = $template->output;
$output = Encode::decode('UTF-8', $output);

print Encode::encode('UTF-8', $output);

Cannot decode string with wide characters at /System/Library/Perl/5.10.0/darwin-thread-multi-2level/Encode.pm line 162.


Bahahaha!

Take those decode/encode lines (I know it looks strange to one, right after the other ) and you'll still get a weird output:

¡™£¢∞§¶•ªº
¡™£¢∞§¶•ªº

Darned if you do/don't. Those two lines should have the same string. They don't. No amount of encoding/decoding is going to help.


The trick, other than tweaking HTML::Template's source to include file filter layer thingamabobs, is to decode the contents of the file it opens up.

How to do that.

Trolling through the HTML::Template mailing list archives leads to the idea of using a HTML::Template filter that matches everything, that then does our decoding:

#!/usr/bin/perl -w
use strict;
use Encode;
my $content = "
<!-- tmpl_var one -->
\x{a1}\x{2122}\x{a3}\x{a2}\x{221e}\x{a7}\x{b6}\x{2022}\x{aa}\x{ba}
";

my $filename = 'utf8string.tmpl';

open my $fh, '>:encoding(UTF-8)', $filename or die $!;
print $fh $content;
close $fh;


use HTML::Template;
my $template = HTML::Template->new(
     filename => $filename,
     filter => [
       { sub => \&decode_str, format => 'scalar' },
     ],
);
$template->param(
one => "\x{a1}\x{2122}\x{a3}\x{a2}\x{221e}\x{a7}\x{b6}\x{2022}\x{aa}\x{ba}",
);

my $output = $template->output;


print Encode::encode('UTF-8', $output);



sub decode_str {
my $ref = shift;
${$ref} = Encode::decode('UTF-8', ${$ref});
}

This sort of lines up all the data to be UTF-8 encoded and aware and all that stuff that the unicodefaqthingy perldoc tells you to do.

But, oh, it gets better.

DON'T use that filter trick thing if you're using a scalarref, or a properly encoded file handle! You'll get a nice error, like this:

HTML::Template->new() : fatal error occured during filter call: Cannot decode string with wide characters at /System/Library/Perl/5.10.0/darwin-thread-multi-2level/Encode.pm line 162.
at /Library/Perl/5.10.0/HTML/Template.pm line 1697
HTML::Template::_init_template('HTML::Template=HASH(0x1008aafb8)') called at /Library/Perl/5.10.0/HTML/Template.pm line 1238
HTML::Template::_init('HTML::Template=HASH(0x1008aafb8)') called at /Library/Perl/5.10.0/HTML/Template.pm line 1124



Brilliant.


So I don't know what the best advice is to give. If you're passing the template as a scalarref, DON'T use that filter, unless you want to, perhaps encode your template beforehand (which makes little sense?)

If it's a filename, use that filter trick perhaps (or edit the sourcecode of HTML::Template).

Perl, UTF-8 Email Messages, MIME::Enity and QuotedPrintable encoding

Some findings after much bashing of head:

#!/usr/bin/perl -w
use strict;

use MIME::Entity;
use Encode;

# My UTF-8 string -
# ¡™£¢∞§¶•ªº
# Basically using Mac OS X, just hold down the alt/option key and hit the 1 through 0 keys, in succession:
#
my $content    = "\x{a1}\x{2122}\x{a3}\x{a2}\x{221e}\x{a7}\x{b6}\x{2022}\x{aa}\x{ba}";

# Build the message, using MIME::Entity.
# MAKE SURE TO ALWAYS encode('UTF-8', 'string') BEFORE ADDING
# Always.

my $pt_entity = MIME::Entity->build(
 Type    => 'text/plain',
 Data    => Encode::encode('UTF-8', $content),
 Encoding => 'quoted-printable',
);



# MAKE SURE TO ALWAYS decode('UTF-8', 'string') BEFORE WORKING WITH STRING
# Always.
my $new_content = $pt_entity->bodyhandle->as_string;
$new_content = Encode::decode('UTF-8', $new_content);

# For example, we're just going to reverse it:
$new_content = reverse($new_content);




my $io = $pt_entity->bodyhandle->open('w');

# YES. You will will need to encode content using the bodyhandle. Always.
# Always.

$new_content = Encode::encode('UTF-8', $new_content);
$io->print($new_content); 
$io->close;
$pt_entity->sync_headers(
 'Length'      =>  'COMPUTE',
'Nonstandard' =>  'ERASE'
);


# And, that's it.



# Before using the content, decode
# Always.
my $result =  $pt_entity->bodyhandle->as_string;
$result = Encode::decode('UTF-8', $result);


# Always encode, before printing.
# Always.
#
# prints, ºª•¶§∞¢£™¡
print Encode::encode('UTF-8', $result);

The trick is to always, always, always encode your data, before creating any sort of entity using MIME::Entity and to always, always always decode the data you get using bodyhandle()

This workflow is strange, since you're told not to encode data, until you're ready to print it. I suspect there's some weird IO::File stuff going on with MIME::Entity (and friends), or, want to think of saving binary data, instead of characters when creating MIME stuff. I don't know.

If you do not encode before, MIME::Entity will barf, when using the quoted/printable encoding, but will probably be just fine with, "8bit" encoding.

This was a huge headache to figure out.

This will all seem to work out, if you don't do that first encode:

#!/usr/bin/perl -w
use strict;

use lib qw(/Users/justin/Documents/DadaMail/git/dada-mail/dada/DADA/perllib);

use MIME::Entity;
use Encode;

# My UTF-8 string -
# ¡™£¢∞§¶•ªº
# Basically using Mac OS X, just hold down the alt/option key and hit the 1 through 0 keys, in succession:
#
my $content    = "\x{a1}\x{2122}\x{a3}\x{a2}\x{221e}\x{a7}\x{b6}\x{2022}\x{aa}\x{ba}";

# Build the message, using MIME::Entity.
# MAKE SURE TO ALWAYS encode('UTF-8', 'string') BEFORE ADDING
# Always.

my $pt_entity = MIME::Entity->build(
 Type    => 'text/plain',
# Data    => Encode::encode('UTF-8', $content),
 Data    => $content,
 Encoding => 'quoted-printable',
);

my $s =  $pt_entity->bodyhandle->as_string;

$s = Encode::decode('UTF-8', $s);

# Let's do a little string manip:
$s = reverse($s);

$s = Encode::encode('UTF-8', $s);

print $s;


Cannot decode string with wide characters at /System/Library/Perl/5.10.0/darwin-thread-multi-2level/Encode.pm line 162.

exception_handler::die in Encode.pm at line 162
Encode::decode in test7.pl at line 28






So do that first encode, please. If you don't follow this formula, your prog may work, until that last encode:

 
#!/usr/bin/perl -w
use strict; 

use lib qw(/Users/justin/Documents/DadaMail/git/dada-mail/dada/DADA/perllib); 

use MIME::Entity; 
use Encode; 

# My UTF-8 string - 
# ¡™£¢∞§¶•ªº
# Basically using Mac OS X, just hold down the alt/option key and hit the 1 through 0 keys, in succession: 
#
my $content    = "\x{a1}\x{2122}\x{a3}\x{a2}\x{221e}\x{a7}\x{b6}\x{2022}\x{aa}\x{ba}";

# Build the message, using MIME::Entity. 
# MAKE SURE TO ALWAYS encode('UTF-8', 'string') BEFORE ADDING
# Always. 

my $pt_entity = MIME::Entity->build(
 Type    => 'text/plain', 
# Data    => Encode::encode('UTF-8', $content),
  Data    => $content, 
 Encoding => 'quoted-printable',
 );

my $s =  $pt_entity->bodyhandle->as_string; 

# NAW, we don't need that
# $s = Encode::decode('UTF-8', $s); 

# Let's do a little string manip: 
$s = reverse($s); 

# Well, that's silly! We don't need that one, either! 
# $s = Encode::encode('UTF-8', $s); 

print $s;


Wide character in print at /Users/justin/Desktop/test7.pl line 37.


And, you will do what I do, and bang your head, some more.

I couldn't fine any info on how to handle things like MIME::Entity and UTF-8 encoding, in the excellent articles available such as this one:

http://ahinea.com/en/tech/perl-unicode-struggle.html

and,

http://perlgeek.de/en/article/encodings-and-unicode

and,

http://juerd.nl/site.plp/perluniadvice


I have this article labeled as, "do not trust"

http://kbinstuff.googlepages.com/perl,unicodeutf8,cgi.pm,apache,mod_perla

Because it states,

6.1. Encode::encode/decode

For start, you should avoid using Encode::encode/decode/from_to to the greatest possible extent in your scripts. This will only lead to great confusion later. You may think you have gotten everything to work, but then a week later, you shall only add a little more functionality to your work and suddenly, everything falls apart and doodles will appear on your web pages.

I guess I understand what they mean - but you'll need to encode your UTF-8 stuff before it exits your program. Always. And, you have to decode UTF-8 info that goes out of your program. Always. How do you do this? Uh-huh, the Encode module.

Like it says in the perldoc for unicode. So, I don't know what this page is yabbering about. I'm sure, behind the scense, Encode is used when open files with a specific encoding:

http://perldoc.perl.org/perluniintro.html#Unicode-I/O

Which, by the way of features, is a pretty rad one.

ANNOUNCE: Dada Mail 3.1 alpha 3 Released

(I thought I'd re-post this announcement, since I don't think many people in the perl community really know about this large project. It IS NOT Modern Perl written, as it dates back to even before Perl 5.6, but it's a very large and complex Perl program, that's still sorta/kinda easy to install. (Royal) we try.

Dada Mail 3.1 alpha 3 Released

After a month and a half of more development from the last release, Dada Mail 3.1 alpha 3 has been released.

Download

tar.gz:

http://github.com/justingit/dada-mail/tarball/v3_1_0_alpha3-08_26_09

zip:

http://github.com/justingit/dada-mail/zipball/v3_1_0_alpha3-08_26_09

Changes

To get caught up in what's changed, please see:

Dada Mail 3.0 to 3.1 Changelog

http://dadamailproject.com/support/documentation-dada-3_1_0-alpha_3/changes_3.0_3.1.pod.html

Dada Mail 3.0 to Dada Mail 3.1 Guide

http://dadamailproject.com/support/documentation-dada-3_1_0-alpha_3/dada3_0_to_dada3_1.pod.html

An overview of some of the new features, including:

Dada Profiles (BIG Feature)

http://dadamailproject.com/support/documentation-dada-3_1_0-alpha_3/features-profiles.pod.html

Multiple Mailing List Sending

http:///dadamailproject.com/support/documentation-dada-3_1_0-alpha_3/features-multiple_list_sending.pod.html

(VERY initial) UTF-8 Support

http://dadamailproject.com/support/documentation-dada-3_1_0-alpha_3/features-UTF-8.pod.html

If you plan on upgrading, please also see the 3.0 to 3.1 migration tool:

http://dadamailproject.com/support/documentation-dada-3_1_0-alpha_3/dada_3_to_dada_3.1_sql.pl.html

which should make things a little more pleasant.

There's also more development docs for the new API, including:

DADA::Profile

http://dadamailproject.com/support/documentation-dada-3_1_0-alpha_3/Profile.pm.html

DADA::Profile::Fields

http://dadamailproject.com/support/documentation-dada-3_1_0-alpha_3/Profile_Fields.pm.html

DADA::Profile::Session

http://dadamailproject.com/support/documentation-dada-3_1_0-alpha_3/Profile_Session.pm.html

DADA::ProfileFieldsManager

http://dadamailproject.com/support/documentation-dada-3_1_0-alpha_3/ProfileFieldsManager.pm.html

What is Dada Mail?

Dada Mail is a completely contemporary, mature and intuitive web-based e-mail list management system, which runs on most any Unix-like hosting account that can run custom CGI scripts (details). Dada Mail is also a conceptual art project.
Dada Mail handles Closed-Loop Opt-in/Opt-out subscriptions, sending complex announce-only and/or discussion mailing list messages with an advanced, fault-tolerant mass mailing monitor, supports the archiving/viewing/searching/resending/syndicating (rss, atom) of sent messages and doing all this and a whole lot more with style.

Dada Mail can handle custom subscriber fields and you can use the information it captures for partial list sending based on a query and Dada Mail's email templating system allows you to create targeted and completely custom email messages for each and every one one of your subscribers.

Dada Mail is bundled with additional plugins and extensions to extend Dada Mail's capabilities. Some of the plugins/extensions support advanced bounce handling, clickthrough tracking, mass mail scheduling, blog interfaces of archived messages, AJAX subscription form trickery and lots of other surprises.

Dada Mail produces XHTML valid web content and sticks to best practices when creating email messages. Write Once: Distribute Everywhere. Dada Mail is free software that you're able to use, modify, share and enhance under the terms of the GNU General Public License. Dada Mail is written in Perl because we love Perl.

A Logo design for Rakudo Perl 6

The below is a post I started on perl6-users@perl.org and is amended by the last weeks of replies.

I'm posting it below to garner a few more eyeballs, before I go from a, "research" stage to a, "playful" stage of trying simple things out and seeing if anything makes sense.

If you're a part of the Perl community, I'd love to hear your input as a Kaduo Perl 6 logo may be something you'll see almost every day:

Hello everyone,

A few weeks ago, probably more than I want to admit (but here! See! My Notes! I've been up to something!), there was one of those, "OMG! Perl is going to DIE!" threads, somewhere and the, "Well, do something about it" call came out, and I sort of replied, "well, alright" and gave my open ended hand to some design work.

Moritz was one of the first to guide it to the idea of making a logo for Rakudo Perl 6 - as there's nothing yet (really) available. I thought that would be a neat project and scratch some of my person itches.

A slight background on me: I'm that guy:

• http://xkcd.com/519/

(Although, there should be two more columns for, "Skateboarding" and, "Rock Climbing" in there). I really started working with Perl during a internship in college which started me in a full time job doin' the stuff, which got me to where I am today: just working for myself.

In all of that, I also managed to get through Art School (guess what paid for *that*!). A lot of my work deals with language and text, writing and communication. Some of my work bordered on generative work, but most of it stayed on the canvas, as I wanted to go to school to learn to paint.

Anyways, I've always wanted to help out on larger-than-myself Perl projects, but, although I think my perl-fu is, well, alright - it's not wizard-like. But! I think an interesting niche that I could fill is as someone who, "gets" Perl and it's wonderful and varied culture and also, "gets" visual communication and all that. I am a firm believer that a healthy community is one full of diversity and successful projects come not out of one genius, but of many just normal people, and that's me: just a normal person.

Here's some notes I've collected about what people have said Rakudo Perl 6 is, which is a good baseline on what a logo should try to reflect and communicate:

Rakudo Perl is:

• Perl 6 on Parrot
• Rakudo as a *implementation* of a *specification*
• Rakuda-do (Japanese): Way of the Camel
• Rakudo (Japanese): Paradise

Camels, Paradise/Oasis, etc. It's good visuals and it's easy to digest. Camels are mean and smelly close up, but we don't have to get all that close - it's good to remember they're also extremely useful and in some places, absolutely critical to ways of life.

It also doesn't stray far from the original Perl 5 image of Camels and Pyramids and all that jazz. The Japanese lean on all this seems important too, perhaps to get a little more mindshare from the folks that find Ruby interesting and attractive (more on that in just a little bit)

One of the main fears with this name, "Rakudo Perl 6" - at least when it first came out, is that describing Rakudo Perl as, "An implementation (one of possibly, many) of the Perl 6 Specification, built on top of the Parrot Virtual Machine", will leave people going,

"Huh?!"

I think this is a good reason as anything, to think of getting a visual representation of this, somewhat complicated idea out ASAP.

It's interesting that Japanese Word(s) were chosen as the new name - it's grown on me - I like it. Japanese Art and Poetry has the idea of the, "haiga" and the, "gō" - which is sort of like a pen-name, but it changes if there's a change in the style of the artist during their career. You could think of, "Rakudo" as a new, "haiga" for Perl - at least the concept makes things clear to me: we've changed course, but it's still the same hand.

I was also, incidentally, doing some research on the origins of the Latin alphabet - I'm very curious about languages in general - mostly how they're abused in media and popular culture, but also in learning new ones - I took a brief stint solo in France armed with a 5 week Free University course in French to see how well I could get along. Eye-opening.

I happened upon the book, "Mysteries of the Alphabet" (Narc-Alain Ouknin - originally written *in* French) in the shelves of someone I was hanging out with. It's main thesis, really is the Latin/Greek alphabet started with something they term the, "Proto-Sinaitic" alphabet, which was created around the time the lost tribe of Israel was figuring itself out. Moses and all.

One of the, well, the third letter in their alphabet is, "gimmel", which comes from the word gamel, which, if you didn't guess already means, "Camel"! It looks either like the neck of a camel - or, perhaps it's hump. If you think of, "gimmel" as our, "C" you can sort of still see a hump of a camel, if you just turn the, "C" 90 degrees, clockwise.

Early written languages like Proto-Sinaitic are sort of the beginning where pictograms that stood for what things looked like, where changed into a way of writing about an idea - a lot of these early letters still hold a lot of the original meanings. Hebrew, for instance still does and each character is rich in back history. I like the Proto-Sinaitic link, since it's less loaded quite as intensely as Hebrew, which is invariably tied to major traditions and religions.

The, "Gimmel" character is no exception. Start from, "Mysteries of the Alphabet"
------------------------------------------------------------------------------
:: Original Meanings
Carrying the Primal Power beyond, outside the domestic setting

:: Derivative Meanings
Outgoing, break, carry to another, do good, return a favor

:: Acquired Meanings, Perpetuated By The Hebrew Language

Ripen, Ween, Enable to ripen
Release Oneself, Break Away From
------------------------------------------------------------------------------

Which I think beautiful illustrates a nice connection between Perl 5 and Perl 6, Rakuda-do/Rakudo and haiga pen-names - and the, "Big Picture" of what exactly Perl 6 is all about - standing apart from the lineage, but still being, well, "Perl", using the Camel (gimmel) as a fundamental icon from way WAY back in time there -

and it all sounds so Perlish of picking and choosing the best from many ideas and languages.



Taking a step back from this soup of ideas and thinking of a logo itself, it seems that it would help to produce something that's made of somewhat interlocking and inter-related pieces: Perl on Parrot has two separate pieces that come together and complete an idea. But there's other things that could take, "Perl's" place, so it's really,

$x on Parrot

It seems that if a logo would be made, we can modularize, say, that Parrot part and use it for other things - same with the Perl part, if you get into it. Modularization is a way of getting ready for the future. It also allows us to be lazy: we can use other people's work already and - well, you all know this already. It would be an interesting idea to use common programming best practices in the logo of Perl itself.

I bring that all up, because when I look at the various Perl projects, they all look extremely interesting, but fragmented. It would almost make sense to create a logo where pieces can be reused for related thingies. Starting from a logo for Rakudo Perl, one could make a simple Style Guide even, with easy-to-acquire graphic elements that say, an application written using Rakudo Perl could use (at their discretion) to enhance their own project and tie it back into Rakudo Perl (and Perl in general).

So that's sort of the other thing I'm proposing: not only designing a logo for Rakudo Perl, but having the concept of modularization of the logo's basic elements part of the logo design itself and the sharing and remixing of the design elements for related projects, to help strengthen the, (and I'm not a fan of using this word) "branding" (sigh) of Perl and making it not so much the "invisible language that glues everything together", but have it where it belongs: in a positive light with the general (geeky) public.

Right now, all that means to me is perhaps a simple style guide and the image and graphic resources easy to grab in open formats. This also means that simple-is-better when it comes to the design of all these different elements of the logo, as they'll be combined in interesting ways and things will have a tendency to be cluttered. Simplification also lends itself to universality: the simpler the idea is, the more we relate ourselves *to* the idea itself.

Finally, the design of the Rakudo Perl logo should lend itself to change. Just like any other part of the project, the ideas and concept and what the darn thing actually look like should be able to move and change quickly - to be able to be upgraded - hey, why not? I'm not interested in, "owning" the design, but am more interested in playing a part in shaping the whole. And all that stuff. Just like in code, some people do more than adopt a project.

What I really *really* need now is some feedback for any and all of the above research and braindump. This, obviously, is just one regular guy taking the first jump into something a whole lot bigger and more complex than he can even really imagine. The first step is the hardest and the most confusing - the rest of the journey is mostly, one foot in front of the other, until the end,

Here's a few important notes about the above:

Love Marks vs. Trademarks:

Sounds like a great idea. Go for it. Here is a document I prepared for Larry Wall and the Perl Foundation about a trade mark strategy for Perl. You may find some ideas in here to help - I like your idea of interlocking imagery etc.

http://thegoo.org/love-marks-for-perl.pdf

This document encourages the use of memorable, positive, interconnected trademark. The Perl Foundation will be the custodian for such a mark. This is all wonderful. I don't have any problems with giving over the rights for the project to TPF and Perl at large - sort of the point, really.

This is also a very important part of the document above:

If the Perl Community accidentally borrowed “The Camel” it appears O’Reilly accidentally borrowed
the domain name “perl.com”. Goodwill continues to accrue in the trade mark “PERL” and that
goodwill, including any domain names needs to be directed to its rightful owner: TPF.

With the Rakudo Perl 6 logo, we don't want to add even more confusion - Camels, it seems are out. I wouldn't mind a gesture of a camel-like idea - for example, my gimmel idea above, but that idea may just be a stepping stone for something much large - for example, trading the Rakudo Perl 6 logo as a type of primitive written alphabet, which can be combined, with descretion with other people's and project's ideas.

I think, at least in the "playful" stage, the camel be used as a design element, just because it could lead to more interesting things.

BUT - Richard Dice of the Perl Foundation does make this situation absolutely clear:

O'Reilly is the only organization that can have trademarks that incorporate
a camel in reference to the Perl programming language. This statement is a
first-order approximation, but damn good one. Basically, the message is:
stay away from using camels

Patrick Michaud had this to say:

I'll start with my summarized thoughts first, then a few specifics below.
First, I totally agree that Rakudo Perl needs a logo, and the sooner
we can get one to start using on our website (which we're now creating)
and other materials, the better. So, your thoughts and ideas are
quite welcome.

Beyond that, I think your message has neatly captured many of the facets
of what we're working with (and yes, there are a lot of facets). The
message reminds me of some of Larry's "State of the Onion" addresses --
looking at many sides of the Perl (in this case Rakudo Perl 6) world.

So, my primary comment is that I find myself very much in agreement
with your proposal, and a strong hope that you'll continue on in this
direction. I agree with the idea of the style guide and some simple
elements that people can combine together in very expressive and
beautiful ways. Indeed, it sounds very much like you're proposing
an ideographic system for Rakudo Perl, where can combine graphic
symbols for ideas together. That feels very Perlish and Rakudoish
to me -- I like it.

So, I'd say "press on" and let us see what you come up with. :-)

The more I think about the design problem, the more it it seems that a simple but very flexible solution would be best and that reflecting and taking the actual design process and specification of Perl 6 itself as a major influence would be a great way to go about things.

I can't really speak for what Parrot will want or need, as they already
have a (very nice) logo. But having a symbol to represent Parrot in
the system you propose would likely be very workable.

Exactly!

I may do more research with Patrick and see what the current needs are for a Rakudo logo and how that could be worked with a, "Grand Scheme" as well as start playfully working on some ideas.

What I could use now is a whole lot more feedback. Sound off!

How to handle completely clueless users?

I admit I'm sort of a email traffic controller.

It irks me to the point of a pet peeve, when I spatter the contact page of my project with notes that say, "DO NOT CONTACT ME FOR SUPPORT UNLESS YOU WANT TO BUCK UP SOME MONEY", and I get tons of support emails, wanting free support.

Sigh.

What I don't want to do, is take the time and go,

"Look. We have forums. We have mailing lists. Use those. Thanks." And then not answer they're question.

What I do is exactly that.

I don't want to do this, because I know how much of a pain it is for the person to now, *rewrite* what they just ask me. It also is a pretty weird introduction to what should be a really valuable exchange - someone actually wants to talk to *me* about *my* thingy.

The curious thing is, I don't want to be hired to fix some stupid bug - I have better things to do. I'd rather help fix the bug for *free* - and its beneficial - users let me know it's there, I (or them - ha!) open up a bug report, it gets fixed (probably by me) and *there's a thread of, "something just happened"* that lives far away from my inbox and takes on a life of its own. I don't have to write the same thing twice and with a little work by the user (or if they dare help themselves) they can find it themselves, and keep a thread going. When I say, "thread" - I mean, anything - a forum post about a problem, with a reply of a confirmation that it's a bug, with a link posted to a bug report, that may then give you a patch in a day or so - which then gets referred in another forum post - etc. But, a trail is there.


The thing is, there's better ways to ask the questions.

What's the better way to point them to the right way?



For a while, I just had a auto responder that said,

"Hey, if this is a support question - go here, and here and here and read this and that and then post to this thing or that thing and away you go! America!"



"...but, if this isn't a support question, well, disregard all that, and I'll get back to you, soon"

That seemed a little foolish.


Right now, I'm trying something different - I'm just making up a mail template that first, welcomes the new person emailing me to the *community* and then gives a very brief list of resources that say, "Here's what to read", "Here's where to ask questions"

Then,

well, then if I can, I'll answer they're question - I haven't yet said, "And next time, POST HERE", cause again, that makes me think I'm reprimanding someone for something they may have just missed. Or something. Anyways - let's think positive and we're all humans here, and we probably all have a strong sense to belong.

But, what I want to do is get people active in the program and community. It's pretty hard, since, well, usually my audience is general users and they just have Work They Need To Do, and aren't nerdy enough to get excited about some program. Understandably.

Perhaps, by simply giving a welcome, some encouragement, some links to check out and a head-start on solving the problem, they'll feel as if they're welcome in a community, instead of just a bother to me.

As far as I'm concerned, community in anything is everything. If you're not building a community, you have almost nothing, no matter how great you are, you have nothing without people.

Anyways, I'll let you know how it goes, but I'd be interested in hearing other ideas you yourself may have,

Problem with Email::Valid and newlines

Found a strange problem in Email::Valid:

 
#!perl
use strict;

use Test::More tests => 4;

BEGIN {
 use_ok('Email::Valid');
}
my $email;

$email = 'withnewline@example.com' . "\r";
ok(Email::Valid->address(-address => $email) eq undef, 'addresses with "\r" at the end return undefined');
diag(Email::Valid->address(-address => $email));

$email = 'withnewline@example.com' . "\r\n";
ok(Email::Valid->address(-address => $email) eq undef, 'addresses with "\r\n" at the end return undefined');
diag(Email::Valid->address(-address => $email));

$email = 'withnewline@example.com' . "\n";
ok(Email::Valid->address(-address => $email) eq undef, 'addresses with "\n" at the end return undefined');
diag ("'" . Email::Valid->address(-address => $email) ."'");

From what I expect, that last test should pass. Email::Valid's docs do say that if address can munge the address, it'll return the munged address, instead of undef - but I have munging (or "fudging, in this module" *off* - as it is off by default). Carriage returns aren't munged, so why newlines?

Perhaps it's the last line in the module:

 
# Regular expression built using Jeffrey Friedl's example in
# _Mastering Regular Expressions_ (http://www.ora.com/catalog/regexp/).

$RFC822PAT = <<'EOF';
# Possibly the most insane regex... possible
EOF

$RFC822PAT =~ s/\n//g;

The test for validity changes the actual string being validated. Drrrrr.


And that kind of brings up another point: Why is a module called, Email::Valid and its validating method (address) also doing double-duty with wanting to munge the string I give it so it may possibly be validated? Sounds like you just need to methods there, validate() and, gimme_back_an_address_if_at_all_possible(),

or,

whatever.