Emoji and Unicode

Emoji is basically a Japanese term for emoticons used on mobile devices. The Wikipedia article has more information on the concept, but this part is the most interesting:

When transmitted, emoji symbols are specified as a two-byte sequence, in the private-use range E63E through E757 in the Unicode character space, or F89F through F9FC for Shift-JIS.

Now, that’s not true for Apple’s Emoji implementation — it’s apparently completely incompatible with the handsets from the Japanese carriers (see OpenRadar #6402446 for more info) — but the part about them being implemented as unicode symbols is still true in iOS, the code points used are just different.

I’ve made this demo page that lists unicode symbols and the hexadecimal code points for all the different Emoji that are available on my iPhone (iOS version 4). The demo page has a cache manifest so it should be available offline as well. If you’re currently on a desktop computer you won’t be able to see them so here’s a picture of what it looks like on the iPhone:

In order to be able to use these icons you need to have easy access to the hexadecimal unicode code points so this page should come in handy. If you instead prefer a native app for this, there are many you can choose from.

Using Emoji Symbols in HTML and CSS

User agent detection is usually cited as a bad thing, and with good reason — it’s often used when feature detection would be more appropriate, causing unexpected breakage in many edge conditions. When planning on using the iOS emoji on a web page, though, you really need to sniff the user agent in order to know whether the client is an iOS device and thus has a font that implements these unicode symbols in the expected way.

Here’s an example (in PHP) of how you’d detect iOS devices:

When you’ve detected the client to be an iOS device, you can offer them an additional stylesheet:

In this stylesheet you’ll then insert the Emoji symbols into whatever locations you’d like to use them in, and to do this you can use CSS2′s content generation mechanisms (in the examples below, we use code point E00E, which is the “thumbs up” Emoji):

Containing all the iOS-specific embellishments in one separate CSS file is, I think, the way to go, but if you want you can also insert these symbols directly into your HTML as special entities:

Big Caveats to Consider

This is a great way to add little graphical cues to your web app without having to deal with images (i.e. making or obtaining them, serving them, having the app load them, possibly caching them) but of course the downside is that they are only available on Apple’s iOS devices, which makes this trick quite useless unless you’re concentrating completely on the iPhone/iPad/iPod Touch axis.

The key takeaway from this should be that there’s this neat trick you can use but if you need your icons to be visible in all clients you should forget about these and just use images for everyone (it’ll be much easier).

Choose Between GeekTool and NerdTool

GeekTool and NerdTool are two programs that do the same thing: display stuff on on your desktop. The choice between which one to use is up to you but allow me to give a few suggestions:

• If you want to change the text colors used in the display, choose NerdTool (as far as I know, GeekTool can only change the default text color but not the actual colors used for “red”, “blue background”, “bright green” etc.)
• If you want to hack on the program yourself, choose NerdTool (it’s open source).
• If you don’t care about any of the above, choose GeekTool.

Install the Apps

When you’ve chosen which one you’d like to use, download and install it. GeekTool is a preference pane so it lives in your System Preferences, while NerdTool is a regular application which you drag into your Applications folder.

In order to install icalBuddy, download the latest version and unzip the package by double-clicking on it. Open the unzipped folder, double-click on install.command and follow the instructions (you’ll have to press return, input your password and icalBuddy will be installed).

Figure out Your icalBuddy Configuration

So now you need to figure out what to put into the command field in Geek-/NerdTool. First you need to go to the icalBuddy Usage Examples page and browse through it — you might find the kind of output you’re looking for right there, in which case you can just get the command you see there (note that some of the examples require you to add something to the configuration file).

If you’d like to take some of the examples and modify the output a bit, you can fire up your Terminal.app and start experimenting with different arguments. In this case the documentation links will come in handy.

Add Your icalBuddy Command into GeekTool/NerdTool

Once you have your command, you can put it into the app by creating a new “shell” entry:

• GeekTool: Drag the “Shell” geeklet icon (see below) onto your desktop. The properties window for this entry should appear.



• NerdTool: Click the “plus” button (see below) and select “Shell” from the popup. A new entry is created into the list above.

You can then insert your icalBuddy command into the command field in whichever app you’re using (and set the refresh interval in order to specify how often the command should be run):

That’s it. You can then delve into the settings that both GeekTool and NerdTool offer you, like changing the colors and fonts used for the output.

Demo

Here’s a small demo where you can see the feature at work: Font List Example. The sources shown below are used in this demo.

Implementation

We need a small Flash app and some Javascript. Below is the source code for the Flash app.

Note that you don’t need the Flash authoring app from Adobe in order to compile this; you can simply use the mxmlc compiler in the Flex SDK (which is open source).

Notice how the Flash app calls a Javascript function populateFontList() via the ExternalInterface. Now we just need to implement that function in the page where we embed the Flash object:

I read somewhere that Safari (or the OS? or iTunes?) sometimes decides to “fix” the mappings of iTunes URL protocol handlers, forcing you to “play cat and mouse” if you decide to prevent iTunes from automagically opening up by removing it as the itms: protocol handler, so I decided that it would be best to try and fix this problem by messing with the iTunes website source via GlimmerBlocker, which I already was using for ad blocking.

I couldn’t find any straightforward directions on how to do this with GlimmerBlocker so I decided to post it here myself once I figured out how to do it. So this is how I prevented iTunes from automatically opening up whenever I open up a store link in my browser:

• Open the GlimmerBlocker “filters” tab and add a new filter, as well as a new rule for this filter.
• Select Whitelist URL, optionally modifying content as the action
• For the Host section, select is as the predicate and type in itunes.apple.com
• Select ignored for both Path and Query
• Switch to the transform tab and type in the following:
replace(/detectAndOpenItunes\(\);/g, "<!-- detectAndOpenItunes() zapped -->");
• Make sure the Only for content-type selection in the bottom-right corner of the dialog sheet is html
• Save

This seems to correctly prevent the iTunes app from opening up whenever a store website is opened, but it still allows for opening the viewed product in the stand-alone app via the “View in iTunes” link on the page.

It’s obvious we need a trash command that we can use instead of rm when we’re not 101% sure we need to actually remove these files, like, right now. Here are a couple of existing solutions I’ve found, my impressions of them, and then my version at the bottom:

osxutils’ trash by Sveinbjörn Þórðarson

This is what I’ve used up until now. It’s a small Perl script that simply manually moves the specified files into the trash folder under the current user’s home directory:

The biggest problem with this is that it moves all trashed files onto the same volume instead of using each volume’s own trash folders (this is not a good idea). It also deals with filename collisions manually, which is a bit volatile. The good thing about this is that it doesn’t follow “leaf” symbolic links, which is in my opinion the expected behaviour (what if I want to trash the link instead of its target?).

osx-trash by Dave Dribin

This one is a lot better. It’s a Ruby script that uses the Scripting Bridge to ask Finder to perform all the actual “heavy lifting” (e.g. moving the files to the trash) which means that each volume’s own trash folders are utilized properly and filename collisions are handled in a standard way:

It also allows you to list all the files that are currently in the trash and empty it (normally or securely). The few minor negative things are the fact that it follows leaf symbolic links and the fact that when you try to trash multiple files you don’t have access rights for, Finder will pop up an authentication dialog separately for each file.

my trash

Due to the few things that I wanted the trash command to do differently, I wrote my own. In order to make sure that filename collisions are handled in the standard manner and that each volume’s trash folders are properly used (just like with Dave Dribin’s script) this one first tries to use the standard system API for trashing files, and if that fails due to the user not having the correct access rights, then asks Finder to trash those files (Finder can authenticate the current user as an administrator and move the files into that user’s trash — if you simply run this program as root, the files will be moved to root’s trash, which is not what we want):

The difference is that this one doesn’t follow leaf symlinks, and when deleting multiple files that you don’t have access rights for, Finder will only show one authentication dialog (I had to implement this kind of “manually” because I couldn’t find any way to accomplish it with the Scripting Bridge). I also copied the idea and implementation of emptying the trash and listing its contents via the Scripting Bridge from Dave’s script. This trash is written in (Objective-)C, for all those extra milliseconds that are so goddamn important when waiting for files to be trashed.

You can go through my trash here.

The Application Kit additions to the NSAppleScript class provide an implementation of the syntax highlighting (which would be the hardest part in implementing something like this) and I already have the ANSIEscapeHelper class that provides the translation of an NSAttributedString to an ANSI-escaped NSString (which would be the second-hardest part in implementing something like this) so writing a command-line program around these two parts was quite simple in the end. The program is called asprint and it has its own page on this site where you can download it.

This is what using asprint looks like:

There was nothing on the horizon about any further additions to GeekTool (or any similar programs) that would provide the features I wanted (although version 3 of GeekTool seems to be in the making, and there’s also the GPL’d NerdTool project which aims to implement a clone of v2 with some changes and fixes), but luckily Yann Bizeul, the author of GeekTool, had released the source code to it, which enabled me to put this feature (and a few others) in. As you can see from the screenshot below, I also took the liberty to rearrange the GUI a bit:

I’ve been running this custom build for a few months now without any problems so I feel reasonably confident about releasing it. No open-source license has been specified for GeekTool 2.1.2 so I unfortunately won’t be releasing the source code for the changes I made. Sorry :(

Download link

First the most important part — here’s the download link:

GeekTool-2.2.1-unofficial-hasseg.org-build.zip (Latest version. Requires Leopard (Mac OS 10.5+)) (Sep 6, 2009: Recompiled the preference pane as 32/64-bit binary with support for garbage collection so that it would not force System Preferences to restart on Snow Leopard.)

GeekTool-2.2.0-unofficial-hasseg.org-build.zip (Requires Leopard. Based on source code from version 2.1.2)

Disclaimer: I will most probably *not* fix bugs in this version or develop it further in any way. This is simply a temporary solution until the next official version of GeekTool comes around (if it includes these features, which I don’t know if it does), or someone else releases something similar.

In order to install it, do the following:

1. Extract the files from the zip archive by double-clicking on it in Finder
2. Double-click on the GeekTool.prefPane file icon in Finder to install it
3. If you already had GeekTool installed, System Preferences will ask if you want to replace it. Select “Replace”, quit and restart System Preferences and open the GeekTool preference pane, verifying that the version label now says “unofficial hasseg.org build.”
4. Unselect the “Enable GeekTool” checkbox in order to kill the old version’s background process, and then select it again in order to run the updated one.

Added Features

ANSI Escape Sequence Support

As said, this version includes support for ANSI escape sequences, including all the supported foreground and background colors (each of which you can change yourself on an entry-by-entry basis in GeekTool’s preferences), intensity (bolding) and underlining (both single and double). Here’s an example of how my (customized) icalBuddy output looks in my GeekTool setup:

In order to learn how to customize icalBuddy’s ANSI-formatted output, please refer to the icalBuddyConfig man page. Also remember that you need to use the -f (or –formatOutput) argument to get formatted output in the first place.

Here’s what the new little preferences window that you can use to customize the actual colors used for displaying ANSI escape colors looks like:

You can specify different colors for each GeekTool entry you have.

UTF-8 Support

This version now supports UTF-8, which means that unicode characters (umlauts, arabic, hebrew etc.) will display correctly (version 2 only supported ASCII).

Text Writing Direction Setting

I had received word from one particular user of icalBuddy from a country where the language is written from right to left (arabic, to be precise,) who had issues with the bullet point sometimes shifting to the right side of the row instead of staying at the left. This was due to the OS X text layout algorithm trying to figure out if it should print characters in a string onto the screen from right to left or from left to right and sometimes failing due to mixtures of latin and arabic characters being on the same line. Now problems such as this can be fixed by explicitly specifying the text layout direction:

Benefits of Declarative Diagramming

Web Sequence Diagrams is great. It uses a simple, readable syntax you can use to type in descriptions of sequential workflows and based on that it will generate the corresponding UML sequence diagrams for you.

Working with these diagrams in this way has a couple of nice benefits over more intricate/professional methods:

• They’re quick to create and quick to modify
• The syntax is pretty easy to read as-is
• The source is plain text, so all the benefits of using plain text apply (e.g. can put under source control, can use simple diff tools, file sizes are small etc.)

Helper Script and Syntax Highlighting

When I was sure I wanted to use this method again in the future, I hacked up a few quick and simple tools for this:

Script for generating the diagram images from source files

At the time of this writing, there’s a suggestion called “Create a command-line version of this application” on the service’s UserVoice site that’s labeled “started”, so hopefully in the future we’ll have a command-line sequence diagram generator from the author of the online service. In the meantime, though, here’s my solution. This is a Python script that can be used for generating the diagram images (using this service) from “source files” that contain the declarative workflow syntax. You just need to:

1. type up your diagram description in a text editor and save it into a file
2. begin the first line in the file with #wsd (so that the script knows this is a websequencediagrams.com syntax file)
3. (optional:) specify the diagram style you’d like to use on the first line with the syntax #wsd:style=modern-blue
4. call the script with the path to this file as an argument to generate the diagram image

The script uses a temporary SQLite database to save hashes of file contents so that even if you call it repeatedly for the same file, it’ll notice if the file contents haven’t changed and decide not to send a request over to the server. I did this in order to reduce unnecessary load on the server. I tested the script on Mac OS 10.5.7, Windows XP SP3 and Ubuntu 9.04 and it seems to work just fine on all of those (of course, on Windows you’ll first have to install Python manually).

Download the script here: wsd.py

Here’s an example on how to use it (to see the script’s usage info, run it without any arguments):

Save this into a file called ax-diagram:

#wsd

participant "Component A" as A
participant "Component X" as X

X -> A: what up
A -> A: processWhatUp()
note over A: A is confused
A -> X: go away
destroy X

Then run the script:

wsd.py ax-diagram

…and the file ax-diagram.png is created:

jEdit syntax highlighting mode

This is a simple jEdit syntax highlighting mode for the Web Sequence Diagrams syntax. You can download the file here: websequencediagrams.xml. Refer to this other blog post of mine for instructions on how to install modes for jEdit.

Here’s a screenshot of the highlighted syntax on my machine:

The Markdown jEdit mode

Markdown is my favourite markup language. Its primary design principle, that the markup written in it should be as readable as possible even by itself, is something my plain text -loving mind can easily agree with. If I’m writing a document that needs to be represented as HTML later on, Markdown is my first choice (too bad it doesn’t work very well with WordPress).

Here’s my jEdit mode file for Markdown: markdown.xml.

To install this mode, add the following to the modes/catalog file in your jEdit user settings directory (select Utilities → Settings Directory in jEdit to find this):

<MODE   NAME="markdown"

        FILE="markdown.xml"

        FILE_NAME_GLOB="*.markdown"

/>

Then copy the markdown.xml file into the same directory with this file. See the Installing Modes section of the jEdit users’ guide for more information.

This mode is not perfect; I could not figure out how to properly differentiate between code blocks and list item paragraphs (both are indented by 4+ spaces or 1+ tabs). Here’s what I wrote into the mode file related to these:

<!-- PROBLEM: code blocks

     

     Both code blocks and list paragraphs are specified by indenting

     (4+ spaces or 1+ tab) - how to differentiate between them? We

     don't want markdown syntax coloring to apply within code blocks

     but we _do_ want it to apply within list paragraphs. As it stands,

     we can't have both (or at least I don't know how to do that with

     jEdit's current mode syntax implementation.)

     

     The next line applies one syntax color type for all lines that

     begin with at least 4 spaces or 1 tab, which means that all such

     lines are treated as code blocks (this disables markdown syntax

     coloring for indented list paragraphs). Comment or uncomment it

     according to your preferences.

-->

<EOL_SPAN_REGEXP TYPE="LITERAL2" AT_LINE_START="TRUE" HASH_CHARS=" &#09;">(\s{4,}|\t+)</EOL_SPAN_REGEXP>

<!-- The next line applies one syntax color type for all lines that

     are indented "twice" (i.e. begin with at least 8 spaces or 2 tabs).

     This should safely match only code blocks (no list paragraphs can

     be indented twice like this, if I understand the spec correctly.)

-->

<EOL_SPAN_REGEXP TYPE="LITERAL2" AT_LINE_START="TRUE" HASH_CHARS=" &#09;">(\s{4,}|\t+){2,}</EOL_SPAN_REGEXP>

Here’s how you’d generate a HTML file from a Markdown file:

perl Markdown.pl myDocument.markdown > myDocument.html

The POD jEdit mode

When writing documentation for icalBuddy, I had to enter the wonderful world of creating man pages for the first time. The raw syntax for writing these things is, to say the least, awful. Just awful. Unfortunately I can’t use Markdown for writing them, but a pretty nice alternative I found was POD (or “Plain Old Documentation”), which is mainly used to document Perl code. Now, the important thing to note about this mode is that I don’t know any Perl, I don’t code in Perl and I have no idea how POD is used with Perl, so this mode is not suitable for simultaneous use of Perl and POD — I just wrote it for syntax highlighting in the pure POD documents I use to write man pages. Adjusting it to work for the purposes of documenting Perl code might just be trivial for someone who does know Perl and how POD is used with it, but I can’t do it.

Here’s my jEdit mode file for POD: pod.xml. Refer to the installation instructions for the Markdown mode above to see how this should be taken into use — the process is similar.

Here’s how you’d generate a man page from a POD file:

pod2man --section=1 --release=1.0 --center="manpage_title" --date="2009-04-21" myManPage.pod > myManPage.1

Footnotes

1. The Pragmatic Programmer (ISBN 0-201-61622-X): Chapter 3 (“The Basic Tools”), Sections “The Power of Plain Text” (pages 73-77) and “Code Generators” (pages 102-106).

So I wrote a Quick Look plugin for just that. I call it ImageFolderQLGenerator and its results, when viewed within Finder, look like this:

Implementing the plugin itself wasn’t very difficult, even for someone as inexperienced in C as me, but the biggest headache has been something that I’ve yet to find a solution to: as you can see, Finder automatically wraps folder thumbnails into these “icon decors” with their page curls and drop shadows and whatnot — I haven’t found any way to prevent this from happening. As far as I know, Finder decides if it wants to do that based on the type of the content (in this case, folders) and there’s nothing you can do to influence its choice. If someone knows a cure for this, please let me know!

You can download ImageFolderQLGenerator here. I’ve released it under the Apache License version 2.0.