fcshctl: the Flex Compiler Shell Controller

Introduction

Like I've mentioned before in this blog, I develop Flex applications using the Flex SDK and jEdit instead of Adobe's Flex Builder IDE. This setup has worked very well for me but one annoying issue I've had to deal with because of this is the slow compilation speed: every time I make a small change into one of my projects and recompile it, the mxmlc compiler (that my build scripts are calling) has to load the JVM into memory and recompile my whole project from scratch, which obviously takes a while. Compiling things in Flex Builder is a lot faster, and the reason for that is the Flex Compiler Shell, which it uses for compilation instead of mxmlc.

The basic idea behind the Flex Compiler Shell is that it would stay in memory between compilations (thus avoiding the cost of loading up the JVM every time you want to compile something) and use a technique called incremental compilation that, as the name suggests, is based on the idea of only compiling the parts of your project that have changed since the last compilation (thus making the process a lot faster). When you run the Flex Compiler Shell executable (called fcsh), you're thrown into an interactive shell where you can run commands in order to compile your projects and control the operation of fcsh itself.

The Problem

When I found out about fcsh, tried it out and saw how much faster it was than mxmlc, of course I wanted to immediately start using it instead in my build scripts. The biggest problem with doing this, though, is the fact that the basic operation of these two programs is fundamentally different: when you run mxmlc, it compiles the project, prints some messages to standard output and then exits (with the exit status 0 if the compilation was successful or 1 if it wasn't), which is perfect for integrating it into build scripts. Fcsh, on the other hand, throws you into a shell of its own and expects you to essentially operate it from within itself (instead of simply calling it with some arguments and then having it return with some standard output and an exit status like mxmlc).

The other issue that makes fcsh more difficult to use in general is that in order to use the incremental compilation feature, you have to first compile your project like you normally would (using the "mxmlc" command), then make note of the "compile target id" number that fcsh then assigns to this project and specifies in its output, and for future compilations use the "compile N" (where N is the compile target id) command.

The Solution

I tried to search for a solution to this problem that somebody else might have come up with but didn't find an adequate one (a list of the ones that I could find is further down), so I decided to give it a shot myself. My approach, called fcshctl (short for "fcsh controller"), is basically a simple bash script that:

1. starts the fcsh process if it isn't already running
2. forwards its arguments to fcsh as commands
3. reads the output of that command from the fcsh interactive shell and prints it to the standard output
4. exits with status 0 (success) or >0 (failure)

In addition to this, I also made the script automatically and transparently invoke the incremental compilation feature whenever applicable so that I wouldn't have to worry about keeping track of the compile target ids and using a different command for the compilation whenever one had already been assigned.

Below is a summarization of the pros and cons of this solution:

Pros:

• It's quite simple -- just one shell script (a few others are included in the .zip that are only for convenience and not required)
• It runs the fcsh instance in a GNU screen session, which means that:
• The running fcsh instance is accessible to all user processes (i.e. not just within a single shell session, for example)
• It is possible for the user to enter the running fcsh interactive shell at any time (by attaching to the screen session via the command screen -r -S fcsh) and manually interact with it
• It automatically checks whether you've compiled a specific file before and keeps track of the compile target ids that fcsh assigns so that it can automatically and transparently invoke the incremental compilation feature and let you access the speed gains that are fcsh's main offering without any additional work. This feature can also be turned off by setting an environment variable.
• A proxy script (called fcshctl-mxmlc) that can be used in place of mxmlc is provided. It simply calls fcshctl with "mxmlc" as the first argument, followed by all the other arguments you specify, so that you can simply use this script as a stand-in for mxmlc in any build scripts that you may be using without having to modify them any further.

Cons:

• It uses the GNU screen logging mechanism and polling for changes in the log file in order to catch the output of commands sent to fcsh, which is not very elegant and causes a slowdown of maybe about a couple hundred milliseconds before the output is printed to stdout.
• The output of the executed fcsh command is not printed to the standard output as it comes, but instead all at once when the command finishes (this is not a problem for me personally, but others might want to fix this).
• It's implemented as a bash script, which means that it's not the fastest monster out there (we're talking about milliseconds here, though, so it doesn't bother me at all).

Download

You can download fcshctl here:

• fcshctl-v0.5.1.zip (latest)
• fcshctl-v0.5.zip
• fcshctl-v0.4.zip
• fcshctl-v0.3.zip

fcshctl has been tested with fcsh versions 3.0.0 build 477 and 3.1.0 build 2710 on Mac OS 10.5.5. It is licensed under the MIT License.

The script requires the following to be in your $PATH: rm, mv, cat, ps, sleep, tail, head, grep, awk, screen, xargs and basename. You also need to have the FLEX_HOME environment variable set to point to the Flex SDK root path.

Example of use

fcshctl $ export FLEX_HOME=/Users/username/code/SDKs/flex_sdk_3.1
fcshctl $
fcshctl $ ./fcshctl     

Usage: fcshctl <commands>
  Substitute <commands> with any commands you
  would like to send fcsh, just like you would
  enter them in the interactive fcsh shell
  session.
  To get a list of the available commands, run:
    fcshctl help

fcshctl $
fcshctl $ ./fcshctl help
List of fcsh commands:
mxmlc arg1 arg2 …      full compilation and optimization; return a target id
compc arg1 arg2 …      full SWC compilation
compile id               incremental compilation
clear [id]               clear target(s)
info [id]                display compile target info
quit                     quit
fcshctl $
fcshctl $ ./fcshctl mxmlc ~/code/Flex/projects/testbed/testbed.mxml
Loading configuration file /Users/username/code/SDKs/flex_sdk_3.1/frameworks/flex-config.xml
Recompile: /Users/username/code/Flex/projects/testbed/testbed.mxml
Reason: The source file wasn’t fully compiled.
Files changed: 0 Files affected: 1
/Users/username/code/Flex/projects/testbed/testbed.mxml(28):  Error: Incorrect number of arguments.  Expected 2.
    func1(“toka”);

fcshctl $ echo $?
1
fcshctl $
fcshctl $

Other similar solutions

Below is a list of other approaches to the problem of integrating fcsh to non-standard (i.e. non-Flex Builder) workflows (roughly in order from most promising to least promising), and short lists of reasons why I personally didn't want to use them. If my solution doesn't seem good to you, you should maybe check some of these out.

fcsh tools by Mike Rowe

• Runs fcsh in a daemonized process, which means you won't be able to enter the interactive shell of a running fcsh process and manually interact with it
• You can only specify the filename to compile; has no support for additional mxmlc arguments
• For GNU/Linux (but seems to work just fine on OS X as well)

flex-compiler-shell-daemon by Leonardo Soto Muños (also see his blog post about it)

• I couldn't get it to work out of the box on OS X
• Runs fcsh in a daemonized process, which means you won't be able to enter the interactive shell of a running fcsh process and manually interact with it

FCSH Wrapper by Mihai Vasilache

• For Windows
• Designed to be used with Ant
• A bit complicated

http://code.google.com/p/fsch/ by nimrod97

• For Windows

BigSource Zarkov by BigSource GbR

• For Eclipse & Ant

21 Comments