Aaron Feng

UPDATED: Part 2 of this translation has been added.

After reading the ANTLR book, I decided to build a simple translator that will recognize C# like syntax and output PL/SQL. Even though I heard about ANTLR two years ago, I have not kept up with it, so I decided to start out simple. More complicated translator often involves creating an Abstract Syntrax Tree (AST), and a tree parser that will walk the tree during translation. In my translator, I will only use token rewrite and String Template in order to keep it simple. Rewrite allows you to "rewrite" the input token to something else. String Template is a template engine that will be used to specify the overall structure of the output.

The input language will look like the following:

public class Users {
    public int UserId;
    public string FirstName;
    public string LastName;
}

The output will be a script to create a Users table with 3 columns defined in the Users class. It will also create a sequence and a trigger on UserId column to auto increment the value on insert. Before creating any objects in the database it will check for the existence and drop them if they already exist.

The grammar used to generate lexer and parser is shown below (CSharpSQL.g):

grammar CSharpSQL;

options {
 output = template;
 rewrite = true;
}

@members {
  String className;
  List columns = new ArrayList();
}

// parser
program : declaration+ -> translate(
 name = { className }, 
 id = { className.substring(0,className.length() - 1) + columns.get(0) }, 
 columns = { columns } 
) ;

declaration : class_statment '{' (variable_statment)* '}' ;


class_statment : scope_modifier 'class' ID
{  
  className = $ID.text; 
} ;

variable_statment : scope_modifier type ID  ';'  
{    
  String tmp = $ID.text;
  if(tmp.toLowerCase().equals("id"))
   tmp = className.substring(0, className.length() - 1) + tmp;
  columns.add(tmp + " " + $type.text + " NOT NULL");
} ;

scope_modifier : 'public' ;

type : 'string' -> template() "nvarchar(255)"
     | 'int' -> template() "integer"
     | 'decimal' -> template() "number(21,6)"
     | 'DateTime' -> template() "date" ;
     // add more types if needed

// lexar
ID  :   ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'0'..'9'|'_') * ;

WS  :   ( ' ' | '\t' | '\r' | '\n' )+ { $channel = HIDDEN; } ;

The grammar used here is a combined grammar which means it contains the rule for the lexer and parser. It uses a couple of member variables to remember matched tokens, so it can be passed to the template for the final generation.

The template used for the generation looks like the following (CSharpSQL.stg):

group CSharpSQL;

translate(name, id, columns) ::= <<
BEGIN
EXECUTE IMMEDIATE 'DROP TABLE <name> CASCADE CONSTRAINTS PURGE';
EXCEPTION WHEN OTHERS THEN NULL;
END;
/

BEGIN
EXECUTE IMMEDIATE 'DROP SEQUENCE <name>_SEQ';
EXCEPTION WHEN OTHERS THEN NULL;
END;
/

CREATE TABLE <name>
(
  <columns; separator=",\n">
)
/

CREATE SEQUENCE <name>_SEQ
        START WITH 1
        INCREMENT BY 1
/

CREATE OR REPLACE TRIGGER <name>_TR
BEFORE INSERT ON <name>
FOR EACH ROW
DECLARE TEMP_NO int;
 BEGIN
     SELECT <name>_SEQ.NEXTVAL INTO :NEW.<id>Id FROM DUAL;
     SELECT <name>_SEQ.CURRVAL INTO GLOBALPKG.IDENTITY FROM DUAL;
END;
/
>>

The recognizer can be tested with this test class (Test.java):

import org.antlr.runtime.*;
import org.antlr.stringtemplate.*;
import java.io.*;

public class Test {
 public static void main(String[] args) throws Exception {
  FileReader groupFileReader = new FileReader("CSharpSQL.stg");
  StringTemplateGroup templates = new StringTemplateGroup(groupFileReader);
  groupFileReader.close();

  ANTLRInputStream input = new ANTLRInputStream(System.in);
  CSharpSQLLexer lexer = new CSharpSQLLexer(input);

  TokenRewriteStream tokens = new TokenRewriteStream(lexer);

  CSharpSQLParser parser = new CSharpSQLParser(tokens);
  parser.setTemplateLib(templates);

  CSharpSQLParser.program_return r = parser.program();
  StringTemplate output = r.getTemplate();
  System.out.println(output.toString());
 }
}

The language I defined here is fairly limited, but it can be expanded if needed. The output can be easily retargeted to a different database if a different template is specified. The reason I created this translator is because writing PL/SQL can be a tedious task, and error prone. With a simple translator, it can do most of the work for me.

I have been following ANother Tool for Language Recognition (ANTLR) on and off for about two years now. I always felt ANTLR is one of the most exciting, and not as well known tools, out there. With ANTLR you can build a program that is able to recognize input, a recognizer, by specifying the grammar for a given input (language). ANTLR does all the dirty work for you- it builds lexical analyzers (input character stream, and output tokens) and parser (input token output syntax tree) for you when you feed it BNF grammar. Once you have a recognizer you can do all kinds of cool things like transformation. Basically you can build a Domain Specific Language (DSL) that can help you solve a problem.

I am glad to see there will be a Definitive ANTLR Reference hitting the book store soon. Currently the book is in beta and it is available for purchase in pdf form. After reading the book, I feel that I understand ANTLR at a much deeper level. I am planning to build some simple transformation tools with ANTLR in the near future. I will talk about it in later posts. If you want to be a savvy programmer, this book is a must read.

In less than two months I managed to get a kernel panic on my Mac. Generally kernel panic occurs on UNIX like operating systems when it encounters an error it cannot recover from. It is similar to the blue screen of death on Windows. My MacBook has been intermittently shutting it's self off when I tilt it. It was annoying, but it turned right back on when I pressed the power button. However, last week kernel panic happened when I tried to turn it back on. No matter how many times I tried to reboot the machine, I received the same kernel panic screen. I contacted Apple, and they could not diagnose the problem over the phone, so they told me to drop the laptop off at an Apple service center.

I left my Mac at the Apple service center for two days, and they could not find any hardware related problems. They basically told me I had to pay them money to reinstall the OS in order to fix it. I decided to get the MaBbook back because I can handle the reinstall myself. Since no hardware problems were found, I had to pay 48 bucks to the Apple service center for a diagnose fee. Apparently this is standard at an Apple service center, but not at an Apple retail store, I later found out.

After I got home, I decided to boot up my MacBook. To my surprise, the kernel panic went away. At this point, I was still not convinced it was a software related issue because the laptop still shuts it's self off if I tilt it the wrong way. I called Apple back, and complained about the 48 bucks that I wasted. As reimbursement, they sent me an Airport Express. Since my MacBook is still shutting off on its own, I will have to take it back soon. Next time I will tale it to an Apple store, not a service center.

This is why I'm hot. Yup, that's my baby.

A couple of days ago I was poking around Park Place source code and I saw something resembling the following syntax:

# ... is some arbitrary string in this case
class A < B '...'
 # ...
end

I am no Ruby expert, in fact, I have written very little Ruby code so far. However, the code above looked really odd to me. It looks like class A is inheriting from class B, and class B is taking some arbitrary string as an argument.

Well, after closer inspection I was completely wrong on what B is. B is not a class at all, in fact, it is a method under cover.

def B str
 # do something with str variable here
end

'...' is some argument to the B method, which makes a lot of sense, but how can class A inherit from a method? If you try the code above, whatever is in method B will execute, but you will get an error when class A is trying to inherit from B. The reason is simple, class A is expecting to inherit from a class not a method. To prevent the error, you need to somehow construct a class in method B.

def B str
 # do something with str variable here
 Class.new
end

Ruby automatically returns the last line as the return value. In this case, you will not get the error anymore.

The first question one should ask is, why would anyone want to do this? The code I showed above is a very simple version of the real code that appeared in Park Place. In Park Place it creates a new class and adds methods to the class based on the input. This is powerful because it allows one to hide the base class since it does not actually exist. On top of that, using this technique one can write code that writes code. Ruby on Rails uses similar techniques to achieve all the magic.

Park Place is an Amazon S3 clone in Ruby minus the SOAP support. It is amazing how little amount of code is required to achieve what S3 does.

Another interesting aspect of Park Place is that it uses Camping microframework. Camping is similar to Rails in the sense that it is a web framework that utilize the MVC design pattern. It is small web framework that weighs less than 4k! It is described as "a little white blood cell in the vein of Rails" on [Camping's][camp] wiki. You can see all the code in the framework in one page of your browser.

You can actually migrate from Camping to Rails fairly easily. That got me thinking, maybe [Shoechicken][shoechicken] should use Camping instead of Rails to start with. We can then migrate to Rails as the need arises.

Since Shoechicken will provide RESTful web services, it is only natural to check out some existing web services that are already in the wild. One of the most well known web services that provides a REST API is the Amazon Simple Storage Service (Amazon S3).

S3 provides web services that allow a subscriber to store and retrieve data. The subscriber is charged based on bandwidth and storage used. It is especially important for Amazon to validate your identity so you do not get falsely charged from unauthorized requests. Most important of all, you do not want your data to be compromised.

How does S3 validate your identity? When you sign up for S3 you will receive two keys: Access Key ID and Secret Access Key ID. Access Key ID is included in every request and before you put your request on the wire, you sign the header with your Secret Access Key ID. Of course, no one knows your Secret Access Key besides you and Amazon. This is fairly interesting because you can allow access to others by constructing the request for them.

Maybe we can do something similar to S3 in Shoechicken when the user wants to retrieve recommendations for his or her feeds.

For more information about S3 please refer to the official documentation.

Even though Shoechicken is a Rails application, currently it has no plans for any user interface. It will only expose web services for external clients to consume. In the web services world, you have two camps: SOAP or Representational State Transfer. With that said, we had to make a decision as to which style of web services we will provide: SOAP, REST or both.

One big advantage of using SOAP is the code generation tools that are available in various languages. It works extremely well when you control both sides. However, it can get a little more hairy when you do not have the luxury of a homogenous environment.

On the other hand, REST is very simple. It just uses the old, trusty standard HTTP to transport requests without wrapping it. All the client need to consume a service is a way to send HTTP requests and parse XML.

We want to keep Shoechicken as simple as possible. I feel REST style of web services will be easier to implement and easier for clients to consume with fewer interoperability issues.

During the day I work on a distributed financial system that provides SOAP style web services. It would be fun to learn what is on the other side of the fence. That is one of the main reasons I went with REST for Shoechicken.

Currently on Shoechicken we have three team members. Each one of us lives in a different city. Not everyone is on the same time zone, and our schedule is slightly different. It can be difficult to gather everyone together to have a discussion around Shoechicken.

At first we tried to use instant messaging, it is very light weight, so we can type to each other whenever as long as we are online. However, most IM software is targeted for one-on-one chatting, and it does not save the history in a central location for later consumption. We eventually moved to Campfire, since it has everything we wanted and still kept it simple. It allows multiple people to chat at the same time and saves the transcripts. However, since everyone is on a different schedule, we still had a hard time getting people into Campfire around the same time.

We eventually went back to the old trusty email. I was really surprised email actually worked better for us. I was avoiding using email because I prefer real-time communication over delayed.

Part of the reason email worked well for us so far is because we are still in the very beginning of the project. Everyone tends to have lots of little questions about various parts of the system. It is hard to get across lots of unorganized information via chat style of communication. The screen tends to scroll fairly quickly, and the conversations are often lost. Especially when you have multiple people chatting at the same time.

In general I am not a big fan of delayed communication. I feel it divides up the team and often takes longer to get a response. But in this case, we are all working on Sheochicken as a side project, so some delay is not a problem.

At one time we thought shoechicken could possibly do the Netflix competition. With a closer examination, there are some major differences on how shoechicken provides recommendation compared to Netflix. James Horsley wrote a good post on it. That is a good news to everyone else, the money is still available!

p.s. The link to James' blog was broken in the last feed, it has been fixed.