Tommy Chheng : Programming Blog

Setting up a Hudson Continuous Integration Testing Server for Ruby(and Rails)

If you work on a project with multiple developers, a continuous integration testing setup is a must. A CI package will run automated tests on a server on a set(daily) or evented(whenever commits are made) interval.

There are a number of continuous integration packages for Ruby software including CI Joe, CruiseControl.rb. I chose Hudson because it is extendable via plugins and has great support for running automated testing for non-ruby software as well.

Install hudson and jetty:

Hudson itself is a self contained Java servlet. You can either run it in a servlet container like jetty by sticking hudson.war into the webapps directory or just running it using java -jar hudson.war

On your testing server:

Set a working dir:

export HUDSON_HOME=/data/apps/test/hudson


Install ci_reporter:

sudo gem install ci_reporter

Install plugins:

cd /data/apps/test/hudson/plugins
wget http://hudson-ci.org/latest/ruby.hpi
wget http://hudson-ci.org/latest/git.hpi
wget http://hudson-ci.org/latest/rake.hpi
wget http://hudson-ci.org/latest/rubyMetrics.hpi


Setting up a Project in Hudson

We can follow these tasks to create a CI job for a Ruby on Rails 3.0 project.
Hudson will be accessible from http://localhost:8080 or http://localhost:8080/hudson if you used a servlet container.

Click ‘New Job’
Click ‘Build a free-style software project’
This lets you use a custom ruby app.

On the project configuration page:
set the git repo and the option to poll SCM
Add a Execute Shell option with:

bundle install

This will install all the necessary gems from the Gemfile.

Finally, add the rake task for the actual testing itself.
Invoke Rake

ci:setup:testunit
test
CI_REPORTS=results
RAILS_ENV=staging


You can test the setup by clicking the “Build Now” link. In the future, the “Poll SCM” will just run your tests on a set interval whenever code is pushed to your git repo.

Command line script to Pretty Print a JSON URL

Don’t you hate it when you curl an api for testing and get something ugly like:

curl http://search.twitter.com/search.json?q=ruby
q=ruby","next_page":"?page=2&max_id=20407667370&q=ruby","results_per_page":15,"page":1,"completed_in":0.018639,"query":"ruby"}

Here’s a quick one line bash script to pretty print a json url using curl and ruby:

#!/bin/bash
curl $* | ruby -e "require 'rubygems';require 'json'; jj JSON.parse(STDIN.gets)"


Save it as ppcurl, set the permissions (chmod a+x ppcurl) and run it:
ppcurl http://search.twitter.com/search.json?q=ruby

    {
      "created_at": "Thu, 05 Aug 2010 18:25:41 +0000",
      "profile_image_url": "http://a3.twimg.com/profile_images/326153315/twitterProfilePhoto_normal.jpg",
      "from_user": "der_kronn",
      "text": "interesting gem: "Zucker" http://bit.ly/9kiJ3I - cool to see, how flexible ruby is /cc @rbJL",
      "to_user_id": null,
      "metadata": {
        "result_type": "recent"
      },
      "id": 20407489891,
      "geo": null,
      "from_user_id": 20887268,
      "iso_language_code": "en",
      "source": "<a href="http://termtter.org/" rel="nofollow">Termtter</a>"
    }
  ],
  "since_id": 0,
  "refresh_url": "?since_id=20407667370&q=ruby",
  "next_page": "?page=2&max_id=20407667370&q=ruby",
  "page": 1,
  "results_per_page": 15,
  "completed_in": 0.0169860000000001,
  "query": "ruby"
}

Much nicer!

Call a JRuby method from Java

JRuby is a great way of reducing verbose Java code. Here’s a way of implementing a Java interface in JRuby and calling the method from Java.

Java Interface

We’ll start with a simple Java interface:

interface JavaInterfaceExample{
  int add(int a, int b);
}

Compile it using

javac JavaInterfaceExample.java


Implement the Java Interface in JRuby

We will implement this method in JRuby. When importing interfaces from Java code, you should use the Java namespace as in include Java::JavaInterfaceExample. The signature is important since JRuby is dynamically typed. If it isn’t specified, the parameters will be a Java Object.

require 'java'
class JrubyAdderImpl
  include Java::JavaInterfaceExample
  java_signature 'int add(int, int)'
	def add(a, b)
		a+b
	end
end

Compile it using:

jrubyc --javac -cp . JrubyAdderImpl.rb


Creating a Java class calling a JRuby method

Then let’s create a Java class which calls the jruby method:

class JavaCallerApp {
    public static void main(String[] args) {
	    JrubyAdderImpl jrubyImpl = new JrubyAdderImpl();
	    System.out.println("Adding 3+5=" + jrubyImpl.add(3,5)); // Display the string.
    }
}

Compile it:

javac -cp /usr/local/jruby/lib/jruby.jar:. JavaCallerApp.java


and run it:

java -cp /usr/local/jruby/lib/jruby.jar:. JavaCallerApp


Output:

Adding 3+5=8


Downside

While this works, it’s not a very optimally solution for calling JRuby code from Java. When you compiled JrubyAdderImpl.rb, it created a JrubyAdderImpl.java file which runs a JRuby runtime layer against the JRuby script file. I imagine this isn’t very performant.

I recommend using Mirah or Scala as alternative JVM languages for greater interoperability with Java.

I placed the code on a github repo.

Android And iPhone Development Thoughts

I recently received a Motorola Droid from Google for signing up for Google IO 2010. Unfortunately not the incredible, but the droid is actually a nice phone. The screen is really sharp. The physically keyboard lets you type faster. It’s a little too early to say, but I might keep this phone over my iPhone 3g. Anyhow, I thought Android will become more common in devices other than phones, why not make a quick app to learn a useful skill? Since I have previous iPhone development experiment, I’ll give a brief comparison of different areas i prefer in developing native iPhone and Android apps.

Apps

I created two native iPhone apps(voodoo doll and tweet show) with a few friends a year ago. Both are free and relatively simple. TweetShow actually hit the “Featured” page when it was named “TwitterTime.” It lets you watch tweets hands-free(ie. if you are eating).

For Android development, i used the Android 2.1 sdk with Eclipse. Android lets you use any IDE or even develop in the console as all of the build tools are command line-based. I made a useful Android app called Pastedroid which allows you to copy a piece of text(address, phone number, etc) to your phone. With the Pastedroid app installed, users go to pastedroid.com or use the javascript bookmarklet, copy text there and it’ll appear in the pastedroid Android app. It’s more similar to pastefire on the iPhone rather than tapbot’s awesome and more full featured paste helper, Pastebot.

Development Comparison

UI: iPhone

iPhone’s built-in UI makes it really easy to make an app good looking and consistent. Android on the other hand makes it really easy to make a horrible looking app. The iPhone interface builder even lets you drag/drop elements whereas the Android doesn’t.
Android Eclipse has a strange interface builder. It’s easier to edit the XML directly but making it look nice is difficult. Editing Android UI feels like the pre-css HTML days where table tags ruled.

UI Interaction: Android

To connect a ui button to your code, it’s a multi-step process in the iPhone sdk.
In the interface builder, click and drag an outlet to its file owner.
Then in the header file, define the outlet:

@property (nonatomic, retain) IBOutlet UISearchBar* tweetSearchBar;

Then in the code file, i can use the object.

Developing interactions is easier on Android. Just name the button in the UI xml file and you can access it in the code using:

refreshButton = (ImageButton) findViewById(R.id.refreshButton);

To make an event call a method, just set the method handler in the XML UI file.

Documentation: iPhone

I tend to learn by examples and it was definitely an easier time to find iPhone help and code examples. For Android code samples, beware of which version the help/code examples are targeted for. Not sure how true this is, but it felt the iPhone API changes were more stable than Android over the different versions.

Publishing: Android

Wow. I cannot believe how easy it is to publish an app on Android compared to iPhone. It literally took 5 minutes to get an app signed and published on the Android. I remember the first time I published an iPhone app, it took at least 5 hours of debugging various problems in the signing and build configuration.

Partial Functions in Scala

I ran into a problem where i wanted to get a formatted string for each tuple in a list but ended on a detour to learning about partial functions. Consider this simple example:

scala> val wordCount = List(("a",1),("b",2))
wordCount: List[(java.lang.String, Int)] = List((a,1), (b,2))

scala> wordCount.map(wc => wc._1+":"+wc._2)
res4: List[java.lang.String] = List(a:1, b:2)


I can just use ._1 and ._2 on each tuple but it isn’t very readable. If someone else is reading the code, will they know what ._1 is for?

We can use a case statement to the rescue:
scala> wordCount.map(case(word,count) => word+":"+count)
:1: error: illegal start of simple expression
wordCount.map(case(word,count) => word+":"+count)
^
Opps, dpp mentioned on the scala mailing list that case needs to be a partial function so this syntax won’t work.

Partial Functions

Partial Functions are a subset of regular functions but can have a restricted input  range. The Partial Function trait has two abstract members: isDefinedAt(x:A):Boolean and apply(v1:A):B

A partial function which will only be called if the isDefinedAt method is true:

scala> object PartialDef extends PartialFunction[(String, Int), String]{
| def isDefinedAt(x:(String,Int)):Boolean = x._2 > 5
| def apply(x:(String,Int)):String = x._1 + ":" + x._2
| }
defined module PartialDef

A better way would be to use the case statement as this lets us use pattern matching to name arguments:

scala> val myRestrictedPartial:PartialFunction[(String,Int),String]= {case(word,count) if count > 5 => word+":"+count}
myRestrictedPartial: PartialFunction[(String, Int),String] = <function1>
scala> myRestrictedPartial("hi", 1)
scala.MatchError: (hi,1) (of class scala.Tuple2)
scala> myRestrictedPartial("hi", 10)
res1: String = hi:10

For our example, we don't need to define a restricted argument range so we'll remove the if count > 5:

scala> val myPartial:PartialFunction[(String, Int),String]= {case(word,count) => word+":"+count}
myPartial: PartialFunction[(String, Int),String] = <function1>


We can apply the partial function to each tuple explicitly:

scala> wordCount.map(x => myPartial(x))
res14: List[String] = List(a:1, b:2)

or with the implied argument:

scala> wordCount.map(myPartial)
res15: List[String] = List(a:1, b:2)

We can also pass the partial function as an anonymous function(or closure) as an argument:

scala> wordCount.map({case(word,count) => word+":"+count})
res16: List[java.lang.String] = List(a:1, b:2)

and remove the the extra parentheses:

scala> wordCount.map {case(word,count) => word+":"+count}
res17: List[java.lang.String] = List(a:1, b:2)


As we see, my original error was that I didn’t  realize the case statement needed to be nested in an anonymous function(the curly braces)  so that it could be passed into the map function as an argument.

Thanks to dpp for bringing partial functions to my attention while fixing a minor problem.

EDIT: I originally mistaken partial functions with partially applied functions. I removed the offending code to avoid confusion.

A Better Search Engine for Research Grants

I have been working ResearchWatch, a search engine for US federal research grants. It’s ideal for scientists, researchers and government officials who need to know where research trends/money are being invested.

You can perform queries like:

science research grants

It’s still a rough prototype right now. It only has the NSF grants but I think even the current incarnation is doing a better job than the ‘official site.’

ResearchWatch provides faceted search interface and content aggregation. The content aggregation can quickly tell you the distribution of funds and location.

I’ll be using it to implement some search ideas, mainly query segmentation and entity recognition. Let me know if you have any suggestions to improve the system!

When to call methods with or without parentheses () in Scala

I ran into this method call quirk in Scala:

If the method has no parameters, you must call it without the parentheses unless the method declaration was defined with empty parentheses.

If you are new to Scala, you’ll probably see this manifest when using iterators.

scala> val wordListIter = wordList.iterator()
:9: error: wordList.iterator of type Iterator[(java.lang.String, java.lang.String)] does not take parameters
scala> val wordListIter = wordList.iterator
wordListIter: Iterator[(java.lang.String, java.lang.String)] = non-empty iterator
scala> val lines= Source.fromPath(filename).getLines
:6: error: missing arguments for method getLines in class Source;
follow this method with `_' if you want to treat it as a partially applied function
Error occured in an application involving default arguments.
val lines= Source.fromPath(filename).getLines
scala> val lines= Source.fromPath(filename).getLines()
lines: Iterator[String] = non-empty iterator

It would have been helpful if the empty parens are listed for the getLines scalaDocs

The convention ensures the uniform access principle. This convention is unlike languages like Ruby where you can call parameter-less methods with or without parentheses.

This except from page 212 of Programming in Scala explains the rationale:

instead of:
def width(): Int
the method is defined without parentheses:
def width: Int

Such parameterless methods are quite common in Scala. By contrast, methods defined with empty parentheses, such as def height(): Int, are called empty-paren methods. The recommended convention is to use a parameterless method whenever there are no parameters and the method accesses mutable state only by reading fields of the containing object (in particular, it does not change mutable state). This convention supports the uniform access principle which says that client code should not be affected by a decision to implement an attribute as a field or method.

Processing in Scala!

One of the great things about scala is that it inter-operate with any existing Java library. Both ways, use scala classes from java and java from scala. Here’s an quick example of using Processing in Scala.

I’m using IntelliJ IDEA 9.0.1 with the Scala plugin.
To get Processing working in IntelliJ:

1. Download the processing.org jars
2. Create a Scala project.
3. Add them to your libraries list in the modules setting.
4. Create a new scala object example below.

How is this useful? Scala’s syntax and its core data structures allow processing of data in a much more concise and understandable manner. Imagine you want to visualize xml or csv files with the Processing/Scala combo. Scala makes this easy with its xml and string manipulation libraries.

This example code is from Ben Fry’s Visualizing Data Book

import processing.core._
object Draw extends processing.core.PApplet {

  override def setup() {
    size(200, 200)
    noStroke( )
    fill(0, 102, 153, 204)
  }

  override def draw() {
    background(255)
    rect(width-mouseX, height-mouseY, 50, 50)
    rect(mouseX, mouseY, 50, 50)
  }

  def main(args: Array[String]) {
    val frame = new javax.swing.JFrame("Draw")
    frame.getContentPane().add(Draw)
    Draw.init

    frame.pack
    frame.setVisible(true)
  }
}

How to make a Javascript Bookmarklet

There’s a niffy trick in browsers that you you execute Javascript code in the url location bar. (This is how a lot of security holes are created too!)
Try typing the below into your browser url bar:

javascript:alert('look how cool this is')

Cool, it runs javascript!

You may have noticed a lot of websites take advantage of this to enhance a service. Instapaper’s “Read Later” saves the current page and Pastefire’s “Pastefire this” saves the selected text.

To make your own, just write javascript inside a link tag as a one liner. Then just ask users to click and drag the link to the bookmark toolbar.

Here’s an example for my Next Sprocket site which will start you on a new open source task with the title and description partially filled out:
Sprocket it!


<a href="javascript:(function(){location.href='http://nextsprocket.com/tasks/new?task[title]='+encodeURIComponent(document.title)+'&task[description]=More details at '+encodeURIComponent(window.location.href)})()">Sprocket it!</a>


Gettext On Rails 3.0.0.beta: first open source task accomplished via Next Sprocket!

Great to wake up on Monday morning to see that someone has accepted and paid for the first open source task on our website Next Sprocket! Very happy to see our concept working.

The task was for making gettext rails 3.0 compatibility and providing a sample Rails 3.0 app. Check out the solution here.

If you have a gem that you need upgraded or any open source problem, please post the task on Next Sprocket. Someone will get it done and it will be an open source benefit.