I have been working with Pig as a more productive layer against Hadoop. Definitely check it out if you need to use map reduce. Textmate is my primary editor so I made a little bundle for syntax highlighting and script running. I’ll add snippets in the future.

Check it out at the github project page

Pig is a language designed to process large datasets. The execution layer runs uses Hadoop’s MapReduce framework.

This textmate bundle gives you syntax highlighting and the ability to run Pig scripts from Textmate.

Install:

mkdir -p ~/Library/Application\ Support/TextMate/Bundles
cd ~/Library/Application\ Support/TextMate/Bundles
git clone git://github.com/tc/pig-latin-tmbundle.git "PigLatin.tmbundle"
osascript -e 'tell app "TextMate" to reload bundles'

To run Pig scripts from Textmate:
Add the pig bin directory to your path env variable.
ex.

export PATH="/Users/tc/bin/pig-0.3.0/bin:$PATH"

Then, you can just press “Apple-R” to execute the active pig script in Textmate.