I wrote a couple follow up posts you might also be interested in:

• About concurrency and the GIL
• Data safety and GIL removal

Installing Go is actually really straight forward on a Mac, and slightly harder on Linux, check this guide to see how to build Go in a few minutes.

Once all setup, you might want to read the documentation to see how to code in Go. Go is actually a kind of nice version of C with a simplified syntax, no header files, really fast compilation time, a garbage collector and a simple way to approach object inheritance without turning in the complicated mess C++ is. The language is designed around the concept of goroutines, a very nice way to handle concurrency. It also has some features that Rubyists, Pythonistas and Javascripters wouldn’t want to live without such as closures and some they probably wish they had such as defer. But of the things we are used to with dynamic languages is the concept of reflection. In a nutshell, at runtime, your code can reflect on the type of a given object and let the developer act accordingly. Depending on your programming background that might be obvious or you might not see the value. To be honest, that’s not the question here. What I’m interested in showing you is how it works.

For the sake of this demo, let’s pretend we want to have a “Dish” data model, each instance of the “Dish” type will have a few attributes, an id, a name, an origin and a custom query which really is a function that we store as an attribute. Here is how we would represent that model in Go:

// Data Model
type Dish struct {
  Id  int
  Name string
  Origin string
  Query func()
}

This is more or less the equivalent of the following Ruby code:

class Dish
  attr_accessor :id, :name, :origin, :query
end

Ruby works slightly differently in the sense that defining attribute accessors create getters and setter methods but doesn’t technically create instance variables until they are used. Here is what I mean:

shabushabu = Dish.new
shabushabu.instance_variables # => []
shabushabu.name = "Shabu-Shabu"
shabushabu.instance_variables # => ["@name"]
shabushabu.origin = "Japan"
shabushabu.instance_variables # => ["@name", "@origin"]

Another way of checking on the accessors is to check the methods defined on the object:

shabushabu.methods - Object.new.methods
=> ["name", "name=", "origin", "origin=", "id=", "query", "query="]

But anyway, this post isn’t about Ruby, it’s about Go and what we would like is to reflect on an object of “Dish” type and see its attributes. The good news is that the Go language ships with a package to do just that. Here is the full implementation:

package main
 
import(
  "fmt"
  "reflect"
)
 
func main(){
  // iterate through the attributes of a Data Model instance
  for name, mtype := range attributes(&Dish{}) {
    fmt.Printf("Name: %s, Type %s\n", name, mtype.Name())
  }
}
 
// Data Model
type Dish struct {
  Id  int
  Name string
  Origin string
  Query func()
}
 
// Example of how to use Go's reflection
// Print the attributes of a Data Model
func attributes(m interface{}) (map[string]reflect.Type) {
  typ := reflect.TypeOf(m)
  // if a pointer to a struct is passed, get the type of the dereferenced object
  if typ.Kind() == reflect.Ptr{
    typ = typ.Elem()
  }
 
  // create an attribute data structure as a map of types keyed by a string.
  attrs := make(map[string]reflect.Type)
  // Only structs are supported so return an empty result if the passed object
  // isn't a struct
  if typ.Kind() != reflect.Struct {
    fmt.Printf("%v type can't have attributes inspected\n", typ.Kind())
    return attrs
  }
 
  // loop through the struct's fields and set the map
  for i := 0; i < typ.NumField(); i++ {
    p := typ.Field(i)
      if !p.Anonymous {
        attrs[p.Name] = p.Type
      }
     }
 
  return attrs
}

Unfortunately, my code highlighter doesn’t support the Go syntax, but GitHub does, so here is a pretty version.

There are ways of running Go source code like Ruby or Python scripts but in this case, we’ll use the compiler & linker provided with Go. I named my source file “example.go”, and here is how I compiled, linked and run it:

$ 6g example.go && 6l example.6 && ./6.out
Name: Origin, Type string
Name: Id, Type int
Name: Query, Type 
Name: Name, Type string

As you can see each attribute is printed out with its name and type. The code might seem a bit odd if you never looked at Go before.
Here is a quick rundown of the code:

In our main function, we create a new instance of type Dish on which we call attributes on. The call returns a map on which we iterate through and print the attribute name (key) and type (value).
The attributes function is defined a bit below and and it takes any type of objects (empty interface) and returns a map, which is like a Hash or a Dictionary. The map has keys of String type and values of “Type” type. The “Type” type is defined in the reflect package. Inside the function, 23 then use the previously mentioned reflect package to check on the type and the name of each attribute and assign it to a map object. (note that I’m explicitly returning the map, but I could have done it in a more implicit way)

So there you go, that’s how you use reflection in Go. Pretty nifty and simple.

In the example above, I added a browse button and a text field.

I would like my users to click on the browse button, locate a folder and display it in the text field.

In your MacRuby controller, use a simple action method as well as an accessor to the text field:

1
2
3
4

attr_accessor :destination_path
 
def browse(sender)
end

Now, in Interface builder bind the destination_path outlet to the text field you want to use to display the path and bind the button to the browse action.

Let’s go back to our action method and let’s create a dialog panel, set some options and handle the user selection:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

def browse(sender)
  # Create the File Open Dialog class.
  dialog = NSOpenPanel.openPanel
  # Disable the selection of files in the dialog.
  dialog.canChooseFiles = false
  # Enable the selection of directories in the dialog.
  dialog.canChooseDirectories = true
  # Disable the selection of multiple items in the dialog.
  dialog.allowsMultipleSelection = false
 
  # Display the dialog and process the selected folder
  if dialog.runModalForDirectory(nil, file:nil) == NSOKButton
  # if we had a allowed for the selection of multiple items
  # we would have want to loop through the selection
    destination_path.stringValue = dialog.filenames.first
  end
end

That’s it, your user can now browse for a folder and the selection will be displayed in the text field. Look at the NSOpenPanel documentation for more details on the Cocoa API.

This is actually not that hard. The approach is simple and fast forward if you are used to Cocoa.

Everything starts in Interface Builder, add a custom view instance to your window.

Now switch to your project and a new file with a class called KeyboardControlView and make in inherit from NSView. We are creating a subview of NSView so we will be able to make our top view “layer” use this subclass.

class KeyboardControlView < NSView
  attr_accessor :game_controller
 
  def acceptsFirstResponder
    true
  end
 
end

As you can see in the example above, I added an attribute accessor. attr_accessor class method creates getters and setters. It’s basically the same as writing:

 def game_controller=(value)
  @game_controller = value
end
 
def game_controller
  @game_controller
end

MacRuby is keeping an eye on these accessors and let bind outlets to them.
But let’s not get ahead of ourselves, we’ll keep that for another time.

Let’s go back to our newly created class. Notice, we also added a method called `
acceptsFirstResponder` and returns true. acceptsFirstResponder returns false by default.
But in this case we want it to return true so our new class instance can be first in the responder chain.

Now that our class is ready, let’s go back to Interface Builder, select our new custom view and click on the inspector button.

Click on the (i) icon and in the Class field choose our new KeyboardControlView.
Yep, our new class just shows up by magic, it’s also called the lrz effect, just don’t ask
So now when our application starts, a new instance of our NSView class is created and Cocoa will call different methods based on events triggered.

The two methods we are interested in reimplementing are keyDown and keyUp. They get called when a key gets pressed or released.

def keyDown(event)
  characters = event.characters
  if characters.length == 1 && !event.isARepeat
    character = characters.characterAtIndex(0)
    if character == NSLeftArrowFunctionKey
      puts "LEFT pressed"
    elsif character == NSRightArrowFunctionKey
      puts "RIGHT pressed"
    elsif character == NSUpArrowFunctionKey
      puts "UP pressed"
    elsif character == NSDownArrowFunctionKey
      puts "DOWN pressed"
    end
  end
 super
end

I don’t think the code above needs much explanation. The only things that you might not understand are ‘event.isARepeat’. This method returns true if the user left his/her finger on the key. The other thing is the use of the ‘super’ call at the end of the method. Basically, we reopened a method that was already defined and we don’t want to just overwrite it, we just want to inject out code within the existing method, so once we are done handling the event, we just pass it back to original method.

Final result:

class KeyboardControlView < NSView
  attr_accessor :game_controller
 
  def acceptsFirstResponder
    true
  end
 
  def keyDown(event)
    characters = event.characters
    if characters.length == 1 && !event.isARepeat
      character = characters.characterAtIndex(0)
      if character == NSLeftArrowFunctionKey
        puts "LEFT pressed"
      elsif character == NSRightArrowFunctionKey
        puts "RIGHT pressed"
      elsif character == NSUpArrowFunctionKey
        puts "UP pressed"
      elsif character == NSDownArrowFunctionKey
  	puts "DOWN pressed"
      end
    end
    super
  end
 
  # Deals with keyboard keys being released
  def keyUp(event)
    characters = event.characters
    if characters.length == 1
      character = characters.characterAtIndex(0)
      if character == NSLeftArrowFunctionKey
       puts "LEFT released"
      elsif character == NSRightArrowFunctionKey
        puts "RIGHT released"
      elsif character == NSUpArrowFunctionKey
       puts "UP released"
      elsif character == NSDownArrowFunctionKey
        puts "DOWN released"
      end
    end
    super
  end
 
end

Now it’s up to you to handle the other keystrokes and do whatever you want. That’s it for this tip, I hope it helps.

Before we get started, let me introduce each component:

• Ruby : if you are reading this blog, you more than likely know at least a little bit about, what I consider, one of the most enjoyable programming language out there. It’s also a very flexible language that lets us do some interesting things. I could have chosen Python to do the same project but that’s a whole different topic. For this project we will do something Ruby excels at: reopening existing classes and injecting more code.
• Rack: a webserver interface written in Ruby and inspired by Python’s WSGI. Basically, it’s a defined API to interact between webservers and web frameworks. It’s used by most common Ruby web frameworks, from Sinatra to Rails (btw, Rails3 is going to be even more Rack-focused than it already is). So, very simply put, the webserver receives a request, passes it to Rack, that converts it, passes it to your web framework and the web framework sends a response in the expected format (more on Rack later).
• CouchDB: Apache’s document-oriented database. RESTful API, schema-less, written in Erlang with built-in support for map/reduce. For this project, I’m using CouchRest, a Ruby wrapper for Couch.

Goal: Log Couch requests and analyze data

Let’s say we have a Rails, Sinatra or Merb application and we are using CouchRest (maybe we are using CouchRest and ActiveRecord, but let’s ignore that for now).

Everything works fine but we would like to profile our app a little and maybe optimize the DB usage. The default framework loggers don’t support Couch. The easy way would be to tail the Couch logs or look at the logs in CouchDBX. Now, while that works, we can’t really see what DB calls are made per action, so it makes any optimization work a bit tedious. (Note that Rails3 will have some better conventions for logging, making things even easier)

So, let’s see how to fix that. Let’s start by looking at Rack.

Rack Middleware

Instead of hacking a web framework specific solution, let’s use Rack. Rack is dead simple, you just need to write a class that has a call method.
In our case, we don’t care about modifying the response, we just want to instrument our app. We just want our middleware to be transparent and let our webserver deal with it normally.

Here we go … that wasn’t hard, was it? We keep the application reference in the @app variable when a new instance of the middleware is created. Then when the middleware is called, we just call the rest of the chain and pretend nothing happened.

As you can see, we just added some logging info around the request. Let’s do one better and save the logs in CouchDB:

Again, nothing complicated. In our rackup file we defined which Couch database to use and we passed it to our middleware (we change our initialize method signature to take the DB).
Finally, instead of printing out the logs, we are saving them to the database.

W00t! At this point all our requests have been saved in the DB with all the data there, ready to be manipulated by some map/reduce views we will write. For the record, you might want to use the bulk_save approach in CouchDB which will wait for X amount of records to save them in the DB all at once. Couch also let’s you send new documents, but only save it to the DB every X documents or X seconds.

As you can see, our document contains the timestamps and the full environment as a hash.

All of that is nice, but even though we get a lot of information, we could not actually see any of the DB calls made in each request. Let’s fix that and inject our logger in CouchRest (you could apply the same approach to any adapter).

Let’s reopen the HTTP Abstraction layer class used by CouchRest and inject some instrumentation:

Again, nothing fancy, we are just opening the module, reopening the methods and wrapping our code around the super call (for those who don’t know, super calls the original method).

This is all for Part I. In Part II, we’ll see how to process the logs and make all that data useful.

By the way, if you make it to RailsSummit, I will be giving a talk on Rails3 and the new exciting stuff you will be able to do including Rack based stuff, CouchDB, MongoDB, new DataMapper etc..

The easiest way to get started requires that you have git installed as well as a gem called thor.

I let you take care of installing git on your machine, Ruby dev without git became quite challenging since GitHub started ruling the Ruby OSS world.

sudo gem install wycats-thor -s http://gems.github.com

Hops, used primarily as a flavoring and stability agent in beer, and also in other beverages and in herbal medicine.

Thor is a sort if mix between rake, sake with a better argument parser and based on Ruby classes.

Thor on its own won’t be very helpful, we need some thor tasks.

Create a folder where you want to store Merb’s source code and cd in it.

Once there download the latest merb thor tasks:

curl -L http://merbivore.com/merb.thor > merb.thor

You can now look at the available task by doing

thor -T

At this point you be overwhelmed by the multitude of options, we are working on making things a be nicer for 1.0. But anyways, let’s get started:

sudo thor merb:edge --install

The command above will install extlib, merb-core and merb-more from git HEAD. You now have the latest version of Merb’s libs installed locally as gems.

You might also want to install Merb extra plugins, if that’s the case do:

 sudo thor merb:edge:plugins --install

If you decided to use DataMapper instead of ActiveRecord or Sequel, you will also need to install the gems you might need:

Install data_objects and an adapter: (replace mysql by the adapter you want to install)

 sudo thor merb:edge:do mysql --install

Install DataMapper core from Git HEAD

 sudo thor merb:edge:dm_core --install

Install DataMapper more from Git HEAD

 sudo thor merb:edge:dm_more --install

You should be all setup by now.

Next time you want to update, just come back to the same folder and do the same thing. Hopefully by then we will have a 1 task solution and will offer a merb-stack version of our gems to let you install everything at once with all setup done.

p.s: this post is dedicated Derek Neighbors

Merb::Router.prepare do |r|
  r.resources :channels do |channels|
    channels.resources :shows do |shows|
      shows.resources :episodes
    end
  end
end

But after updating to the latest version of Merb, I got links looking like:

http://localhost:4000/channels/#<Channel:0x27b7300>/shows

The first thing to do is to read Carl’s wiki about the latest Router changes.

Carl explains that things got cleaned up in the router code and my routes should now look like:

Merb::Router.prepare do |r|
  r.resources :channels do
    resources :shows do |shows|
      resources :episodes
    end
  end
end

However that won’t be enough.. You see my url used to look like that:

url(:channel_shows, :channel_id => channel)

Now I can simplify it to:

url(:channel_shows, channel)

That still won’t fix the problem, since the real problem comes from the fact that I was on Merb HEAD but not DataMapper HEAD. Updating DM clears things up. That’s the price to pay to be on HEAD

FYI the problem comes from the fact that DM doesn’t add a to_params method to its objects. Rails users might recognize that method used to convert an object into a string to create a route, something not really ORM agnostic and frowned upon by the DM/Merb teams.

Merb lets you specify the param to use for your routes using the identify method. Read Carl’s wiki page for more cool stuff and see how to create some cool stuff like url slugs etc..

Note that even if you are using ActiveRecord, you’ll need to update merb_activerecord as the new identify rules were updated in the ORM plugins.

compounds in basil oil have potent antioxidant and is used for supplementary treatment of stress

Merb works very well with ActiveRecord and Sequel but most of the Merbivores get excited about DataMapper.

DataMapper has a lot of cool stuff going for it. I’m planning on writingi few articles about what I particularily like with DM and some of the misconceptions.

I’m going to give a talk about DataMapper during MerbCamp and something I want to cover is the fact that you can write DM adapters for virtually anything. From an adapter for couchdb (available in dm-more) to an adapter for SalesForce API. That’s the kind of stuff that gets me excited, a bit like what Ambition does but built-in in DM.

So, I decided to take some advise from Yehuda and dkubb and wrote my own adapter for Google Video. I had just finished a gem to retrieve google videos for a given google user and thought it would be a perfect exercise to mix a http-scraper with a DM adapter.

Based on the advise I received, I started by defining the API I want to use:

I matched the API calls to the underlying methods I would need to make. I then modified my original gem to support conditional calls.

Once that was done I implemented the required methods for my models to support the Model.first and Model.all calls with conditions.

A custom adapter inherits from AbstractAdapter and can define the default adapter methods:

      def create(resources)
        raise NotImplementedError
      end
      def read_many(query)
        raise NotImplementedError
      end
      def read_one(query)
        raise NotImplementedError
      end
      def update(attributes, query)
        raise NotImplementedError
      end
      def delete(query)
        raise NotImplementedError
      end

Since my adapter only needs to read data, I just had to implement #read_one and #read_many. I implemented a #read private method accessed by #read_one and #read_many as you can see here.

Because DM offers a clean and consistent API, things were pretty easy and you can check the specs to have a better understanding of how things are expected to work.

As you can see with less than 80LOC, I implemented an adapter that I can use and reuse cleanly in my apps. And on top of that, the adapter uses a standard API known by everyone using DM.

Even though, this adapter has a very limited scope, I hope this example will inspire you and at your turn, will write some more cool adapters to share with the rest of us.

UPDATE:

Sam Smoot, also known as Mr DataMapper made a very good comment. I should explain a bit more how you create a collection of objects to return when a user does a .all or .first call.

The whole collection creation is a bit strange at first.

In my read method, “set” is a DataMapper::Collection instance that is passed by the #read_one or #read_many method.

The collection needs to be loaded with an array of ordered params.

For instance in this case, to create a Video collection I need to load the collection with an ordered array of params like docid, title etc..  However, some params might be lazy loaded and in some instance, the query might be in the form of Video.all(:fields => :title)

To figure out what fields/params we need, I used:

properties = query.fields

Which retrieves the required fields. Once you have the fields you need to return the structured data and that’s when I used the #result_values method which basically loop through the fields and retrieves the data by sending the param as a method to the result:

properties.map { |p| result.send(p.field(repository_name)) }

The values once retrieved get loaded, but here is a trick I took from wycat’s saleforce API:

arr ? set.load(values) : (break set.load(values, query))

This snippet is quite simple if we set arr as true, that means we want to return an array so we will keep on looping (used by read_more when we want to return an array of objects). Otherwise we use the break operator which will stop the loop but also return a value. (yes, Ruby is awesome). By returning a single object we do exactly what’s expected and don’t have to call #first on a returned array. Pretty slick

Ginger roots, great for deployment issues

Instead, after a lot of discussions, we decided to add this feature to merb-core and let you bundle all your dependencies in a bundled gem folder. No more support for git submodules as they are hard to keep track of and don’t handle dependencies very well (not at all).

The new freezing strategy is very well described in this merbunity article. However it doesn’t really explain how to deploy a bundled app.

So let’s imagine for a second that we bundle our app, generated the scripts needed to start merb/rake etc…

The key thing to do is to have a decent deployment recipe. Let’s look at my cap recipe:

As you can see there are few key elements:

• recompile native gems for the web server
• call bin/merb and not merb directly

One thing you need to be aware of is that your bundled gems need to be up to date and play well alone. When you test your bundled app locally, you might endup loading system wide gems available which you don’t have on your deployment server.

If when you deploy your app everything seems fine but you can’t access your app, check that you are recompiling native gems during your deployment and make sure you have bundled all the deps you need and that they work well alone.

One more advise, if you can’t figure out what’s going, try ssh’ing to your server, go to your current folder and try bin/merb -i  to see what’s going on.