Posts Tagged ruby

About concurrency and the GIL

During RubyConf 2011, concurrency was a really hot topic. This is not a new issue, and the JRuby team has been talking about true concurrency for quite a while . The Global Interpreter Lock has also been in a subject a lot of discussions in the Python community and it’s not surprising that the Ruby community experiences the same debates since the evolution of their implementations are somewhat similar. (There might also be some tension between EngineYard hiring the JRuby and Rubinius teams and Heroku which recently hired Matz (Ruby’s creator) and Nobu, the #1 C Ruby contributor)

The GIL was probably even more of a hot topic now that Rubinius is about the join JRuby and MacRuby in the realm of GIL-less Ruby implementations.

During my RubyConf talk (slides here), I tried to explain how C Ruby works and why some decisions like having a GIL were made and why the Ruby core team isn’t planning on removing this GIL anytime soon. The GIL is something a lot of Rubyists love to hate, but a lot of people don’t seem to question why it’s here and why Matz doesn’t want to remove it. Defending the C Ruby decision isn’t quite easy for me since I spend my free time working on an alternative Ruby implementation which doesn’t use a GIL (MacRuby). However, I think it’s important that people understand why the MRI team (C Ruby team) and some Pythonistas feels so strongly about the GIL.

What is the GIL?

Here is a quote from the Python wiki:

In CPython, the global interpreter lock, or GIL, is a mutex that prevents multiple native threads from executing Python bytecodes at once. This lock is necessary mainly because CPython’s memory management is not thread-safe. (However, since the GIL exists, other features have grown to depend on the guarantees that it enforces.) [...] The GIL is controversial because it prevents multithreaded CPython programs from taking full advantage of multiprocessor systems in certain situations. Note that potentially blocking or long-running operations, such as I/O, image processing, and NumPy number crunching, happen outside the GIL. Therefore it is only in multithreaded programs that spend a lot of time inside the GIL, interpreting CPython bytecode, that the GIL becomes a bottleneck.

The same basically applies to C Ruby. To illustrate the quote above, here is a diagram representing two threads being executed by C Ruby:

Fair thread scheduling in Ruby by Matt Aimonetti

Such a scheduling isn’t a problem at all when you only have 1 cpu, since a cpu can only execute a piece of code at a time and context switching happens all the time to allow the machine to run multiple processes/threads in parallel. The problem is when you have more than 1 CPU because in that case, if you were to only run 1 Ruby process, then you would most of the time only use 1 cpu at a time. If you are running on a 8 cpu box, that’s not cool at all! A lot of people stop at this explanation and imagine that their server can only handle one request at a time and they they rush to sign Greenpeace petitions asking Matz to make Ruby greener by optimizing Ruby and saving CPU cycles. Well, the reality is slightly different, I’ll get back to that in a minute. Before I explain “ways to achieve true concurrency with CRuby, let me explain why C Ruby uses a GIL and why each implementation has to make an important choice and in this case both CPython and C Ruby chose to keep their GIL.

 

Why a GIL in the first place?

  • It makes developer’s lives easier (it’s harder to corrupt data)
  • It avoids race conditions within C extensions
  • It makes C extensions development easier (no write barriers..)
  • Most of the C libraries which are wrapped are not thread safe
  • Parts of Ruby’s implementation aren’t threadsafe (Hash for instance)
As you can see the arguments can be organized in two main categories: data safety and C extensions/implementation. An implementation which doesn’t rely too much on C extensions (because they run a bit slow, or because code written in a different language is preferred) is only faced with one argument: data safety.

 

Should C Ruby remove its GIL?

  • No: it potentially makes Ruby code unsafe(r)
  • No: it would break existing C extensions
  • No: it would make writing C extensions harder
  • No: it’s a lot of work to change make C Ruby threadsafe
  • No: Ruby is fast enough in most cases
  • No: Memory optimization and GC is more important to tackle first
  • No: C Ruby code would run slower
  • Yes: we really need better/real concurrency
  • Yes: Rubber boots analogy (Gustavo Niemeyer)
Don’t count the amount of pros/cons to jump to the conclusion that removing the GIL is a bad idea. A lot of the arguments for removing the GIL are related. At the end of the day it boils down to data safety. During the Q&A section of my RubyConf talk, Matz came up on stage and said data safety was the main reason why C Ruby still has a GIL. Again, this is a topic which was discussed at length in the Python community and I’d encourage you to read arguments from the Jython (the equivalent of JRuby for Python) developers, the PyPy (the equivalent of Rubinius in the Python community) and CPython developers. (a good collection of arguments are actually available in the comments related to the rubber boots post mentioned earlier)

 

How can true concurrency be achieved using CRuby?

  • Run multiple processes (which you probably do if you use Thin, Unicorn or Passenger)
  • Use event-driven programming with a process per CPU
  • MultiVMs in a process. Koichi presented his plan to run multiple VMs within a process.  Each VM would have its own GIL and inter VM communication would be faster than inter process. This approach would solve most of the concurrency issues but at the cost of memory.
Note:  forking a process only saves memory when using REE since it implements a GC patch that makes the forking process Copy on Write friendly. The Ruby core team worked on a patch for Ruby 1.9 to achieve the same result. Nari & Matz are currently working on improving the implementation to make sure overall performance isn’t affected.

Finally, when developing web applications, each thread spend quite a lot of time in IOs which, as mentioned above won’t block the thread scheduler. So if you receive two quasi-concurrent requests you might not even be affected by the GIL as illustrated in this diagram from Yehuda Katz:

This is a simplified diagram but you can see that a good chunk of the request life cycle in a Ruby app doesn’t require the Ruby thread to be active (CPU Idle blocks) and therefore these 2 requests would be processed almost concurrently.

To boil it down to something simplified, when it comes to the GIL, an implementor has to chose between data safety and memory usage. But it is important to note that context switching between threads is faster than context switching between processes and data safety can and is often achieved in environments without a GIL, but it requires more knowledge and work on the developer side.

 

Conclusion

The decision to keep or remove the GIL is a bit less simple that it is often described. I respect Matz’ decision to keep the GIL even though, I would personally prefer to push the data safety responsibility to the developers. However, I do know that many Ruby developers would end up shooting themselves in the foot and I understand that Matz prefers to avoid that and work on other ways to achieve true concurrency without removing the GIL. What is great with our ecosystem is that we have some diversity, and if you think that a GIL less model is what you need, we have some great alternative implementations that will let you make this choice. I hope that this article will help some Ruby developers understand and appreciate C Ruby’s decision and what this decision means to them on a daily basis.

, , , , , , , , ,

9 Comments

Ruby optimization example and explanation

Recently I wrote a small DSL that allows the user to define some code that then gets executed later on and in different contexts. Imagine something like Sinatra where each route action is defined in a block and then executed in context of an incoming request.

The challenge is that blocks come with their context and you can’t execute a block in the context of another one.

Here is a reduction of the challenge I was trying to solve:

class SolutionZero
  def initialize(origin, &block)
    @origin = origin
    @block = block
  end
 
  def dispatch
    @block.call
  end
end
 
SolutionZero.new(42){ @origin + 1 }.dispatch
# undefined method `+' for nil:NilClass (NoMethodError)

The problem is that the block refers to the @origin instance variable which is not available in its context.
My first workaround was to use instance_eval:

class SolutionOne
  def initialize(origin, &block)
    @origin = origin
    @block = block
  end
 
  def dispatch
    self.instance_eval &@block
  end
end
 
SolutionOne.new(40){ @origin + 2}.dispatch
# 42

My workaround worked fine, since the block was evaluated in the context of the instance and therefore the @origin ivar is made available to block context. Technically, I was good to go, but I wasn’t really pleased with this solution. First using instance_eval often an indication that you are trying to take a shortcut. Then having to convert my block stored as a block back into a proc every single dispatch makes me sad. Finally, I think that this code is probably not performing as well as it could, mainly due to unnecessary object allocations and code evaluation.
I did some benchmarks replacing instance_eval by instance_exec since looking at the C code, instance_exec should be slightly faster. Turns out, it is not so I probably missed something when reading the implementation code.

I wrote some more benchmarks and profiled a loop of 2 million dispatches (only the #disptach method call on the same object). The GC profiler report showed that the GC was invoked 287 times and each invocation was blocking the execution for about 0.15ms.
Using Ruby’s ObjectSpace and disabling the GC during the benchmark, I could see that each loop allocates an object of type T_NODE which is more than likely our @block ivar converted back into a block. This is quite a waste. Furthermore, having to evaluate our block in a different context every single call surely isn’t good for performance.

So instead of doing the work at run time, why not doing it at load time? By that I mean that we can optimize the #dispatch method if we could “precompile” the method body instead of “proxying” the dispatch to an instance_eval call. Here is the code:

class SolutionTwo
  def initialize(origin, &block)
    @origin = origin
    implementation(block)
  end
 
  private
 
  def implementation(block)
    mod = Module.new
    mod.send(:define_method, :dispatch, block)
    self.extend mod
  end
end
 
SolutionTwo.new(40){ @origin + 2}.dispatch
# 42

This optimization is based on the fact that the benchmark (and the real life usage) creates the instance once and then calls #dispatch many times. So by making the initialization of our instance a bit slower, we can drastically improve the performance of the method call. We also still need to execute our block in the right context. And finally, each instance might have a different way to dispatch since it is defined dynamically at initialization. To work around all these issues, we create a new module on which we define a new method called dispatch and the body of this method is the passed block. Then we simply our instance using our new module.

Now every time we call #dispatch, a real method is dispatched which is much faster than doing an eval and no objects are allocated. Running the profiler and the benchmarks script used earlier, we can confirm that the GC doesn’t run a single time and that the optimized code runs 2X faster!

 

Once again, it’s yet another example showing that you should care about object allocation when dealing with code in the critical path. It also shows how to work around the block bindings. Now, it doesn’t mean that you have to obsess about object allocation and performance, even if my last implementation is 2X faster than the previous, we are only talking about a few microseconds per dispatch. That said microseconds do add up and creating too many objects will slow down even your faster code since the GC will stop-the-world as its cleaning up your memory. In real life, you probably don’t have to worry too much about low level details like that, unless you are working on a framework or sharing your code with others. But at least you can learn and understand why one approach is faster than the other, it might not be useful to you right away, but if you take programming as a craft, it’s good to understand how things work under the hood so you can make educated decisions.
 

Update:

@apeiros in the comments suggested a solution that works & performs the same as my solution, but is much cleaner:

class SolutionTwo
  def initialize(origin, &block)
    @origin = origin
    define_singleton_method(:dispatch, block) if block_given?
  end
end

, ,

12 Comments

Discussion with a Java switcher

For the past 6 months, I have had regular discussions with an experienced Java developers who switched to Ruby a couple years ago. Names have been changed to protect the guilty but to help you understand my friend ‘Duke’ better, you need to know that he has been a developer for 10 years and lead many complicated, high traffic projects. He recently released two Ruby on Rails projects and he has been fighting with performance issues and scalability challenges.

Duke is a happy Ruby developer but he sometimes has a hard time understanding why things are done in a certain way in the Ruby community. Here are some extracts from our conversations. My answers are only based on my own experience and limited knowledge. They are probably not shared by the entire  community, feel free to use the comment section if you want to add more or share your own answers.

Threads / Concurrency

Duke: Why does the Ruby community hate threads so much. It seems to be a taboo discussion and the only answer I hear is that threads are hard to deal with and that Ruby does not have a good threading implementation. What’s the deal there? If you want concurrent processing, threads are important!

Me: This is a very good question and I think there are two main reasons why threads and thread safety are not hot topics in the Ruby world. First, look at Ruby’s main implementation itself. If you are using an old version of Ruby (pre Ruby 1.9) you don’t use native threads but green threads mapping to only 1 native thread. Ilya has a great (yet a bit old) blog post explaining the difference, why it matters and also the role and effect of the Global Interpreter Lock (GIL). Also, even though Rubyists like to say that they live in the edge, most of them still use Ruby 1.8 and therefore don’t really see the improvements in Ruby 1.9 nor yet understand the potential of fibers.

The other part of the explanation is that the Rails community never really cared until recently. Yehuda Katz recently wrote a good article on thread safety in Ruby and if you read his post and Zed Shaw’s comment you will understand a bit better the historical background. As a matter of fact, the current version of Rails is not multi-threaded by default and developers interested in handling concurrent requests in one process should turn on this option. Thread safety appeared for the first time in Rails 2.2 but from what I saw, most people still don’t enable this option. There are many reasons for that. First, enabling thread safety disables some Rails features like automatic dependency loading after boot and code reloading. A lot of Rails developers take these two features for granted and don’t understand that they are technically “hacks” to make their lives easier. I do believe a lot of Rails developers don’t understand how threads, thread safety, concurrency, blocking IO and dependencies work. They care about getting their app done and meet their deadlines. They usually use and know Rails without paying too much attention to how Rails extends Ruby. Imagine what would happen if their code wasn’t thread safe and Rails wasn’t not using a global lock by default. Now you see why things are not exactly as you expect and also why some Rubyists are getting excited about new projects like node.js which takes a different approach.

The other thing to keep in mind is that at least 90 to 95% of the Rails apps out there don’t get more than a dozen requests/second (a million requests/day). You can scale that kind of load pretty easily using simple approaches like caching,  optimize your DB queries, load balancing to a couple servers. As a matter of fact, compared to the amount of people using Rails on a daily basis, only a very little amount of people are struggling with performance and scalability like you do. This is not an excuse but that explains why these people don’t care about the things you care about.

Rails is slow

Duke: I don’t understand why Rails developers are not more concerned about the speed/performance penalty induced by Rails.

Me: Again, Rails is fast enough for the large majority of developers out there. As you know, as a developer you have to always make compromises. The Rails team always said that development time is more expensive than servers and therefore the focus is on making development easier, faster and more enjoyable. However to get there, they have to somewhat sacrifice some performance. What can be totally unacceptable for you is totally fine for others and your contribution is always welcome. This is probably the root cause of the things you don’t like in Rails. Rails was built for startups, by startup developers and you don’t fall in this category. People contributing new features and fixes are the people using Rails for what it is designed to do. There is no real ‘Enterprise’ support behind Rails and that might be why you feel the way you feel. Since you find yourself questioning some key Rails conventions and you are struggling with missing features, it looks  to me that you chose the wrong tool for the job since you don’t even use 70% of the Rails features and are dreaming of things such 3 tier architecture. Sinatra might be a better fit for you if you want lower level control, less conventions and less built-in features.

Object allocation / Garbage Collection

Duke: I recently read that Twitter was spending 20% of its request cycles in the GC, am I the only finding that concerning?

Me: Most people don’t realize how the GC works and what it means to allocate objects since Ruby does that automatically. But at the same time, most of these people don’t really see the affect of the Garbage Collection since they don’t have that much traffic or they scale in ways that just skips their Ruby stack entirely. (Or they just blame Ruby for being slow)

If you are app deals with mainly reads/GET requests, using HTTP caching (Rails has that built-in) and something like Varnish/Rack-cache will dramatically reduce the load on your server apps. Others don’t investigate their issues and just add more servers. As mentioned in a previous post, some libraries like Builder are allocating LOTS more objects than others (Nokogiri), use the existing debugging tools to see where your object allocations occur and try to fix/workaround these. In other words, Ruby’s GC isn’t great but by ignoring its limitations, we made things even worse. My guess is that the GC is going to improve (other implementations already have better GCs) and that people will realize that Ruby is not magic and critical elements need to be improved.

Tools

Duke: I really have a hard time finding good tools to help scale my apps better and understand where I should optimize my code.

Me: It is true that we area lacking tools but things are changing. On top of the built-in tools like ObjectSpace, GC::Profiler, people interested in performance/debugging are working to provide the Ruby community with their expertise, look at memprof and ruby-debug for instance. Of course you can also use tools such as Ruby-prof, Kcachegrind, Valgrind and GDB. (1.9.2 was scheduled to have DTrace support but I did not check yet). Maybe you should be more explicit about what tools you miss and how we could solve the gap.

ActiveRecord

Duke: ActiveRecord doesn’t do what I need. How come there is no native support for master/slave DBs, sharding, DB view support is buggy,  suggested indexes on queries is not built-in and errors are not handled properly (server is gone, out of sync etc..)?

Me: You don’t have to use ActiveRecord, you could use any ORM such as Sequel, DataMapper or your own. But to answer your question, I think that AR doesn’t do everything you want because nobody contributed these features to the project and the people maintaining ActiveRecord don’t have the need for these features.

What can we do?

We, as a community, need to realize that we have to learn from other communities and other programming languages, this kind of humorous graph is unfortunately not too far from reality.

Bringing your expertise and knowledge to the Ruby community is important. Looking further than just our own little will push us to improve and fulfill the gaps. Let the community know what tools you are missing, the good practices you think we should be following etc…

Take for instance Node.js, it’s a port of Ruby’s EventMachine / Python’s twisted. There is no reasons why the Ruby or Python versions could not do what the Javascript version does. However people are getting excited and are jumping ship. What do we do about that? One way would be to identify what makes node more attractive than EventMachine and what needs to be done so we can offer what people are looking for. I asked this question a few weeks ago and the response was that a lot of the Ruby libraries are blocking and having to check is too bothersome. Maybe that’s something that the community should be addressing. Node doesn’t have that many libraries and people will have to write them, in the mean time we can make our libs non-blocking. Also, let’s not forget that this is not a competition and people should choose the best tool for their projects.

Finally, things don’t change overnight, as more people encounter the issues you are facing, as we learn from others, part of the community will focus on the problems you are seeing and things will get better. Hopefully, you will also be able to contribute and influence the community to build an even better Ruby world.

,

14 Comments

How and why I joined the “suit people”

It is now official: I have traded my freedom & home office for a job title, an Aeron chair in a cubicle and a 401K.

I received my new employee package and, in less than a week, I will officially become a full-time employee at  SCEA (Sony Computer Entertainment America).

I’m going to work in the PlayStation department, working on PS3, PS2 and PSP game titles developed by various game studios.

Why ‘o why?

Why leave behind a happy life of indie contracting to join corporate America?

For many reasons actually:

  • A Team

Being a consultant I have been working with other independent consultants and existing teams. Nonetheless, I really miss being part of a stable team which grows together and learns from each other as we go through new projects and maintain old ones.

  • Long term plan

As a consultant, I usually start projects or “rescue” existing projects. I work on them for a little while and then move on. It’s exciting and rewarding but you don’t really pay the consequences of your mistakes. You usually don’t have to maintain the code you wrote and you rarely deal with the mistakes you made.

It sounds good, nobody likes to maintain the crappy awesome code they wrote 2 years ago and most developers love working on new stuff. But at the same time, to become a better engineer you need to learn from you mistakes and assuming responsibility for your bad decisions is part of the process.

It might sound weird, but I’m actually excited to work on long term projects and feel some sort of ownership over the projects. Having to support games for many years means that I’d better not mess up the implementation. And if I do, I hope I’ll quickly learn from my mistakes.

  • Avoiding burn out

There is no secret: when you are passionate about what you do, you have a hard time stopping and taking a break. I’m a recovering workaholic and it’s really hard for me to say no when I’m presented the possibility to work on interesting projects. I love what I do and I keep writing code even after I’m done with client work.

The problem is that this can start me on the slippery slope to isolating myself from friends, family and people who don’t share the same passion. I’m really lucky that my wife is a geek and loves hanging out at conferences, looking at code and playing with my buggy prototypes. But still, I spend too much time “playing” with my computer and I just can’t manage my free time wisely.

Having a full time position will hopefully help me put boundaries and will hopefully teach me to disconnect from work.

  • Exciting projects

Hey, let’s be honest, how many geeks do you know don’t want to work in the video game industry? By the way, if you don’t have a PS3, they are now at $299 and on top of getting an awesome console you get a blue ray player! (And no, I do not receive any bonuses or commissions for mentioning the console or promoting it in my blog. I had to pay for my own like everyone else.)

Corporate America? Are you going to write Java now?

No, I’m mainly going to stick to the language I love: Ruby.
From time to time I will probably use other languages here and there, but that usually makes me love Ruby even more. The reality is that Ruby’s power and flexibility seem to be appreciated by SCEA, which makes sense when you have tight deadlines and a lot of new technologies to deal with. Ruby is a perfect match!

As you can guess, I can’t go into any detail about how and why Sony uses Ruby, but let me just say that while games are still usually written in C++, they are becoming more and more interactive and need to communicate with game servers where some logic operates. Game players also need to interact with other gamers as well as check their gaming progress online, as well as the progress of the players around them etc… Basically, outside of the game engine and the console SDK, there is a lot of potential for Ruby.

Coming back to Corporate America, I have to say that I’ve known my future manager for a few years now. He’s always been a fervent Ruby advocate and has introduced lots of teams to the happiness of Ruby & Rails development. He’s also a great developer who’s contributing patches to major projects and has a bunch of cool stuff on github. To give you an idea, my job description mentions Rails, Merb, Sinatra, CouchdB, MongoDB, Redis, AWS. All these Ruby technologies are actually already used in production or are being seriously evaluated.

I’m also really looking forward to join the existing team. I know I’m going to love working with a bunch of awesome developers coming from various backgrounds.

Those who know me, know that I’m not a morning person. And while your typical office job is categorized as ’9-5′, don’t feel too bad for me. I will be joining the video game product department, and morning people are rather rare in these kinds of groups ;)

Conclusion

I’m really excited about this opportunity. For me, it is proof again that the Ruby revolution took place and that the Enterprise is evolving. Of course, time will tell if I am right, but I am quite confident.

Also, Sony is always looking for new, talented people who want to push the entertainment world to the next level. Feel free to keep in touch with me if you are interested in joining the fun.

, , ,

42 Comments

The Ruby Revolution, take II

Mohandas Karamchand Gandhi

Mohandas Karamchand Gandhi

My recent ‘Ruby revolution being over‘ blog post generated quite a lot of comments.
Let’s be honest, I did not expect less from the readers.

However, I noticed three types of reactions I would like to address:

  • It was not a Ruby revolution, it was a Rails revolution
  • The revolution has stalled due to no major enterprise backing
  • The revolution will only be over when we will reach a greater adoption

First of all, as Joe correctly mentioned, for me, the revolution is not about specifics or individuals. It’s really about the big picture.

The influence Ruby had and still has on the IT world seems to be undermined by some.
Ruby is a dynamic, truly Object Oriented programming scripting language designed for humans first.
The real paradigm shift is in the fact that Ruby was designed to make programming fast, enjoyable and easy instead of being optimized for the machines running it.
This is for me the essence of the revolution and it is meant to transcend the scope of the Ruby language.

The way I see it, Yukihiro Matsumoto (Matz) is more of an artist than a technician. He had a vision for software development. Programming languages cannot be optimized/designed for both machines and humans, the language designer has to choose which one he wants to privilege.

Most programming languages believe that it’s up to the programmer to make an extra effort since he is smarter and easier to optimize than a machine. Matz questioned this approached and decided to turn things around. The result is one of the reasons why developers seem to just fall in love with Ruby.

It was not a Ruby revolution, it was a Rails revolution.

I am not denying that there *also* was a Rails revolution.
But if you look at it, Rails and its revolution are a direct effect from Ruby’s revolution.
One might argue that it is actually an extension of Ruby’s philosophy. But what is Rails if not a web framework designed to make web development fast, easy and enjoyable?
Without Ruby there would not have been Rails and that was my point, the underlying revolution comes from the language itself.

The revolution has stalled due to no major enterprise backing.

That’s an interesting comment. It is true that .NET and Java are still dominating the enterprise world. But let’s be clear, Ruby was not designed to please “suit people”.
And to this day, there is still a strong feeling, from some individuals against the enterprise.
In the past, DHH openly said that he did not care nor wanted to hear about the enterprise, more recently, Obie Fernandez, during one of his talks said: “Fuck the enterprise” (49:39).
But the truth is that Ruby and the so called enterprise, both, are changing.
The smart people in the enterprise world saw potential in Ruby and decided to give it a chance. An easy way to include Ruby’s philosophy without breaking the fragile enterprise equilibrium was to inject Ruby in the midst of well known and respected technologies such as Java and .NET. The enterprise can now use “re-branded Ruby versions” with “new taste or ‘improved’ flavor” like JRuby, Scala, groovy, IronRuby.
I work for some enterprise clients and I can tell you that they ‘also’ use Ruby. Mainly because developers love the language.
Microsoft, Apple and SAP investing in their own implementation of the language is yet another example that the enterprise recognizes the value of Matz’s work.
Nobody can blame them to try to make Ruby fit more their requirements.
So, at the end of the day, Ruby is not the #1 enterprise language and Rails isn’t used by the large majority of enterprise web apps, but that is NOT the point. Ruby has influenced the enterprise and we will see its effects for many years.

The revolution will only be over when we will reach a greater adoption

Saying that is missing the point entirely. A revolution is a step towards a situation change. Things don’t change right away after a revolution. It takes a long time for mentalities to evolve and for people to change their habits.
The consequences of a revolution are to be studied over the decades following the event. Take smalltalk for instance. Smalltalk adoption was not that great, however it brought a paradigm shift that directly influenced languages such as Ruby, Python and Objective-C.
So, again, do not focus on the adoption but instead look at the influence of the Ruby revolution and the ripple effect around it.

,

12 Comments

The Ruby revolution is over

Liberté guidant le peuple - Eugene Delacroix posted by Matt Aimonetti

According to wikipedia, a revolution (from the latin revolutio, “a turn around”) is a fundamental change in power or organizational structures that takes place in a relatively short period of time.

Somehow, I believe this is exactly what Ruby has done in the programming world, especially with the help of Rails. Over the last few years, Ruby lead a mini revolution in the midst of software development. Thanks to Ruby, developers now look at software development differently. One thing for sure, it pushed DHH to write Rails and then convinced thousands of people to develop Ruby based applications on a daily basis.

How did it happen?

Let’s take a look at history of revolutions. Some people get frustrated their situation, they try to find workarounds until it’s just too much and the revolution kicks in.

Ruby came up with a new holistic perspective on things. Unlike most other programming languages, one of the main key value of Ruby is that writing code should feel right for the developer. You feel good about it because the language was written for humans and not machines. Basically, the language was designed to make you productive because it’s designed to please you.

As people were discovering web 2.0, Ruby also came with an opinionated framework, pushing for productivity, testing, simplicity and elegance. People started to see a new way of doing things and it quickly became the new, cool technology. Rails became a buzz word, developers were hired to work on cool projects, and books were selling by the thousands.

What did it change?

If you ask my mom, she would probably say: nothing, except that now my son works from his home office and he seems to really enjoy what he does for living.

Relatively speaking, Ruby did not change the way we work or live. However, I believe that it has influenced many software developers around the globe. Why else do you think that companies like Microsoft, Apple or SAP are working on their own implementation of the Ruby language?

When I first discovered Ruby, I was amazed at how “right” it felt, at how much fun it was to write code using its syntax and idioms. Now, if I don’t get that feeling when testing a programming language, I think there is something wrong.

The Ruby community also revived the Agile/XP world. Testing being a strong value of the community, we spent a lot of time discussing TDD, BDD, integration test as well as other practices such as pair programming, code review, sprints etc..

A few years ago, when people were asking me what programming language I would write their app in, I would reply Ruby and had to explain what it was, why it is great and would have to answer a lot of questions from potential clients. Nowadays, people don’t even argue, sites like hulu.com, twitter.com, yellowpages.com and many others are written in Ruby and it’s just part of the tools known to work very well.

The revolution is over!

Yes, Ruby made it’s revolution and the world “has changed”. But a real movement doesn’t die after its revolution, that’s actually when it has to be strong and defend its values.

This doesn’t mean that Ruby is dead or that Rails is “passé”. To the contrary, Ruby imposed itself as a new valued and respected player, a new standard if you will.

Ruby is certainly not the “new kid in the block”anymore nor the “popular kid”, however lots of older kids seem to want to have her on their team. (.NET, Java, Objective-C can all use Ruby)

The TDD + Ruby combo doesn’t surprise anyone anymore and the Enterprise is slowly but surely adopting Ruby. Ruby is now just getting better, tools and libraries are improving and the amount of users is growing.

Certainly the Ruby community is still small compared to other software developer communities, but the fundamental change was done and we are now working on improvement and keeping things running smoothly, growing and getting new ideas inspired by our experience and other communities.

Long live Ruby!

See my follow up.

,

34 Comments

RailsSummit – an amazing adventure

I have had the opportunity to go to and speak at many conferences but this year was the first time I had the chance to go to RailsSummit in São Paulo, Brazil.

I was really looking forward to this trip and I have to say it went beyond my expectations. I had really good feedback from people like the Phusion guys (Ninh & Hongli), Gregg Pollack and others.

For those who don’t know, RailsSummit is one of the biggest, if not the biggest, Ruby event in Latin America. It’s organized yearly by Locaweb, the #1 hosting company in Brazil (props to Akita & Cristiane for their work).


As part of my involvement in the Ruby community I have met a lot of Brazilians always willing to help and especially giving time to translate content. (A good example would be the Portuguese version of the Rails wiki or José Valim GSOC contribution) However, I did not realize how fast Rails was growing over here.

To come back to the conference, I have to say it was one of the best conference I have gone to. Chad Fowler, who gave the opening keynote, later told me that it reminded him of the first Ruby conferences in the US . For me, what made a huge difference was the fact that it was a very positive conference. People were happy to be here, eager to share and you could feel the passion. Unfortunately, I missed the first few Ruby conferences, but I can totally imagine how must have been. Passionate people, not trying to push their products but instead, share the cool stuff they’ve been working on. This is exactly the feeling I had during this conference.

Maybe it’s because I don’t understand Portuguese well enough or maybe it’s just a cultural thing, but the people at the conference were just super friendly and always willing to help. I was really glad to meet those who have been using some of my work, some new people to Ruby and people who don’t do Ruby but were just interested in knowing what was going on in the Ruby world.

The schedule was pretty packed and the discussions very interesting, you could certainly feel the excitement in the air. Ruby seems to catching up quickly over there. Brazilian Rubyists seem to be very pragmatic and a good illustration of that was certainly made by Vinicus Teles, Brazil Agile XPert, who shared tips on how to release a successful product.

I stayed a few days after the conference and went to visit the Rubyists in Rio. Rio is a great city. It has some of the best soccer players in the world and some seriously talented software developers. Ruby & Rails are not just the new trendy startup secret to success, companies like globo.com, currently the largest TV network in Latin America and the fourth largest in the world, also started using Ruby and Rails. I had the opportunity to visit their office and meet their teams. It’s very exciting to see how they use Agile/XP and Ruby and how they seem to be so successful. But I will keep that for another post.

Overall, even though the actual traveling to/from Brazil was a bit long, RailsSummit was a blast. I really hope to be able to come back next year, and by then, hopefully my Portuguese will have improved.

, , ,

7 Comments