X.Y.Z

How to debug or test cron script

noreply@blogger.com (xue.yong.zhi) — Mon, 12 Jan 2009 21:31:00 +0000

If your simple cron script (/etc/cron.daily, etc/cron.hourly etc) does not run as expected, do the following:

1. be sure to have '#!/bin/sh' as the first line
2. make sure the script's name does not contains '.'. for example, you should rename 'yourscript.sh' to 'yourscript'.
3. use absolute path if you need to read/write files in the script
4. try run the script in the shell to make sure there is no obvious problem
5. 'run-parts --test /etc/cron.hourly/' will tell you what scripts will run, you should see your script in the list
6. next try to run it like cron does, for example:
cd / && run-parts --report /etc/cron.hourly
7. cron uses syslog for logging. Check your syslog config file ( /etc/syslog.conf) to see where the log is, and check the log for errors.

Ref:
[1] Why are cron.hourly files not running? SOLVED
https://lists.ubuntu.com/archives/ubuntu-users/2007-July/118973.html

Google Summer of Code with Ruby Central

noreply@blogger.com (xue.yong.zhi) — Tue, 25 Mar 2008 01:40:00 +0000

Ruby Central will once again participate the Google Summer of Code in 2008 as a mentoring organization:
http://code.google.com/soc/2008/ruby/about.html

Pat Eyler has put together an idea page, which is a great reference. While every organization may have different policy, Ruby Central has been happy to see students' own ideas in the past, and I believe this year is not an exception. Feel free to submit your own exciting ideas so that the mentors can pick the best ones.

Today is the first day for Google to accept student applications. For student who are interested in participating, my advise is to apply as early as possible: normally students who apply early has a slight advantage because mentors will have more time to read their applications.

And another advise for future GSOC students is: once accepted, please make sure you are well prepared for the program. Google and mentors normally assume you to work near fulltime in the summer, but unfortunately every year we saw 'disappearing' students during the program. If for some reasons you can not make it after being accepted, please communicate with your mentor organization early so that they can make better adjustment.

XRuby license changed to BSD

noreply@blogger.com (xue.yong.zhi) — Tue, 25 Mar 2008 00:57:00 +0000

XRuby had been licensed under GNU General Public License v2. For people who want to create solutions based on XRuby, this means they had to adopt GPL as well, which is not always possible. So since 0.3.3, we changed the license to BSD. Hopefully this will help XRuby gain more acceptance, especially from the corporate world.

XRuby 0.3.1 released

noreply@blogger.com (xue.yong.zhi) — Mon, 24 Sep 2007 14:45:00 +0000

XRuby 0.3.1 is released today and you can download the latest version here:
http://code.google.com/p/xruby/downloads/list

The major changes in this version are:

1. Ruby standard libraries are now pre-compiled and the compilation results (java bytecode) are shipped as part of the xruby.jar. This makes deployment really easy, as there is one jar file to deal with. Pre-compiled libraries means it isn't necessary to compile these code every time by the user which is helpful to performance.

The size of the final jar may cause some concerns: while the number of ruby stdlib supported by the current version of XRuby is still limited, the size of the compiled stdlib has already reached 2,000 KB (the entire xruby-0.3.1.jar is over 4,000 KB). Maybe we will ship ruby stdlib in a separated jar files in the future.

2. Use annotation and code generation to bind Java level method to Ruby level method. We started to make this change in 0.3.0 and finished it in 0.3.1. It helped us to get rid of lots of redundant code. For more information about this change, please read our last announcement and dreamhead's articles(1, 2) on his blogs (in Chinese).

Thank everyone who has contributed to this release.

xruby 0.3.0 released

noreply@blogger.com (xue.yong.zhi) — Wed, 08 Aug 2007 05:05:00 +0000

I am pleased to announce that XRuby 0.3.0 is released:
http://code.google.com/p/xruby/downloads/list

We have fixed lots of bugs and made significant improvement in the code.

Changes from 0.2.1 to 0.3.0:
1. Use annotation and code generation to bind Java level method to Ruby level method (I will talk more about this later).

2. More unit tests passed. We have not eliminated all test failures in test/ruby. But as most of the failures are caused by the implementation of builtin libraries, we will be able to fixed them soon in 0.4.0.

Changes from 0.2.0 to 0.2.1:
1) Dreamhead optimized method/block calls for methods with zero/one
arguments. It makes our performance even better.

2) ZhangYu improved Java integration significantly, he also created a wiki page with lots of good examples:
http://code.google.com/p/xruby/wiki/JavaIntegration

3) Mechiland and I made more ruby unit tests pass.

The most significant change of 0.3.0 is the using of annotation and code generation to bind Java level method to ruby level method. The idea was inspired by the discussions about Java 5 on jruby's maillist, and dreamhead turned it into reality quickly.

As we know, a Ruby method does a little bit more than a Java method. So if we have a method like this in Java:

public class RubyString {
   public RubyFloat to_f() {
       ...
   }
}

To turn it (RubyString.to_f) into a Ruby level method, we have to add a few more code to 'wrap' into a class (extends RubyMethod) and 'register' it (defineMethod), e.g:

public class String_to_f extends RubyNoArgMethod {
   protected RubyValue run(RubyValue receiver, RubyBlock block) {
       return ((RubyString)receiver).to_f();
   }
}
...
RubyRuntime.StringClass.defineMethod("to_f", new String_to_f());

For every method, we need to write similar code and it is not fun to repeat yourself. In 0.3.0, we no longer have to do this anymore. As as long as you add annotation like this:

@RubyLevelClass(name="String")
public class RubyString {
   @RubyLevelMethod(name="to_f")
   public RubyFloat to_f() {
       ...
   }
}

XRuby will turn it into a Ruby level method automatically (using ASM to generate Java bytecode).

I have not used Java 5's annotation feature before, but this looks like an very elegant solution.

Thank everyone who has contributed to this release.

ruby -y

noreply@blogger.com (xue.yong.zhi) — Tue, 12 Jun 2007 15:15:00 +0000

For people who are interested in how ruby's yacc parser works, there is an undocumented command line option("-y") that may be helpful. It will display a trace of the parser's operations.

To use it, you need to clear your RUBYOPT environment variable to NOT use "rubygems" (this will break some ruby applications), otherwise it will make too much noise.

Here is an example:


$ruby -y -e "a=1"
Starting parse
Entering state 0
Reducing stack by rule 1 (line 328), -> @1
Stack now 0
Entering state 2
Reading a token: Next token is token tIDENTIFIER ()
Shifting token tIDENTIFIER, Entering state 34
Reading a token: Next token is token '=' ()
Reducing stack by rule 418 (line 2146), tIDENTIFIER -> variable
Stack now 0 2
Entering state 90
Next token is token '=' ()
Reducing stack by rule 83 (line 827), variable -> lhs
Stack now 0 2
Entering state 73
Next token is token '=' ()
Shifting token '=', Entering state 315
Reading a token: Next token is token tINTEGER ()
Shifting token tINTEGER, Entering state 40
Reducing stack by rule 414 (line 2134), tINTEGER -> numeric
Stack now 0 2 73 315
Entering state 89
Reducing stack by rule 376 (line 1899), numeric -> literal
Stack now 0 2 73 315
Entering state 79
Reducing stack by rule 267 (line 1421), literal -> primary
Stack now 0 2 73 315
Entering state 75
Reading a token: Next token is token '\n' ()
Reducing stack by rule 217 (line 1199), primary -> arg
Stack now 0 2 73 315
Entering state 488
Next token is token '\n' ()
Reducing stack by rule 173 (line 953), lhs '=' arg -> arg
Stack now 0 2
Entering state 74
Next token is token '\n' ()
Reducing stack by rule 40 (line 616), arg -> expr
Stack now 0 2
Entering state 64
Next token is token '\n' ()
Reducing stack by rule 34 (line 596), expr -> stmt
Stack now 0 2
Entering state 63
Next token is token '\n' ()
Reducing stack by rule 6 (line 381), stmt -> stmts
Stack now 0 2
Entering state 62
Next token is token '\n' ()
Shifting token '\n', Entering state 216
Reducing stack by rule 496 (line 2429), '\n' -> term
Stack now 0 2 62
Entering state 220
Reducing stack by rule 497 (line 2432), term -> terms
Stack now 0 2 62
Entering state 300
Reading a token: Now at end of input.
Reducing stack by rule 489 (line 2416), terms -> opt_terms
Stack now 0 2 62
Entering state 299
Reducing stack by rule 4 (line 373), stmts opt_terms -> compstmt
Stack now 0 2
Entering state 61
Reducing stack by rule 2 (line 328), @1 compstmt -> program
Stack now 0
Entering state 1
Now at end of input.

XRuby's ANTLR parser does not have this option. But as ANRLE produces human readable code, you can just read the generated code or step through it in a debugger to learn how it works.

XRuby 0.2.0 released

noreply@blogger.com (xue.yong.zhi) — Tue, 15 May 2007 15:18:00 +0000

I am glad to announce that XRuby 0.2.0 is now available for download at:
http://code.google.com/p/xruby/downloads/list

Here is a summary of major changes in this release:

1. Beanworms started to work on the debugging support and we now have a basic debugger in trunk.
2. Zhang Yu and Haofei Wang added/fixed lots of builtin methods
3. Dreamhead rewrote RubySymbol as the old one was plain wrong
4. As test::unit works on XRuby, we imported unit tests from c ruby (test\ruby) to test our implementation. The result was not very impressive so far, less than half of the tests passed. We are planning to make all tests pass in the coming weeks.
5. Meanwhile, Femto is leading our efforts to create an ANTLR v3 based ruby parser in another branch. Upgrading our parser from ANTLR v2 to v3 is indeed challenging, but he has make very impressive progress.

Thank you everybody who contributed to this release!

InfoQ article on XRuby

noreply@blogger.com (xue.yong.zhi) — Tue, 17 Apr 2007 11:16:00 +0000

Werner Schuster from InfoQ has a very nice article about XRuby, you can read it here.

UPDATE 04/27/2007: InfoQ China had an interview with our core developer Dreamhead. You can read it here (in Chinese). And congratulations to Dreamhead for joining Thoughtworks.

XRuby 0.1.4 released

noreply@blogger.com (xue.yong.zhi) — Tue, 10 Apr 2007 16:15:00 +0000

I am glad to announce that XRuby 0.1.4 is now available for download at: http://code.google.com/p/xruby/downloads/list

The most significant change in this release is: one of our team member, dreamhead, single-handly created a new runtime (the core of the system that manages ruby types, methods...) to make it very close to the classic C ruby, and therefore fixed several compatibility issues with C ruby. As a translator of the Chinese version Ruby Hacking Guide, dreamhead has very good knowledge of the ruby internal and gives a great lift on the XRuby's development.

As we have learned rewriting is very difficult, we learned it again in a hard way. Dreamhead wrote most of code last year, several months before 0.1.0 release. First we tried to make a wholesale replacement of the runtime, but problems were here and there and lots of unit tests broke. Since we do not want to have a broken system in the SVN trunk, we decided to take 'refactoring' approach and merge changes step by step. It is still very hard, but dreamhead made lots of efforts and finished the merge yesterday.

Actually it may take less time if we insist on following the 'rewrite' approach (replace all the old code and fix broken tests), but the benefit of 'refactoring' is significant: it gives us a stable code base so that we made several releases during this big change.

There are more exciting news behind the scene now: Beanworms is working on the debugger support, and Femto and Yuesefa have showed interest to migrate our ruby parser to ANTLR 3.0.

Thank you for people who helps us on making this release. It has been a great learning experience working with the project.

Ruby builtin in pure Ruby

noreply@blogger.com (xue.yong.zhi) — Mon, 12 Mar 2007 03:15:00 +0000

One of the best things I love about rubinius project is: their developers try to keep the dependency on system language (C in their case) minimal, and they take it seriously. Take a look of http://code.fallingsnow.net/svn/rubinius/trunk/kernel/core/, you will find out lots of buitlin libraries are implemented in pure ruby, even lots of string functions.

Pure ruby builtin has some drawbacks, though. One is performance penalty, the other is potential side effects. For example, basically Integer#times can be implemented as simple as this in pure ruby:


class Integer
 def times  
  i = 0  
  while i < self
   yield i
   i += 1;  
  end 
  self
 end
end

But this version of Integer#times does not work exactly the same as Ruby 1.8.5. If an user want to override Fixnum#+ (this may never happen in real life):


class Fixnum
  def + x
    return 9999
  end
end

Our Integer#times's behavior will change, while Ruby 1.8.5 won't. That is because Ruby 1.8.5's implementation (int_dotimes() in Numeric.c) optimizes for Fixnum: it does not call '+' method dynamically for Fixnum, instead, it just increases the integer directly. If you want to implement this method as same as Ruby 1.8.5 then you have to write code in system language.

This kind of optimization is all over the place in c ruby. I am not clear about its motivation, but I guess performance is one of the reasons. For example, "30000000.times {|x|}" is about twice faster than "i = 0; while i < 30000000; i +=1; end" in Ruby 1.8.5.

Difference people may have different opinions on what the 'right' behavior should be. As for me, I like the behavior of the pure ruby implementation better.

XRuby is faster than Ruby 1.8.5 in most benchmarks

noreply@blogger.com (xue.yong.zhi) — Wed, 07 Mar 2007 03:18:00 +0000

Two weeks ago, Antonio Cangiano compared the performance of different ruby implementations using Ruby 1.9 (YARV)'s benchmark suite. His numbers got me thinking: all alternative implementations performed badly -- most are even way slower than ruby 1.8.5. Does it signal that JVM and .NET are bad platform for Ruby language?

With this doubt I tried the benchmark with XRuby. XRuby is a ruby compiler. Unlike other implementations, it generates Java bytecode that run directly on JVM. But at first the numbers are not impressive: the 0.1.2 version is still slower than Ruby 1.8.5 in most of the cases.

Maybe I should mention that the XRuby team had done virtually nothing for performance before, and we would avoid optimization as long as possible if it makes our code complicated. But after doing some measurements, it turns out our bad performance are largely caused by a logic 'error': as we know Ruby Fixnum can not have singleton methods, but in 0.1.2 it still lookup an empty method table. And along with some bad code practices (iterating an empty ArrayList without checking if it is empty first etc), it makes method lookup much slower than it should be.

I fixed the problem by adding about 10 lines of code, and got great result: In most benchmarks, XRuby 0.1.3 is faster than Ruby 1.8.5. For some, faster in a significant way. There are still some tests in which we are slower, but it looks like caused by poorly implemented builtin.

The following table shows the benchmark result for XRuby 0.1.3. The best part is: we did it without a method cache. YARV is still faster than XRuby, but we have lots of room to improve too.

>java -Xmx512m -jar xruby-0.1.3.jar benchmark\run.rb

Test	Ruby 1.8.5	XRuby 0.1.3
bm_app_answer.rb	fail	fail
bm_app_factorial.rb	fail	fail
bm_app_fib.rb	20.02	12.29
bm_app_mandelbrot.rb	7.099	8.252
bm_app_pentomino.rb	289.8	538.5
bm_app_raise.rb	4.846	3.986
bm_app_strconcat.rb	5.898	3.234
bm_app_tak.rb	26.14	22.12
bm_app_tarai.rb	20.89	18.35
bm_loop_times.rb	14.28	19.30
bm_loop_whileloop.rb	26.03	19.27
bm_loop_whileloop2.rb	5.257	4.786
bm_so_ackermann.rb	fail	fail
bm_so_array.rb	19.17	46.84
bm_so_concatenate.rb	5.727	9.684
bm_so_count_words.rb	2.944	45.50
bm_so_exception.rb	9.793	7.399
bm_so_lists.rb	3.666	24.59
bm_so_matrix.rb	6.249	8.452
bm_so_nested_loop.rb	15.17	13.45
bm_so_object.rb	21.49	7.991
bm_so_random.rb	6.169	5.888
bm_so_sieve.rb	2.042	2.753
bm_vm1_block.rb	64.57	38.69
bm_vm1_const.rb	47.47	25.57
bm_vm1_ensure.rb	45.54	20.01
bm_vm1_length.rb	55.50	40.89
bm_vm1_rescue.rb	39.61	20.64
bm_vm1_simplereturn.rb	56.02	29.06
bm_vm1_swap.rb	76.35	30.52
bm_vm2_array.rb	19.34	8.532
bm_vm2_method.rb	33.72	19.63
bm_vm2_poly_method.rb	45.23	20.62
bm_vm2_poly_method_ov.rb	12.64	8.261
bm_vm2_proc.rb	21.08	17.86
bm_vm2_regexp.rb	13.09	30.87
bm_vm2_send.rb	11.71	15.75
bm_vm2_super.rb	13.92	7.510
bm_vm2_unif1.rb	11.30	8.292
bm_vm2_zsuper.rb	15.71	7.740
bm_vm3_thread_create_join.rb	0.110	1.331

* The test environment is Intel Pentium M 1G CPU, 1G Memory, Windows XP SP2, Java 1.5.0_09.

XRuby 0.1.2 released

noreply@blogger.com (xue.yong.zhi) — Thu, 01 Mar 2007 03:13:00 +0000

XRuby 0.1.2 is now available for download from the project download page: http://code.google.com/p/xruby/downloads/list

This latest release contains more built-in libraries and a few bug fixes for the compiler. We expect to release a new stable version every month.

Last week, Antonio Cangiano did a very interesting job to compare the performance of different ruby implementations using YARV's benchmark suite. XRuby is not included in his test as Antonio may not know our project exists. For people who are interested, here is the XRuby's performance data (due to my limited resource, only ruby 1.8.5 interpreter was compared):

>java -Xmx512m -jar xruby-0.1.2.jar benchmark\run.rb

Test	Ruby 1.8.5	XRuby 0.1.2
bm_app_answer.rb	fail	fail
bm_app_factorial.rb	fail	fail
bm_app_fib.rb	20.42	33.84
bm_app_mandelbrot.rb	7.270	15.33
bm_app_pentomino.rb	294.2	956.9
bm_app_raise.rb	4.906	4.597
bm_app_strconcat.rb	5.917	5.858
bm_app_tak.rb	26.46	38.95
bm_app_tarai.rb	21.08	35.51
bm_loop_times.rb	14.83	93.55
bm_loop_whileloop.rb	26.26	76.5
bm_loop_whileloop2.rb	5.327	16.15
bm_so_ackermann.rb	fail	fail
bm_so_array.rb	19.35	300.2
bm_so_concatenate.rb	5.868	26.58
bm_so_count_words.rb	2.253	46.43
bm_so_exception.rb	9.893	9.973
bm_so_lists.rb	3.696	41.04
bm_so_matrix.rb	6.328	21.29
bm_so_nested_loop.rb	15.38	68.20
bm_so_object.rb	21.97	20.77
bm_so_random.rb	6.269	15.69
bm_so_sieve.rb	1.992	5.476
bm_vm1_block.rb	65.67	98.91
bm_vm1_const.rb	48.05	82.31
bm_vm1_ensure.rb	46.19	75.38
bm_vm1_length.rb	55.62	105.6
bm_vm1_rescue.rb	36.47	80.91
bm_vm1_simplereturn.rb	57.12	88.16
bm_vm1_swap.rb	76.21	85.73
bm_vm2_array.rb	18.64	19.65
bm_vm2_method.rb	33.41	35.32
bm_vm2_poly_method.rb	45.42	47.78
bm_vm2_poly_method_ov.rb	12.81	25.98
bm_vm2_proc.rb	21.54	31.23
bm_vm2_regexp.rb	13.40	52.35
bm_vm2_send.rb	11.87	31.60
bm_vm2_super.rb	13.98	20.25
bm_vm2_unif1.rb	11.46	19.95
bm_vm2_zsuper.rb	15.84	21.92
bm_vm3_thread_create_join.rb	0.120	1.402

The test environment is Intel Pentium M 1G CPU, 1G Memory, Windows XP SP2, Java 1.5.0_09. It may be worth to mention we have done almost nothing for optimization, not even a simple method cache. I think it is quite possible for a JVM based ruby compiler to beat the classic Ruby interpreter on performance, while YARV's number is going to be very hard for us to reach.

On-Ruby interview

noreply@blogger.com (xue.yong.zhi) — Mon, 19 Feb 2007 03:33:00 +0000

Pat Eyler recently interviewed XRuby team, the first episode is now posted on his popular On-Ruby blog: http://on-ruby.blogspot.com/2007/02/serial-xruby-interview-episode-i.html

XRuby 0.1.0 released

noreply@blogger.com (xue.yong.zhi) — Mon, 29 Jan 2007 22:16:00 +0000

[Link of this article: http://xruby.blogspot.com/2007/01/xruby-010-released.html]

Today I am glad to announce the release XRuby 0.1.0 , a ruby compiler which compiles ruby script (.rb) to java bytecode (.class). You can download it from here: http://code.google.com/p/xruby/downloads/list

XRuby 0.1.0 is the first release of XRuby. It is able to pass all tests in samples/test.rb - a decent test suite that comes with ruby installation.

As a compiler, XRuby compiles ruby source code so that your program can run on top of Java Virtual Machine. For example, if you have a ruby script like this:

class MyClass
  def say_hello_three_times
    3.times {puts "hello"}
  end
end

MyClass.new.say_hello_three_times

You can compile it with XRuby (assuming its name is test.rb):

>java -jar xruby-0.1.0.jar -c test.rb

The compiler will generate a test.jar. It can be launched as a regular java application, with the following command:

>java -jar test.jar
hello
hello
hello

For your convenience, you can also run the script directly (without -c option), just like the classic ruby interpreter. Under the hood it compiles the script then run the bytecode:

>java -jar xruby-0.1.0.jar test.rb
hello
hello
hello

While little known, this project has been under active development for one year and eight months. I started the project around mid 2005, and it took me about eight months to write a ruby parser with Antlr, and then one year to implement the compiler and runtime. Some people joined me during the second half of 2006 and made great contribution. It is just the beginning of an exciting journey.

We still have a long way to go before claiming XRuby is a very competent alternative implementation of ruby. And comparing with our peers we lag behind in implementing ruby's built-in libraries and do not have Ruby-Java bridge. Our next focus is to improve this situation and make major ruby libraries and framework work under XRuby. And we hope by end of 2007 you can use XRuby to compile your Ruby On Rails application to a .war file and run it directly on a J2EE server.

We appreciate your feedback. Our development maillist is at:
http://groups.google.com/group/xruby-devel

I also setup a new blog to be used as XRuby's team blog: http://xruby.blogspot.com . While new articles will be cross posted for quite a while, I encourage you to subscribe http://feeds.feedburner.com/xruby if have not done so.

Thank you and have fun.

XRuby 0.1.0 will be released in January, 2007

noreply@blogger.com (xue.yong.zhi) — Tue, 02 Jan 2007 16:45:00 +0000

Well, the release date of xruby 0.1.0 (our first release) has to be pushed back a little due to the lack of time of our projects members.

The goal of the xruby 0.1.0 is to pass all the tests in ruby's sample\test.rb, and we are very close to the goal. Right now we still have a few failed tests to be fixed, the majority of them are serialization (Array#pack, Marshal) and eval related. Most technical obstacles of compiling ruby have been solved, it just takes time to do the implementation (especially for the large amount of builtin libraries).

2007 is going to be an exciting year:)

XRuby Hacking Guide on wiki

noreply@blogger.com (xue.yong.zhi) — Tue, 24 Oct 2006 18:05:00 +0000

Update 12/20/2006: As Google Code now provides Wiki by itself, I just move the project wiki over there as it is better integrated.

Update 12/21/2006: Oops, now I regret what I did...Google Code only allow project members and owners add and edit wiki pages, which make it almost useless.

Right now the only document we have is XRuby Hacking Guide. The goal of this guide is to help users/developers understand XRuby's implementation.

XRuby project is now hosted on Google Code

noreply@blogger.com (xue.yong.zhi) — Sun, 10 Sep 2006 17:51:00 +0000

XRuby is an open source Ruby compiler which compiles Ruby source code (.rb) to Java bytecode (.class). On last Friday I moved the source code from my local Subversion server to Google Code(code.google.com/p/xruby/), so the project is now public available (GPL license).

Although I have never announced this project, I have been working on it for more than one year. This February I made the parser public available, now the main effort is on the compiler backend (java bytecode generation). I expect to release the first stable version at the end of 2006.

So far I am not very satisfied with my progress. I have a busy day job which has nothing related to Ruby and Java, so can only manage to work on XRuby about one hour a day(except weekends and holidays). When I started the project I thought XRuby is going to be the first ruby compiler in the world. But now Gardens Point Ruby.NET has released their first alpha version, and I expect JRuby will start to do their compiler in a few months (since they have been hired by Sun to work on JRuby full time).

Anyway, the project gives me lots of fun and the implementation starts to gain momentum. It looks very promising to see the first stable release around this new year, stay tuned:)

Operator overloading in ruby

noreply@blogger.com (xue.yong.zhi) — Wed, 09 Aug 2006 17:37:00 +0000

Most operators in ruby (e.g. +, -, *, /, % etc) are actually method calls, and they can be overloaded just like other method calls. But unlike C++, Ruby does not have operators like +=, -=, *=, /= etc, so a += 1 is translated directly to a = a + 1.

This involves some performance penalty in some cases, for example:


a1 = [1, 2, 3]
a2 = [4, 5, 6]
a1 += a2

Behind the scenes, a new array is created (as a1 += a2 is equivalent to a1 = a1 + a2, operator + will create a new array, then a1 is assigned to hold the reference to the new array, and the old a1 will be garbage collected). If you just want to append a2 to a1, the following code is a little bit more efficient:


a1.concat(a2)

concat method will directly update the content of a1 without creating a new array.

And it is surprising to know that not all ruby's operators are method calls, so you can not overload !, not, &&, and, ||, or.

UPDATE 09/12/2006: Chris pointed out an error in my original article(I shouldn't use << operator as the alternative of +=), see comments for details. Thanks, Chris!

Reference:
[1]Are +, -, * ... operators?

Rubyfront 0.2.0 released

noreply@blogger.com (xue.yong.zhi) — Sun, 30 Apr 2006 19:15:00 +0000

This new version of Rubyfront fixed a bug found when parsing Ruby on Rails 1.1.2. You can download the source code from here.

In rubyfront 0.2.0, I changed the way it handles expression substitution. The older versions of rubyfront ignores the content in expression substitution. For example, if the input is:


"hello, #{name}"

The lexer will first start to scan the input as a double quote string. After it sees "#{", it will go find the first '}' and think that is the end of the expression substitution. Unfortunately, expression substitution may contain '}' itself. For example:


"begin #{1.times {"block"}} end"

The lexer can not match the above input as one string. Expression substitution can contain any legal ruby program (compoundStatement), and only the parser understands the structure.

To correctly handle this situation, the new lexer will suspend its state when the start of expression substitution is seen (only 'begin ' is matched so far). After the parser finishes its processing of expression substitution, it will notify the lexer and the lexer will resume its state and matches the rest of the string (' end').

Btw, when you run parser smoke test against Ruby on Rails 1.1.2, you will see the following failures:


C:\Documents and Settings\xxx\workspace\rubyfront>build parsersmoketest
<...>
parsersmoketest:
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\activerecord-1.14.2\test\connections\native_sqlite3\in_memory_connection.rb: line 18:76: expecting EOF, found ')'
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\environments\environment.rb: line 8:1: unexpected token: <
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\controller\templates\controller.rb: line 1:7: unexpected token: <
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\controller\templates\functional_test.rb: line 1:41: unexpected token: ..
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\controller\templates\helper.rb: line 1:8: unexpected token: <
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\integration_test\templates\integration_test.rb: line 3:7: unexpected token: <
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\mailer\templates\mailer.rb: line 1:7: unexpected token: <
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\mailer\templates\unit_test.rb: line 4:7: unexpected token: <
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\migration\templates\migration.rb: line 1:7: unexpected token: <
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\model\templates\migration.rb: line 1:7: unexpected token: <
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\model\templates\model.rb: line 1:7: unexpected token: <
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\model\templates\unit_test.rb: line 1:41: unexpected token: ..
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\plugin\templates\generator.rb: line 1:7: unexpected token: <
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\plugin\templates\unit_test.rb: line 3:7: unexpected token: <
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\scaffold\templates\controller.rb: line 1:7: unexpected token: <
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\scaffold\templates\functional_test.rb: line 5:7: unexpected token: <
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\scaffold\templates\helper.rb: line 1:8: unexpected token: <
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\session_migration\templates\migration.rb: line 1:7: unexpected token: <
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\web_service\templates\api_definition.rb: line 1:7: unexpected token: <
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\web_service\templates\controller.rb: line 1:7: unexpected token: <
     [java] parser exception for C:\ruby\lib\ruby\gems\1.8\gems\rails-1.1.2\lib\rails_generator\generators\components\web_service\templates\functional_test.rb: line 1:41: unexpected token: ..
     [java] 2428 ruby programs have been parsed, 21 failed.

Those errors are expected as those files are in .rhtml format (not valid ruby, ROR requires erb to preprocess them).

Compile-time type inference for ruby has its limit

noreply@blogger.com (xue.yong.zhi) — Tue, 18 Apr 2006 01:49:00 +0000

I have been thinking about doing type inference at compiler time for ruby. Whiles lots of the cases can be handled in a straightforward way, type inference won't work in all situations.

One of the problems is, ruby supports multiple return types:


def f
  if rand(2) == 1
       return 1
  else
       return "hello"
  end
end

In the above code, method 'f' will randomly return 1 (Fixnum) or "hello" (String). At compiler time there is no way for a compiler to know what type to return. All it can do is to insert meta data so that typing checking can be done at runtime.

And multiple return types is not uncommon in real code. For example, Ruby' s array or hash can hold objects of different types:


  a = [1, 1.5, "hello"]

In the above code, Array 'a' contains three different types of objects: Fixnum, Float and String. And element reference("[]") is an example of method that has multiple return types.

First release of Rubyfront

noreply@blogger.com (xue.yong.zhi) — Fri, 17 Feb 2006 04:55:00 +0000

UPDATE 02/27/2006: a new version of rubyfront was released, and now it is capable to parse ruby's stable snapshot and Ruby on Rails. Thanks Mauricio Fernandez for reporting the bugs.

I am glad to announce the first release of rubyfront (a ruby parser powered by antlr), you can download it here.

Overview
The goal of this project is to create an error checking tool for ruby. Currently we are using antlr as the parser generator and java as the implementation language.

Current Status
Even though Rubyfront is in its very early stage, the current parser can accept the entire ruby standard library. So far the parser only do error checking at syntax level, no semantic checking is implemented.

What's next
We will focus on improving the parser to make it closer to C ruby's yacc counterpart.

Requirement
J2SE 5.0 and up.

Demonstration
1. Open build.xml and change "rubycode.dir" to the directory of your ruby source code.
2. Run the smoke test. On windows, you can simply type "build parsersmoketest".

For example, if you have Ruby 1.8.4's source code installed on "c:\ruby", you are going to see the following:


parsersmoketest:
 [java] 1574 ruby programs have been parsed, 0 failed.

The above result shows that all 1574 ruby programs are accepted by the parser.

By the way, if you run smoke test against Ruby's windows One-Click installer, you are going to see a few failures from fxruby 1.2.6 and others. The good news is: they are known to be broken (You can use "ruby -c" to verify). If you have Ruby on Rails installed, you are going to see more failures. But all of the them are expected, since those are actually .rhtml files.

Ruby interpreter crashes with invalid input

noreply@blogger.com (xue.yong.zhi) — Mon, 09 Jan 2006 16:51:00 +0000

Last week I found a bug in ruby's parser, NULL pointer reference will happen when the following program is parsed:


C:\ruby-1.8.4-i386-mswin32\bin>ruby -e 'def a=.a=;end'
-e:1: identifier a= is not valid
-e:1: [BUG] Segmentation fault ruby 1.8.4 (2005-12-24) [i386-mswin32]
This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

Now it was fixed in CVS.

This is not a serious security problem. I tried to find if it will result in a remote DOS vulnerability in Ruby on Rails, but find nothing.

Why "undef" is keyword in ruby

noreply@blogger.com (xue.yong.zhi) — Fri, 06 Jan 2006 17:58:00 +0000

undef is a keyword in ruby. But its syntax:
"undef method-name (, method-name)*", looks just like a normal command call (function call without parameter, e.g. print "hello"). Then why not just implement it as a built-in method? As we know, "private", "protected" and "public", which are keywords in lots of other language, are just implemented as plain method in ruby.

What makes "undef" special is in attribute methods. In ruby we can define method like this:


class A
  def a=(value)  
    @value = value  
  end
end

And later, we can undefine "a=" with "undef":


undef a=

"a=" is not a valid identifier(because of the '=') unless the parser is expecting a method name, for example, after "alias", "def" or "undef".

Note ":a=" is valid symbol. "private", "protected" and "public" are normal methods since they only accept symbols:


class MyClass

   def method1
   end

   public :method1
  #This is invalid: public method1
end

Since "undef" is followed by method name, it has to be implemented as a keyword, so the parser/lexer knows when to change its state to accept the extra '='.

MiniRails

noreply@blogger.com (xue.yong.zhi) — Fri, 16 Dec 2005 18:48:00 +0000

Lots of people are using web technologies in their desktop applications, and it is especially common to find a windows application which embeds WebBrowser control to render great GUI elements. There are some bumps, for example, you can not do disk/socket IO with javascript(even java applet), which most applications have to reply on.

To overcome this, we can either use ActiveX control(I am using it myself and am satisfied with it), or launch a web server on the background. The second has an additional advantage: the server and client are decoupled, so in the future you can easily migrate to a pure web environment without touching too much code. While this approach looks attractive, however, we have to face a reality: most web application servers can be too big for most desktop applications to distribute. Note I am taking web application servers here, not simple web servers. It is easy to find a small footprint web server or even write one by yourself, but your development style will have to go back to the 80s (CGI in C). Even though Tomcat Embedded 5.5.12 can be downloaded as a 3.1 MB zip file, your customers may not have JVM installed in the first place. Cassini suffers the same problem, Microsoft has not pushed hard enough to make .NET universal.

How about the Ruby on Rails? It turns out to be a great choice. While a complete distribution takes lots of disk space, it is easy to customize so that we only ship what we need.

Here is how I build my minimum ruby distribution that rail can run on(use windows as example):

1. Have ruby installed to C:\ruby. Later we need to copy files from it.

2. Download rails stand-alone packages, uncompress it to C:\rails, and empty doc/ folder to save space.

3. Copy ruby.exe and msvcrt-ruby18.dll from C:\ruby\bin to C:\rails.

4. Copy the following OS independent libraries (implemented in pure ruby) from C:\ruby\lib\ruby\1.8 to C:\rails\lib\ruby\1.8:


 base64.rb
 benchmark.rb
 cgi.rb and cgi/
 date.rb
 date/format.rb
 delegate.rb
 drb.rb and drb/
 e2mmap.rb
 erb.rb
 English.rb
 fileutils.rb
 ipaddr.rb
 irb.rb and irb/
 kconv.rb
 logger.rb
 mutex_m.rb
 net/http.rb
 net/protocol.rb
 net/smtp.rb
 observer.rb
 optparse.rb and optparse/
 parsedate.rb
 pathname.rb
 pstore.rb
 rational.rb
 rexml.rb and rexml/
 set.rb
 singleton.rb
 soap.rb and soap/
 tempfile.rb
 thread.rb
 time.rb
 timeout.rb
 tmpdir.rb
 uri.rb and uri/
 webrick.rb and webrick/
 xmlrpc/
 xsd/
 yaml.rb and yaml/

5. Copy the following OS dependent libraries from C:\ruby\lib\ruby\1.8\i386-mswin32 to C:\rails\lib\ruby\1.8\i386-mswin32:


 digest.so
 digest/md5.so
 digest/sha1.so 
 fcntl.so
 nkf.so
 rbconfig.rb
 socket.so
 strscan.so
 stringio.so
 syck.so

6. Now we may still need a database (optional). Here is how I get sqlite3 installed:
a) Download precompiled binaries. Copy sqlite3.exe and sqlite3.dll into C:\rails folder.
b) Download SQLite-Ruby. Copy sqlite3_api.so to C:\rails\lib\ruby\site_ruby\1.8\i386-msvcrt\, copy sqlite3.rb and sqlite3/ to C:\rails\lib\ruby\site_ruby\1.8
That's it! You can start WEBrick now:


C:\rails>ruby script\server
=> Booting WEBrick...
=> Rails application started on http://0.0.0.0:3000
=> Ctrl-C to shutdown server; call with --help for options
[2005-12-16 15:23:09] INFO  WEBrick 1.3.1
[2005-12-16 15:23:09] INFO  ruby 1.8.2 (2004-12-25) [i386-mswin32]
[2005-12-16 15:23:09] INFO  WEBrick::HTTPServer#start: pid=440 port=3000

And do whatever rails developement as you want.

This distribution takes 6.28M on disk. If you zip it, it shrinks to a 2.3M file.

Good experience with boost memory pool

noreply@blogger.com (xue.yong.zhi) — Tue, 13 Dec 2005 23:55:00 +0000

With increasing number of source code to compile, the compiler I maintain becomes unacceptable slow. After profiling the code with AQTime, I found about 30% percent CPU is used by malloc(), and half of them are for tree node allocation(fixed size).

After testing the performance of boost memory pool, I simply replaced malloc() with boost::object_pool::malloc(). The result is great, it almost eliminated the cost of tree node allocation. The total amount of time used for malloc is down to 13%. Still plenty of room for improvement, but the performance is at least acceptable.

There is one thing to remember: boost::object_pool is designed for "allocate many objects and then destroy all of them" type of scenario, so freeing memory from the pool can be very slow. For example, at the beginning I also replaced free() with boost::object_pool::free(), which make the performance even worse: boost::simple_segregated_storage::find_prev() and boost::simple_segregated_storage::nextof() start to take over all the CPU. So I do not do free anymore and simply let ~object_pool() to release the memory. This works perfect for my compiler, but may not be good for other applications.