Comments on: WideFinder, Ruby Pic, and Scaling Up, Out and Away http://labnotes.org/2007/09/30/widefinder-ruby-pic-and-scaling-up-out-and-away/ Mon, 26 Jul 2010 17:36:41 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Assaf http://labnotes.org/2007/09/30/widefinder-ruby-pic-and-scaling-up-out-and-away/comment-page-1/#comment-138941 Assaf Sat, 01 Dec 2007 02:49:02 +0000 http://blog.labnotes.org/2007/09/30/widefinder-ruby-pic-and-scaling-up-out-and-away/#comment-138941 Mikael, the code is written to run "processes", in the pi-calculus notion of a process. Some of the tests run them as separate threads (green or native, depending on the VM) in the same OS process, and some run them in separate OS processes. But design wise, they're modeled as independent processes. Tim, code e-mailed your way. Mikael, the code is written to run “processes”, in the pi-calculus notion of a process. Some of the tests run them as separate threads (green or native, depending on the VM) in the same OS process, and some run them in separate OS processes. But design wise, they’re modeled as independent processes.

Tim, code e-mailed your way.

]]> By: Tim Bray http://labnotes.org/2007/09/30/widefinder-ruby-pic-and-scaling-up-out-and-away/comment-page-1/#comment-138782 Tim Bray Sat, 03 Nov 2007 18:21:26 +0000 http://blog.labnotes.org/2007/09/30/widefinder-ruby-pic-and-scaling-up-out-and-away/#comment-138782 I don't see a link to the actual code... post it or email it to me and I'll try it on the big many-core SPARC. I don’t see a link to the actual code… post it or email it to me and I’ll try it on the big many-core SPARC.

]]> By: Assaf http://labnotes.org/2007/09/30/widefinder-ruby-pic-and-scaling-up-out-and-away/comment-page-1/#comment-138525 Assaf Tue, 02 Oct 2007 06:16:58 +0000 http://blog.labnotes.org/2007/09/30/widefinder-ruby-pic-and-scaling-up-out-and-away/#comment-138525 Charles, the pingpong test runs twice as fast on CRuby/YARV as JRuby. I did not try the JRuby compiler yet, wouldn't be surprised if it ends up beating CRuby in future release. Charles, the pingpong test runs twice as fast on CRuby/YARV as JRuby. I did not try the JRuby compiler yet, wouldn’t be surprised if it ends up beating CRuby in future release.

]]> By: Mikael Lind http://labnotes.org/2007/09/30/widefinder-ruby-pic-and-scaling-up-out-and-away/comment-page-1/#comment-138515 Mikael Lind Mon, 01 Oct 2007 09:54:21 +0000 http://blog.labnotes.org/2007/09/30/widefinder-ruby-pic-and-scaling-up-out-and-away/#comment-138515 "The second run is parallelized, each process handles a different log file (ten in total), still single core pegged at 100%, completes in 1m30s. 2 seconds difference from pretending to parallelize." Should "process" not actually be "thread" here? I am also curious about why multi-threading (if that is what you are describing) is slower than multi-processing here. Have you considered asynchronous I/O? “The second run is parallelized, each process handles a different log file (ten in total), still single core pegged at 100%, completes in 1m30s. 2 seconds difference from pretending to parallelize.”

Should “process” not actually be “thread” here? I am also curious about why multi-threading (if that is what you are describing) is slower than multi-processing here. Have you considered asynchronous I/O?

]]> By: Charles Oliver Nutter http://labnotes.org/2007/09/30/widefinder-ruby-pic-and-scaling-up-out-and-away/comment-page-1/#comment-138513 Charles Oliver Nutter Mon, 01 Oct 2007 04:38:46 +0000 http://blog.labnotes.org/2007/09/30/widefinder-ruby-pic-and-scaling-up-out-and-away/#comment-138513 Just FYI, there's a couple facts about JRuby you should know: 1.0.1 didn't compile more than about 50% of Ruby syntax, which equated to a far smaller percentage of whole methods that would compile. That means 1.0.1 is still usually running interpreted. Also, there's still an unfortunate bottleneck in JRuby regexp process, since we have to represent strings as byte[] and all Java regex engines work with char[]. On JRuby trunk JRuby is less than 2x slower than Ruby, largely because of this bottleneck. On non-regexp benchmarks, it's always faster than Ruby. I'd expect JRuby to continue improving as we knock down the remaining bottlenecks, so check back or try trunk right now. Just FYI, there’s a couple facts about JRuby you should know:

1.0.1 didn’t compile more than about 50% of Ruby syntax, which equated to a far smaller percentage of whole methods that would compile. That means 1.0.1 is still usually running interpreted.

Also, there’s still an unfortunate bottleneck in JRuby regexp process, since we have to represent strings as byte[] and all Java regex engines work with char[]. On JRuby trunk JRuby is less than 2x slower than Ruby, largely because of this bottleneck. On non-regexp benchmarks, it’s always faster than Ruby.

I’d expect JRuby to continue improving as we knock down the remaining bottlenecks, so check back or try trunk right now.

]]>