Nginx, Passenger, and UTF-8 in Urls

A while ago I upgraded Nginx and moved from mongrel to passenger on my companies rails cluster. Our application has a url canonocalization enforement which will redirect urls to the cirrect url if they get mangles in any way.

The mangled url http://realtravel.com/h-178793-paris_hotel-kk

Will get redirected to the conanocal form: http://realtravel.com/h-178793-paris_hotel-kk_hotel_cayré

This enforcement is done with a check to make sure that the current url matched a newly generated one.

When I upgraded nginx and started using passenger the urls I was getting inifinate redirects because passenger encodes the UTF-8. THe fix was easy, but I was a bit surprised that the urls would not be decoded by default.

URI.decode('http://realtravel.com/h-178793-paris_hotel-kk_hotel_cayr%C3%A9')

Redirects are now fixed.

Advertisements

LibXML-Ruby and XPath with namespaces

So, have you ever wasted a half hour coding while also driving yourself absolutely insane? Was it when you were playing with libxml-ruby and xpath?

Minutes ago I was coding up a xml-rpc webservice when I realized that I was unable to get the nodes that I was looking for with xpath.

As usual I searched google looking for other people having the same issue and nothing helpful came up. I knew I had to write this post when I sawthis.

So my response xml looked somthing like this:

response = <<-REMOTE_XML
<?xml ...?>
<rootNode xmlns="http://happythanksgiving.com/htgn">
  <list>
    <item>hey</item>
    <item>there</item>
  </list>
</rootNode>
REMOTE_XML

My ruby was something like this:

document = XML::Parser.string(response).parse
namespace = 'http://happythanksgiving.com/htgn'
turkeys = document.find('/htgn:rootNode//item', namespace)

But turkeys.sizewas always 0.

I the found out that I needed to add the namespace prefix to each element in the xpath find…. duhh!

document = XML::Parser.string(response).parse
namespace = 'http://happythanksgiving.com/htgn'
hotels = document.find('/htgn:rootNode//htgn:item', namespace)

Note the xpath ”/htgn:rootNode//item” changed to ”/htgn:rootNode//htgn:item” (added the namespace prefix)

Hope this helps some poor hacker or me next July when I forget and start searching google. 😉

Sharing shell with ytalk on Ubuntu

A good friend of mine years ago used to use a command-line app called ytalk to show me around the bash shell (thanks Sione!). After a short while I stopped needing his help and so I stopped using ytalk. At work we really wanted to shell-share with remote team members who were unable to use the iChat screenshare because of OS and bandwidth limitations.

I remembered that ytalk was such a good tool for being able to see what someone else was doing in the shell and to show off your bash skills. I thought it was going to be easy to setup on Ubuntu, but as it turns out, although its still an available package, it is dead on install.

So…. here is what I ended up doing and I hope that if you do the same you will be ytalk’in in no time..

On ubuntu install ytalk:


sudo apt-get install ytalk

Change the default/broken inetd.comf configuration:


talk            dgram   udp    wait    nobody.tty    /usr/sbin/in.talkd      in.talkd
ntalk           dgram   udp    wait    nobody.tty    /usr/sbin/in.ntalkd     in.ntalkd

to:


talk            dgram   udp4    wait    root    /usr/sbin/in.talkd      in.talkd
ntalk           dgram   udp4    wait    root    /usr/sbin/in.ntalkd     in.ntalkd

Note the “4” after the udp and the “nobody.tty” change to “root”

In the /etc/services file, make sure the following lines are in there:


paul@box:~$ sudo grep talk /etc/services
talk            517/udp
ntalk           518/udp

I didn’t have to change anything, but its a good idea to confirm things.

Using YTalk

Initiating the chat:

You can do this in a couple of ways, the first and most obvious way is to coordinate with another person/user and ensure that the two of you are only logged in once to the same box ad then type.


paul@box:~$ ytalk fred

Or if your logged on more than once you can specify the tty in the request after finding out which one it is:


paul@box:~$ who
fred      pts/0        2009-11-06 10:50 (208.X.X.X)
fred      pts/2        2009-11-06 10:48 (208.X.X.X)
paul      pts/3        2009-11-06 14:02 (208.X.X.X)


ytalk fred#2

More on that can be found here: http://manpages.ubuntu.com/manpages/intrepid/man1/ytalk.1.html

Thanks to euphemus for the breakthroughs!

Hope you find ytalk as useful and coolific as I do.

Enjoy!

Compiling Ruby on OSX : Readline [Resolved]

An hour before our Daily Scrum this morning I decided to recompile ruby on my Mac (OS X 10.5.8). I did this because I was trying to install passenger for development. More on that later (maybe).

I ran into the following issue with readline while I was building ruby with the--enable-shared option.


readline.c: In function ‘filename_completion_proc_call’:
readline.c:703: error: ‘filename_completion_function’ undeclared (first use in this function)
readline.c:703: error: (Each undeclared identifier is reported only once
readline.c:703: error: for each function it appears in.)
readline.c:703: warning: assignment makes pointer from integer without a cast
readline.c: In function ‘username_completion_proc_call’:
readline.c:730: error: ‘username_completion_function’ undeclared (first use in this function)
readline.c:730: warning: assignment makes pointer from integer without a cast
{standard input}:358:non-relocatable subtraction expression, "_mReadline" minus "L00000000007$pb" 
{standard input}:358:symbol: "_mReadline" can't be undefined in a subtraction expression
{standard input}:356:non-relocatable subtraction expression, "_completion_case_fold" minus "L00000000007$pb" 
{standard input}:356:symbol: "_completion_case_fold" can't be undefined in a subtraction expression
{standard input}:342:non-relocatable subtraction expression, "_mReadline" minus "L00000000007$pb" 

After google’in and coming across some similar but different solutions

Then I found this and realized that I was most likely not using the correct readline lib.

So, the issue was related to having two readline libraries installed, one in /usr/local/lib, which was installed by port as a dependency to postgres, and the other, located in /usr/lib came with OSX.

For whatever reason ruby 1.8.6 does not like to use the port library so all I had to do to get going was specify which realine library I wanted to use.


./configure --enable-shared --enable-pthread CFLAGS=-D_XOPEN_SOURCE=1 --with-readline-dir=/usr/local

Happy compiling!

FAIL: sudo gem install mysql (Fixed)

The other day I had an issue with ruby and so I went to google to fine a fix…. I laughed when the second result was my own blog. 🙂

I figured it wouldn’t hurt to save me some time next time I run into the OS Xnightmare with the mysql gem so here is what happened and what I did to fix it.

After running “sudo gem install mysql” I got the following errors:


/usr/local/bin/ruby extconf.rb
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lm... yes
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lz... yes
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lsocket... no
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lnsl... no
checking for mysql_query() in -lmysqlclient... no

As usual I looked into the mkmf.log found in the gem directory and saw a bunch of these:


"gcc -o conftest -I. -I/usr/local/lib/ruby/1.8/i686-darwin9.6.2 -I. -I/usr/local/include   -D_XOPEN_SOURCE=1  -fno-common -pipe -fno-common conftest.c  -L. -L/usr/local/lib -L/usr
/local/lib -L.      -lruby-static -lmysqlclient  -lpthread -ldl -lobjc  " 
ld: library not found for -lmysqlclient
collect2: ld returned 1 exit status
checked program was:
/* begin */
1: /*top*/
2: int main() { return 0; }
3: int t() { mysql_query(); return 0; }
/* end */

So here is what I did to fix it:


sudo ln -s /usr/local/mysql/include /usr/local/include/mysql
sudo ln -s /usr/local/mysql/lib /usr/local/lib/mysql



[heppy /usr/local/lib/ruby/gems/1.8/gems/mysql-2.7 64]$ sudo gem install mysql
Building native extensions.  This could take a while...
Successfully installed mysql-2.7
1 gem installed
Installing ri documentation for mysql-2.7...

Yeah!

Mongrel to Passenger with CPanel

I host this blog on slicehost and used to have a couple of slices, one for rails, and one for client sites, php, email etc. Just a few hours ago I moved my blog from my Rails slice to what I call my CPanel slice using passenger and the process was smooth sailing. In the process I decided to leverage what I learned about Cpanel and Passenger and I created a gem calledcpanel-passenger which can be found on github.

The gem just installs a command called cpanel-passenger that takes a bunch of parameters to modify the Apache config in a way that will not make Cpanel upset.

There is a lot of work to do to make this script do all that one would want, but at least it makes setting up a rails app on passenger a simpler task with Cpanel. Feel free to fork the gem and add to it. Its just a matter of time and the Cpanel folks will bundle passenger as a supported module, but until then try this out on your VPS that is running Cpanel.

Enjoy!

A default route gone 404 when it should

UPDATE: This worked for Rails < 2.0, but now you should follow something like this

Rails routes are a critical piece of a rails application. One issue about the routes is that there isn’t a default route for the home page of an application. Typically, one would create a controller and create a route for a default controller and default action. Here is what one of mine looks like:

map.root :controller => 'main', :action => 'home'

There is one problem with this. The url http://domain.com/blah%20blah will go to the main controller and will throw an “no action/ no id given” exception which will result in a 500 error. This is not what you want for SEO or otherwise.

The solution is quite simple, all you have to do is add a method missing to the main controller and add a method missing that logs and renders a real 404 page and http status.

  def method_missing(method, *args)
    logger.warn "action #{method} dos not exist, 404" 
    render :file => File.join(RAILS_ROOT, 'public', '404.html'), :status => 404
  end

There may be better ways to do this, but this is one way around the false 500 errors, especially if your likely to get old inbound links to your site.

Making the Rails Request Profiler and KCacheGrind Play

I have been working on optimizing my companies site after porting over many features. I have been finding the newer rails performance tools including the request profiler to be very helpful in this effort. Ryan Bates put out a great screencast on request profiling that will get you started, but if your app has any complexity, you will find out quickly like I did that the html file gets too large and is not very helpful when it crashes your browser. 😉

Assuming that you have already installed KCacheGrind on your Mac usingfink, you can do the following:

# Open up the request_profiler.rb in the actionpack gem (the code that is used by ./script/performance/request)

mate /Library/Ruby/Gems/1.8/gems/actionpack-2.2.2/lib/action_controller/request_profiler.rb

# Add the following lines of ruby to the show_profile_results method at the bottom.

        File.open "#{RAILS_ROOT}/tmp/profile-call-tree.kcg", 'w' do |file|
           RubyProf::CallTreePrinter.new(results).print(file)
           `kcachegrind #{file.path}` if options[:open]
        end

Now next time you run the request profiler you will see the KCacheGrind open up with the call tree output in it, yeah!

Changing Session Store in Rails

TIP: If you change the sessions store in rails, I would recommend also changing the session_id so your app doesn’t blow up with 500 errors on every request.

I changed the store from cookie based sessions (the default) to memcached based sessions.

Automatic hidden form fields and lightview

Ever needed to automatically add a hidden field in a form? Here is what I did to make it happen.

Not sure if its the best solution, but it worked for me… at least until the next rails release. 😉

In the original form_for code it creates a form tag which prints out the templates in the blog that is passed to it. There is a method that creates the opening form tag and it already creates extra_tags. All I do it add an additional concatenated string to the fields with the result of a custom method that I created called my_custom_extra_tags. Anything the method returns will be added to each form.

module ActionView::Helpers::FormTagHelper

  # form_tag_html overridden on line 454 in actionpack-2.2.2/lib/action_view/helpers/form_tag_helper.rb

  # original
  # def form_tag_html(html_options)
  #   extra_tags = extra_tags_for_form(html_options)
  #   tag(:form, html_options, true) + extra_tags
  # end

  # modified
  def form_tag_html(html_options)
    extra_tags = extra_tags_for_form(html_options)
    tag(:form, html_options, true) + extra_tags + my_custom_extra_tags
  end

  def my_custom_extra_tags
     (params[:lightview].blank? ? '' : hidden_field_tag(:lightview, params[:lightview]))
  end

end

I used this to show the same controller action with different templates and in my application controller I determine which template to show from a passed in parameter that cannot be lost or the template will revert back to the default template. Now all I have to do is pass a parameter lightview to the iframe source and the correct template will show before and after the form inside the iframe is submitted.

Hope this was helpful.