For an unknown reason Ebuyer.com was spamming me for years and wouldn’t take me out of their spam distributor. So now they’re blacklisted on the SMTP level. No more spam to any of us or any of our customers/domains.

mail:~# cat /etc/exim4/local_sender_blacklist 
*@*ebuyer.com

Bye bye Ebuyer spam

One would expect that considered the complexity of gdm there would be a way to have per user X11 options. So no, there isn’t one. At least not after the great rewrite after V2.22.

However good old Unix paradigms can help us (this is all under Debian, other Unices will allow a similar trick):

$ vim /usr/local/bin/X

Add something similar like this:

#!/bin/sh
#
# start X with different options depending on user

# check if the parent gdm process that started us contains
# "--username le_gamer" in its commandline
#
if ps -p $PPID -o args= | grep -q 'username le_gamer';
then
  exec /etc/X11/X.orig $* -config /etc/X11/xorg.conf.le_gamer
else
  exec /etc/X11/X.orig $*
fi

We check whether gdm is starting us and using the parameter “–username le_gamer”. If it is, then we’re using a different config file for X.

Of course you’ll need to adapt all this, unless you are using Debian or Ubuntu. You need to adapt the path to the X Server, to the config files etc.

Why did I do that? The problem is that Intel’s X server in Ubuntu Lucid Lynx 10.04 is really unstable with DRI on a Intel GM965/GL960 graphics card.

So unless I’m playing 3D games which are a lot faster with DRI, I don’t want to enable DRI (more over, “NoDRI” is pulling less power out of my battery). So my normal config contains this:

$ cat /etc/X11/xorg.conf
[...]
Section "Device"
        Identifier      "Configured Video Device"
        Option          "NoDRI"
EndSection

So now, the only thing you need to do is replace your “normal” X Server with the new “adaptable” one:

$ sudo mv /etc/X11/X /etc/X11/X.orig
$ ln -s /usr/local/bin/X /etc/X11/X

That’s it,

Tomáš Pospíšek

There might be a reason to upgrade to rubygems 1.7.x, however if you have done so by misstake and find your console swamped by thousands of deprecation warnings:

NOTE: Gem::Specification#default_executable= is deprecated with no replacement. It will be removed on or after 2011-10-01.

and you’re crying desperately for help, then you might try the following very ugly hack, which you should revert at some point in time:

First look where your rubygems are installed:

$ gem list -d rubygems

Find the deprecate.rb file there, and make the def self.skip class method allways return true. That’s it. Happy hacking on. And don’t forget, that you’ll need to undo this change sometime!

Tomáš Pospíšek

Problem: I have a web site/page that I visit regularily which I want to annotate with my notes.

More specifically, I was regularly searching through the Homegate real estate hub looking for a new home. It goes without saying that I was again and again forgetting which objects I had already looked at, which objects were really interesting and I should check out more closely.

Therefore the need to annotate search results.

The here presented approach should be applicable for annotation of other web pages as well. It’s based on the observation, that restful web applications need to operate with asset IDs. These asset IDs can be reused to enrich the asset locally with additional data, such as notes. Thus we’re looking for those IDs in specific elements of the page and add a bit of HTML markup to those places.

Of course that aproach only works as long as the web site doesn’t heavily change its markup and doesn’t rendomly change asset IDs.

Here’s a screenshot of regular Homegate search results:

And here’s the same page after scripting it with Greasemonkey:

You’ll notice the input field with the comment in it.

The idea is simple: add an input box to each search result, where you write your comment. When the focus leaves the input box, the comment is stored to localStorage.

Starting with Greasemonkey is quite easy. There are howtos and templates to start from, such as this one.

However, regular JavaScripting and Greasemonkey JavaScripting do not work in exactly the same way:

One of the differences is that Greasemonkey creates a separate JavaScript environment, in which the Greasemonkey scripts are executed. That is calling Javascript contained in the page from Greasemonkey and the inverse calling Greasemonkey scripts from the page is not possible by default. This is on purpose, so that the web page can not detect and not interfere with the Greasemonkey scripts, because you want your Greasemonkey scripts to work allways on some page, whether or not that page likes it or not. Therefore no interference is possible.

Greasemonkey however provides a standard way to access the web page’s scripts, and that’s through the “unsafeWindow” object, which is a reference to the web page’s environment.

I had two mechanisms I had to make accessible using the “unsafeWindow” handle:

• the first was accessing JQuery, which is included by default by the Homegate page. Since I needed to use JQuery functions in my Greasemonkey script, I got a reference to it via the standard Greasemonkey precedure:

    var jQuery = unsafeWindow['jQuery'];

• the second mechanism that needed to cross the boundaries between the web page and Greasemonkey was callbacks from the web page to my Greasemonkey script. This is necessary, because I’m attaching “input” elements to each search result, which contain a note and which, “onblur”, need to call a function that saves the content of the input box. Here’s part that constructs the input element:

    jQuery("<input onclick='event.cancelBubble = true;'" +
                 " onblur='saveComment(this, immoID);'>").insertAfter(immoElement);

And this is the function that gets called back by “onblur”:

    unsafeWindow.saveComment = function(element, immoID) {
      unsafeWindow.localStorage.setItem(immoID, element.value);
    };

The next interesting thing you’ll note is usage of ‘locaStorage’. Support for the latter in browsers does not seem mature yet. One problem I’ve encountered when developing under Firefox 3.6 was that saving to ‘localStorage’ was not possible when cookies were disabled (see this report). Thus you’ll need to permanently enable cookies for Homegate in order for the script to be able to save its data.

Finally, while developing, a major problem was, that Firefox did not show me errors in the Greasemonkey scripts. Thus either the script would work or not work and fail completely silently. That made debugging a bit painful.

So now, here’s the script.

Tomáš Pospíšek

PS: This script also lives at userscripts.org

to visually sort images under Linux doesn’t seem to be a trivial task. I duckduckgo‘ed for a long time and had a look at various image and file managing applications before finding gthumb. And even there, you first need to create a “catalog” and within the catalog a “library” which will finally allow you to manually sort your images. All of which is not documented.

Once you’ve sorted your images, you’d possibly want to export the sorting? Again, no trace of any help or documentation: gthumb catalogs are saved under $HOME/.local/share/gthumb/catalogs/foobar.catalog.

Tomáš Pospíšek

A while ago I’ve compiled PostGIS 1.5.2 for PostgreSQL 9.0.

To install it on Ubuntu the following steps are required:

apt-get install python-software-properties

add-apt-repository ppa:ubuntugis/ubuntugis-unstable
add-apt-repository ppa:pitti/postgresql
add-apt-repository ppa:pi-deb/gis

apt-get update

apt-get install postgresql-9.0-postgis

A basic template database can be created with the following commands:

sudo su - postgres

createdb template_postgis
psql -q -d template_postgis -f /usr/share/postgresql/9.0/contrib/postgis-1.5/postgis.sql
psql -q -d template_postgis -f /usr/share/postgresql/9.0/contrib/postgis-1.5/spatial_ref_sys.sql
psql -q -d template_postgis -f /usr/share/postgresql/9.0/contrib/postgis_comments.sql
cat <<EOS | psql -d template_postgis
UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template_postgis';
REVOKE ALL ON SCHEMA public FROM public;
GRANT USAGE ON SCHEMA public TO public;
GRANT ALL ON SCHEMA public TO postgres;
GRANT SELECT, UPDATE, INSERT, DELETE
  ON TABLE public.geometry_columns TO PUBLIC;
GRANT SELECT, UPDATE, INSERT, DELETE
  ON TABLE public.spatial_ref_sys TO PUBLIC;
EOS

To test database creation you can do the following:

createdb --template template_postgis test_gis
psql -d test_gis -c "select postgis_lib_version();"

Ubuntu supports parallel installations of different PostgreSQL versions. So if you have already PostgreSQL 8.x installed, PostgreSQL 9.0 will probably be configured to listen on port 5433. So you have to add the option -p 5433 to each command and to specify the full path for the executables. For example:

/usr/lib/postgresql/9.0/bin/psql -p 5433 -d test_gis -c "select postgis_lib_version();"

Recently I had to pre-select a Node inside a TreePanel ExtJS widget. I tried many ways but failed because most of the time when I tried to:

  node.select();

that node would not yet be rendered into the browser's DOM and so the select would fail somewhere inside the extjs.js blob with something like "this ... undefined ...". What I needed was a "rendered" event, but there doesn't seem to be such an event for neither TreePanel not TreeNode or any of their superclasses. Diving into ExtJS code was not really very helpful because it's a framework that does things through layers and layers and as such is not trivial to understand quickly.

Thus, the same ole problem with JavaScript as ever: "show me all the events there are". However, surprisingly, in contrast to standard JavaScript, ExtJS has an easy standard way to accomplish this:

 
  Ext.util.Observable.capture( myTree, function(event) {                                                                                                        
                                         console.log("got an event in myTree");
                                         console.log(event); });

And quickly I discovered that there indeed is an obscure event that I can, out of alternatives, missuse to do what I want, which is expandnode.

Aparently, after a node is exapanded, all its children are put into the DOM and seem to be manipulable then. Thus:

        /*
         * select node with real_id - this is only called once
         * after the tree is rendered for the first time.
         * After that the listener itself is unregistered.
         */
        function select_node(node) {
          node.eachChild( function(child) { 
            if(child.attributes.real_id == real_id ) {
              child.select();
              categories_panel.un('expandnode', select_node);
            }
          });
        };
        myTree.on('expandnode', select_node);

Tomáš Pospíšek

Problem statement:

Which of our strings are not translated yet in our Ruby on Rails app?

Unfortunately there’s no easy way to know for sure. One solution is to log untranslated strings as soon as they appear - the following solution applies to the spree-i18n extension but should be easily adaptable to other contexts.

What we do here is monkey patching I18n#t, to check whether the original I18n#t told us that there’s no translation and log it in that case. Then we return whatever the original I18n#t gave us:

I18n.class_eval do                                                                                    

      class << self                                                                                   
        alias_method :alias_for_t, :t                                                                 
      end                                                                                             

      # make t log missing translations                                                               
      def self.t( keyz, options = {})                                                                 
        translation = self.alias_for_t( keyz, options )                                               
        if translation =~ /translation missing:/                                                      
          MISSING_TRANSLATION.info translation                                                        
        end                                                                                           
        return translation                                                                            
      end                                                                                             
    end                                                                                               

end

To initialize the logger:

class MissingTranslationLogger < Logger
  def format_message(severity, timestamp, progname, msg)
    "#{msg}\n" 
  end 
end

logfile = File.open('log/missing_translations.log', 'a')    
# optional: # logfile.sync = true
MISSING_TRANSLATION = MissingTranslationLogger.new(logfile)

You’ll need to place those two code snipplets in apropriate places. I’ve put the first one in vendor/extensions/site/site_extension.rb and the second one into config/initializers/missing_translation_logger.rb.

Hope somebody’ll find this snippet useful.

Tomáš Pospíšek

Occassionaly we’ve been seeing messages like:

spam acl condition: error reading from spamd socket: Connection reset by peer

in /var/log/exim4/paniclog

But lately the problem has become much more persistent. I found out, that there’s some arsehole spammer sending us 500K spams every ca. 70 minutes. From /var/log/mail.log:

Jul 17 08:52:32 mail spamd[477]: spamd: identified spam (7.9/5.0) for spamd:1004 in 288.5 seconds, 505669 bytes.

Now since spamassassin would take that long (nearly 5 minutes!!!) to find out whether that mail is a spam or not, exim would simply timeout it’s connection to spamassassin and the result was the message seen above.

What I did was to use sa-compile to compile the SA rules to “native code” and to enable “Rule2XSBody”, which on Debian lives in /etc/spamassassin/v320.pre.

This seems to make SA quite a lot faster and to use less CPU.

If this won’t help we could also restrict the maximum size of messages to scan.

Yours Tomáš Pospíšek

Since I found Michel de Boer’s explanation of many of the fundamental SIP concepts in his 2005 dutch presentation of the Twinkle SIP phone very short and easy to follow - contrary to a lot of other material about SIP out there - I’ve translated it to english.

Here it is: Twinkle - a SIP softphone for Linux

Well, sometimes one wonders why people write tools that wheight a megabyte, when it’s easy to stick together one or two tools with a script and have the same thing.

Here’s a ruby script that generates a ping latency graph (target is google):

#!/usr/bin/ruby
#
# based on
#   http://oss.oetiker.ch/rrdtool/prog/rrdruby.en.html example
# with help of the tutorial
#   http://oss.oetiker.ch/rrdtool/tut/rrdtutorial.en.html
# and the docu
#   http://oss.oetiker.ch/rrdtool/doc/rrdcreate.en.html
# 
# placed in the public domain by Tomáš Pospíšek

require "RRD"

$rrd    = `mktemp /tmp/tmp.XXXXXXXX`.chomp
$grafik = `mktemp /tmp/tmp.XXXXXXXX`.chomp
$start  = 0

def now()
  Time.now.to_i
end

def create()
  $start=now()
  RRD.create(
    $rrd,
    "--start", "#{$start - 1}",
    "--step", "#{1}",       # every 1s
    "DS:pinglatency:GAUGE:10:0:200", # 10s of missing data are OK, min=0, max=200 (ping time)
    "RRA:AVERAGE:0.5:1:#{60*30}") # 60 seconds * 30 minutes
  puts "RRD database created #{$rrd} at #{$start}"
  puts "Your graph is at #{$grafik}"
end

def ping_latency
   `ping -c 1 -n -w 1 google.com` =~ /.* time=(\d+)/
  # 64 bytes from 74.125.43.147: icmp_seq=1 ttl=52 time=31.6 ms
  return $1
end

def update()
  puts "updating #{$rrd} with #{now()}:#{ping_latency()}"
  RRD.update($rrd, "#{now()}:#{ping_latency()}")
end

def generate_gfx()
  # puts "generating graph #{$grafik}"
  RRD.graph(
   "#{$grafik}",
    "--title", " Ping latency to Google",
    "--x-grid", "SECOND:10:MINUTE:1:MINUTE:2:0:%X",
    "--start", $start.to_s,
#    "--end", # -> now
    "--interlace",
    "--imgformat", "PNG",
    "--width=450",
    "DEF:pinglatency=#{$rrd}:pinglatency:AVERAGE",
    "AREA:pinglatency#00b6e4:ms")
end

#main
create()
while(true)
  update()
  generate_gfx()
  sleep 1
end

It generates a graph like this:

How do you test web applications? You can use Selenium, but Selenium is very heavy.

I tried wget.

To find out what exactly to send to the app, I started Firebug inside Firefox, went to the “Network” section and recorded what I would do, to see, whether the application works. Then I took the accessed URLs together with the parameters and first tried to send them manually from the command line. Observe, that Firebug lets you copy the exact POST/GET parameter as was sent by the browser, when you right click on the POST/GET command in the Firebug Network log. Once I managed to redo all the steps manuall from the commandline I transferred what I did into a script and edited it until it was nice and would run through:

$ history > test_my_web_app

#!/bin/bash
#
# check availability of:
url="http://www.example.org/my-app" 

# exit immediately if something goes wrong -> test fails!
set -e

echo "Testing $url" 

# save an error log in case there's an error so that
# we can analyse it further
error_log=$( mktemp )
cookies=$(   mktemp )

echo "In case the tests fail, you'll find the error log under $error_log" 
# send everything that goes to STDERR to the log
exec 2>$error_log

echo "Getting a cookie..." 
wget --save-cookies $cookies "$url" -O /dev/null

# we could put username/password directly into the script but we're cautuous and don't
echo -n "please enter user for $url: ";     read    user
echo -n "please enter password for $url: "; read -s pass
echo

echo "Logging in..." 
# if we logged in successfully, then the webapp will send us an html page
# that contains the following string:
to_verify="You're logged in" 
# POST to the webapp:
wget --load-cookies $cookies \
     --post-data "login=Login&user_login=$user&user_password=$pass" \
     --save-cookies $cookies \
     --keep-session-cookies \
     "$url/login" \
     -O - \
| grep -q "$to_verify" 

# now we're logged in

echo "Query component DB..." 
# First search the DB (parameters need to be URI encoded...)
num_components_found=$(
  wget --load-cookies $cookies \
       --save-cookies $cookies \
       --post-data 'action=set_params&crit%5Bcolor%5D=red&crit%5Bmin_size%5D=0' \
       "$url/prepare-picture" \
       -O - \
  | grep 'Number of matching components:' \
  | sed 's/.*Number of matching components:<\/td><td>//; s/<.*//' )

echo "Make sure that the app found something..." 
[ $num_components_found -gt 0 ]

echo "Generate and get picture..." 
wget --load-cookies $cookies \
     --save-cookies $cookies \
     --post-data 'index=0' \
     "$url/get-pic" \
     -O /dev/null

echo "Hooray, $url successfully tested" 

rm $error_log
rm $cookies
<pre>

Surprisingly easy, isn't it?
      
    
    
    
  

Here’s a patch:

# diff -u multi_http_responsetime.orig multi_http_responsetime
--- multi_http_responsetime.orig        2010-02-01 10:13:22.000000000 +0100
+++ multi_http_responsetime     2010-02-01 10:30:23.000000000 +0100
@@ -16,6 +16,9 @@
 use Time::HiRes qw( time );
 use IO::Socket;

+$FAILED=0;
+$INFINITE=999999;
+
 # ----- config -----
 push(@url_array, "http://www.google.de");
 push(@url_array, "http://www.t-online.de");
@@ -34,7 +37,7 @@
                                        PeerPort => 80,
                                        Proto => 'tcp'
                                        );
-       return(0) unless ($sock);
+       return($FAILED) unless ($sock);
        print $sock "GET $baseurl HTTP/1.1\nHost: $vhost\nConnection: close\n\n";
        while (<$sock>) {
                $data .= $_;
@@ -63,11 +66,16 @@
                $baseurl =~ s@^\w+://.+?(/)@\1@;

                $tick1 = time();
-               geturl;
+               $retval = geturl;
                $tick2 = time();

-               $tspent = $tick2-$tick1;
-               $msecs = ($tspent * 1000);
+               if ($retval == $FAILED) {
+                       $msecs = $INFINITE;
+                       }
+               else {
+                       $tspent = $tick2-$tick1;
+                       $msecs = ($tspent * 1000);
+                       }

                printf "timespent$i.value %.3f\n", $msecs;
                }

Since upgrading to Karmic I had the problem under Ubuntu that SIP calls under Twinkle would start get all chopped up every few minutes.

Turns out it’s NetworkManager that scans the wireless spectrum every two minutes for networks even if you are allready connected to some AP.

While people seem to be fixing the problem in the proper way, by making scanning interfere less with an active connection, there’s a good soul that provides patched versions of NetworkManager that gives up scanning while there’s an active connection to an AP.

See https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/373680

In https://bugs.launchpad.net/bugs/509411 I had the problem, that locking the screen would just not work any more. It turned out that gnome-screensaver must be perpetualy runnung in the background in order to be able to lock the screen.

Also this would affect the computer not locking the screen when suspending.

It might be that Gnome/Ubuntu are user-friendly, however the bloat is starting to annoy me. Why do I need to be running 214 processes, when the only thing that is actually being used is firefox and a terminal? Why do I need a process (gnome-screensaver) that is using up CPU cycles and virtual memory, when it’s actually never doing anything useful?

Thus I purged gnome-screensaver (8MB) and installed i3lock.

There are three things missing:

1) Make the laptop lock the screen when suspending:

$ cat /etc/pm/sleep.d/01_screenlock
#!/bin/bash
#
# 01_screenlock: lock screen at suspend with i3lock
# based on http://bbs.archlinux.org/viewtopic.php?id=68158
#

USER=henry
DISPLAY=:0.0
IS_ACTIVE="$( pidof i3lock )"

lock_screen() {
  # skip if i3lock is allready running
  if [ ! "$IS_ACTIVE" ]; then
       su - $USER -c "/bin/sh -c 'DISPLAY=$DISPLAY /usr/bin/i3lock'" &
  fi
}

case $1 in
   hibernate)
       lock_screen
       ;;
   suspend)
       lock_screen
       ;;
   thaw)
       # not required.
       ;;
   resume)
       # not required.
       ;;
   *) exit $NA
       ;;
esac

2) Add it to the panel:

$ cat ~/.gnome2/panel2.d/default/launchers/i3lock.desktop
#!/usr/bin/env xdg-open

[Desktop Entry]
Encoding=UTF-8
Version=1.0
Type=Application
Terminal=false
Icon=/usr/share/icons/gnome/scalable/actions/gnome-lockscreen.svg
Name=Lock the Screen
Exec=i3lock

3) integrate it into package management, so that I don’t accidentaly remove packages on which the screen locking depends:

$ cat control
Section: misc
Priority: optional
Standards-Version: 3.6.2

Package: lock-screen
Maintainer: Tomas Pospisek 
Depends: i3lock, pm-utils, gnome-icon-theme
Files: 01_screenlock /etc/pm/sleep.d/01_screenlock
 i3lock.desktop /home/henry/.gnome2/panel2.d/default/launchers/i3lock.desktop
Description: lock screen on suspend/hybernate
 bla
$ equivs-build control
$ sudo dpkg -i lock-screen_1.0_all.deb

Of course, it’s a hack. However it works and is easy to adapt. Let me know at tpo at sourcepole.ch in case you get to make it more generic.

That’s it. Saved runtime bloat. Saved disk space (i3lock is 60K large).