Digital Ambulation

EdgeRouter + Dynamic DNS + Cloudflare

2016-12-30T00:00:00+00:00

Note

This no longer works. The version of ddclient in EdgeOS uses the CloudFlare v1 API, which was deprecated November 2016 and finally retired June 2018. I now use cloudflare-ddns running on a server inside my network.

My home network sits on a fast internet connection behind an Ubiquiti Networks EdgeRouter X but doesn't have the benefit of a static IP address. I've been meaning to set up a dynamic DNS entry for it for a while, but really wanted to make use of the fact I already manage my DNS with Cloudflare, which has a very capable API. Ubiquiti's EdgeOS has support for dynamic DNS by generating ddclient configurations and running them when the IP address changes, however the web GUI doesn't support everything that is possible with its ddclient version.

There are existing solutions to this problem, for example this support article easily found through a search engine. However, the solutions I've found are clunky hacks that will be missed by configuration backups, and therefore more confusing to reproduce in the future if necessary. I wanted something that would exist properly within the EdgeOS configuration system.

Through trial and error, I found there are only two small things preventing Cloudflare support from working, and both can be easily sidestepped.

The web GUI doesn't provide the cloudflare protocol in the drop-down. Using the EdgeOS configuration CLI allows this value to be specified manually.
The EdgeOS configuration schema doesn't allow for the zone setting required by the cloudflare protocol. Because ddclient configuration is basically a comma-separated list of option=value pairs, the setting can be injected as part of another setting, as long as the schema validation doesn't prevent it. The host-name setting currently allows it, e.g. home.example.com becomes zone=example.com,home.example.com.

Armed with this knowledge, the dynamic DNS client can be configured with the following commands:

configure
set service dns dynamic interface eth0 service custom-cloudflare login {{EMAIL}}
set service dns dynamic interface eth0 service custom-cloudflare password {{API_KEY}}
set service dns dynamic interface eth0 service custom-cloudflare protocol cloudflare
set service dns dynamic interface eth0 service custom-cloudflare server www.cloudflare.com
set service dns dynamic interface eth0 service custom-cloudflare host-name zone={{DNS_ZONE}},{{HOSTNAME}}
commit
save
exit

eth0: On my EdgeRouter X, eth0 is configured as the WAN link. Replace as appropriate on other setups, this should be the interface that will have your external public IP address.
{{EMAIL}}: The email address you use to login to Cloudflare.
{{API_KEY}}: Your Cloudflare "Global API Key", which you can get at https://www.cloudflare.com/a/account/my-account.
{{DNS_ZONE}}: The Cloudflare-managed domain that the desired DNS entry exists under, e.g. example.com.
{{HOSTNAME}}: The DNS entry that should be modified, e.g. home.example.com.

Note

This article is based on my experience with an EdgeRouter X running EdgeOS 1.9.0. The instructions may work with older versions of EdgeOS and should work on other EdgeRouter models, but I haven't tested those other possibilities.

(X)ubuntu Linux 14.04 on Asus N550JK laptop

2015-07-24T00:00:00+01:00

I've just acquired a new work laptop (Asus N550JK-CM604H) and for my job the first step was getting my preferred Linux distro (Xubuntu 14.04 LTS) up and running on it. However, the laptop is newer than the distro release, so I didn't expect it to all be smooth sailing...

The hardware

CPU

Intel i7-4720HQ: 2.6 GHz (3.6 GHz turbo), 4 cores, 8 threads, 6MB cache

RAM

2x4GB DDR3 1600 MHz (replaced with 2x8GB)

GPU

NVIDIA Optimus dual GPU setup:

Intel HD 4600, by default using 64MB of system memory as VRAM.
NVIDIA GeForce 850M, 2GB dedicated VRAM

HDMI and mini-DisplayPort external outputs. By default everything goes through Intel GPU; see below for NVIDIA Optimus support.

Display

15.6" LED-backlit 1080p IPS

Hard drive

Seagate Momentus ST1000LM024: 1TB SATA-II 5400rpm (replaced with Samsung 850 EVO 500GB)

Wired network

Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller; supported by r8169 driver without intervention

Wireless network

Intel 7260; supported by iwlwifi driver without intervention

Touchpad

FocalTech touchpad; see below

Touchscreen

Atmel maXTouch Digitizer; using evdev without intervention

Keyboard

It's a keyboard; see below for details of function keys

Webcam

Generic USB video class (UVC) 720p webcam; works without intervention (tested in camorama)

Audio/microphone

Intel High-Definition Audio; using snd_hda_intel without intervention

LTS Hardware Enablement Stack (HWE)

When using an older Linux distro release on a brand new machine, it's likely that some things won't be very well supported by whichever kernel and Xorg versions that release froze on. Ubuntu's answer to the problem is the LTS Hardware Enablement Stack. At the time of writing, the stack based on 15.04 (Vivid) was the latest, so the following command was used:

sudo apt-get install --install-recommends \
    linux-generic-lts-vivid xserver-xorg-core-lts-vivid xserver-xorg-lts-vivid \
    xserver-xorg-video-all-lts-vivid xserver-xorg-input-all-lts-vivid \
    libwayland-egl1-mesa-lts-vivid

The only problem with this is that some packages might not be able to build drivers against the new kernel. I found this to be the case for VirtualBox, but using the latest version available from the VirtualBox site gets around this.

Suspend

I decided to try out suspend and resume early on, since historically it's been a fairly quirky area of hardware support. In this case, the nouveau driver barfed into the logs and prevented sleep. Blacklisting it seems to fix the problem: add blacklist nouveau to /etc/modprobe.d/blacklist-nouveau.conf. This was no big deal, since I'm either using the Intel GPU or the proprietary NVIDIA drivers.

Graphics driver

Like a lot of decent modern laptops, this one has a dual-GPU setup: the Intel HD 4600 as part of the CPU, and an NVIDIA 850M.

As is to be expected, the Intel GPU just works. The HWE upgrade enabled DisplayPort 1.2 Multi-Stream Transport (MST) support, which allowed me to daisy-chain my Dell U2515H monitors which support this feature. (Note: the monitors ship with this feature disabled, you need to enable it in the OSD on each monitor.)

The NVIDIA GPU was less straightforward: at the time of installing, the latest NVIDIA drivers in the Ubuntu 14.04 repos would not build against the 3.19 kernel from 15.04. Luckily, the backported newer drivers were added within a couple of weeks.

There are two ways to enable NVIDIA support on a laptop like this: using "Bumblebee" to allow per-application use of the NVIDIA GPU, or using "NVIDIA Prime" to switch the entire desktop environment between GPUs (requires logging out and back in again). Because I don't intend to make much use of the NVIDIA GPU, I opted for the latter:

sudo apt-get install nvidia-346-updates nvidia-prime

By default the Intel GPU should remain selected. The GPU can be switched from the nvidia-settings control panel.

Touchpad driver

The laptop has a FocalTech touchpad, which I hadn't heard of before, and which isn't handled properly even in the 3.19 kernel from Ubuntu 15.04. Luckily, these touchpads seem to be used in multiple Asus laptops now, and Ask Ubuntu had an answer: Asus X750JA and Ubuntu Gnome 14.04. Following those instructions and rebooting, the touchpad had gained two-finger vertical scrolling, edge scrolling, and also the ability to use synclient to tweak the settings.

In terms of customising the touchpad behaviour, I put the following in my ~/.xprofile:

synclient HorizTwoFingerScroll=1 \
          VertEdgeScroll=0 \
          TapButton3=2 \
          PalmDetect=1

Enable 2-finger scrolling horizontally (vertical 2-finger scroll is enabled by default)
Disable vertical edge scrolling, since I'm just going to be using 2-finger scrolling all the time
Turn 3-finger taps into a "middle button" click (2-finger taps are "right click" by default)
Enable palm detection, to try and get rid of spurious inputs when typing

Keyboard

It's a keyboard. It generally works as well as you'd expect for one of the oldest standard bits of computer hardware. What tends to vary these days is of course the special/"fn" keys on the keyboard. This one has several. Going left-to-right, top-to-bottom:

Sleep key: Is recognised by power management functionality: if you have it set to suspend when the sleep button is pressed, pressing this key will suspend the same as via any other route.
Wireless toggle: Not registered at all; see below for fix.
Decrease/increase display brightness: Not registered at all; see below for fix.
Toggle display: Seems to switch the laptop screen off and on, not sure where this is handled.
Select display mode: This key is horrible. Really really horrible. So horrible it needs its own subsection (see below). Needless to say, it doesn't work properly.
Toggle touchpad: Recognised as XF86TouchpadToggle, but nothing actually uses it.
Mute/increase/decrease volume: Recognised and behave as expected.
Box with an "S" in it?: Recognised as XF86Launch1.
Camera: Recognised as XF86WebCam.
Running person?: Recognised as XF86Launch6.
Stop/prev/play-pause/next: Recognised as XF86AudioStop, XF86AudioPrev, XF86AudioPlay and XF86AudioNext.
Calculator: Launches calculator.

Wireless toggle and display brightness

Out of the box, the display brightness and wireless toggle keys are completely unrecognised and generate no events. It seems that this is a bit of ACPI quirkiness that is documented here: Fixing Display Backlight Hotkeys on ASUS N550JK. The TL;DR is:

add acpi_osi= to your GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub
run sudo update-grub
reboot

That horrible "Select display mode" key

In Windows, there is a Win + P combination that opens a quick display selector (for handling an external monitor). This key seems to try and emulate pressing this key combination. When you press Fn + F8 to use this function, it seems to emit the following keycode events:

133 down
33 down
33 up
133 up

Keycode 133 is known as Super_L in Linux, or the left Windows key in common parlance. Keycode 33 is the P key. But those are keycode events, which means if you use an alternative keyboard layout (e.g. Dvorak) you get a different effect (e.g. Win+L). I give up on doing anything useful with this key.

Final words

The laptop makes a pretty adequate desktop replacement, and all of the hardware should be well-supported by an up-to-date Linux distro. However, for those that opt for a long-term stable releases, I hope documenting the quirks I encountered will help somebody in a similar situation.

Odd-numbered Days

2013-01-25T00:00:00+00:00

Whilst chatting on the #minecraft channel on freenode, somebody comes out with:

<Dinnerbone> I love that 50% of thursdays are odd-numbered days
<Dinnerbone> Makes the near-end-of-week that much more interesting

Which got me thinking... Are 50% of Thursdays on odd-numbered days? Probably not, at least not in every year. Some hacky Python code ensues...

#!/usr/bin/env python3
import sys
import datetime
from collections import defaultdict

def generate_year(year):
    delta = datetime.timedelta(days=1)
    date = datetime.date(year, 1, 1)

    while date.year == year:
        yield date
        date = date + delta

day_numbers = defaultdict(list)

for date in generate_year(int(sys.argv[1])):
    day_numbers[date.isoweekday()].append(int(date.strftime('%m')))

day_odd_prob = {}
for day, nums in day_numbers.items():
    odd = 0
    even = 0
    for n in nums:
        if n % 2 == 0:
            even += 1
        else:
            odd += 1
    day_odd_prob[day] = odd / (odd + even)

DAYS = ('Mon', 'Tues', 'Wed', 'Thurs', 'Fri', 'Sat', 'Sun')

for day in sorted(day_odd_prob.keys()):
    print('{}\t{}'.format(DAYS[day-1], day_odd_prob[day]))

Now if for some bizarre reason you really want to know what proportion of Thursdays (or any other day) fall on odd-numbered days in any given year, you can find out:

$ python3 odd_days.py 2012
Mon     0.49056603773584906
Tues    0.5192307692307693
Wed     0.4807692307692308
Thurs   0.5192307692307693
Fri     0.5
Sat     0.5
Sun     0.5094339622641509

$ python3 odd_days.py 2013
Mon     0.5
Tues    0.49056603773584906
Wed     0.5192307692307693
Thurs   0.5
Fri     0.5192307692307693
Sat     0.5
Sun     0.5

Optional Positional Arguments with argparse

2012-05-29T00:00:00+01:00

Today I helped somebody in the Python IRC channel with a question about combining required, optional and multi-value positional arguments in Python's argparse module. This is actually pretty easy to do, but not immediately obvious, so here's the example I put together. I took hints from the first example in the argparse documentation, and also this StackOverflow answer.

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('foo', default='a')
parser.add_argument('bar', default='b', nargs='?')
parser.add_argument('baz', default=[], nargs='*')
print parser.parse_args()

And does it work? Apparently so. Some example output from running this script:

$ python test.py
usage: test.py [-h] foo [bar] [baz [baz ...]]
test.py: error: too few arguments

$ python test.py x
Namespace(bar='b', baz=[], foo='x')

$ python test.py x y
Namespace(bar='y', baz=[], foo='x')

$ python test.py x y z
Namespace(bar='y', baz=['z'], foo='x')

$ python test.py x y z a
Namespace(bar='y', baz=['z', 'a'], foo='x')

Using Nautilus in Xubuntu 11.10

2012-01-09T00:00:00+00:00

Like many people, I've found the new things in the Linux desktop environment world - i.e. GNOME 3 and Unity - to be more of a hindrance than a help. For this reason I've switched to Xubuntu, which allows me to stick to my previous multi-window multi-desktop multi-monitor workflow.

People making this transition might not find themselves too comfortable with XFCE's file manager, Thunar. Switching the default file manager to Nautilus (GNOME's file manager) is actually pretty simple, however there are a few issues with correctly handling the desktop background and icons. The setup described here will switch the default file manager to Nautilus, but leave XFCE and Thunar in control of the desktop. This is because Nautilus, if allowed to control the desktop icons, will also try and set the desktop background. Complete control of its background settings is only possible by manually editing dconf settings, or using gnome-control-center which depends on the whole of GNOME being installed. Of course if you already have desktop icons disabled you won't notice any change in desktop handling at all.

So on with the show. Obviously, you first need to install Nautilus:

sudo apt-get install nautilus

Next, you need to tell XFCE to use Nautilus as the default file manager. This can be done through: XFCE menu -> Settings -> Settings Manager -> Preferred Applications -> "Utilities" tab -> "File Manager" drop-down.

Now any time you open a folder you should get Nautilus instead of Thunar. Unfortunately the first time you do this Nautilus will realise it isn't managing the desktop and will try to do so. This behaviour can be disabled by setting the appropriate dconf entry:

sudo apt-get install dconf-tools
dconf write /org/gnome/desktop/background/show-desktop-icons false

If you have desktop icons enabled in XFCE what you have is an instance of Thunar displaying the contents of ~/Desktop, which means any directory you double-click on will also launch in Thunar. This probably isn't what we want, but unfortunately we can't change this default behaviour. What we can do is make it easier to launch the directory in Nautilus by adding a "custom action" - the end result will be a "Open in Nautilus" item in the context menu when you right click directories on the desktop.

Launch Thunar and open the "Custom Actions" dialog (Edit -> Configure custom actions...). Add a new action, setting the command to nautilus %F and making sure only "Directories" is checked on the "Appearance Conditions" tab. Also check "Startup notification" and set the icon to be the correct one for Nautilus.

Of course the nicest result would have been to have Nautilus managing the desktop instead. At first glance this looks like it should be possible, since there is a /org/gnome/desktop/background/draw-background setting in dconf, however changing this setting has no effect on Nautilus' behaviour.

Spoofing MAC address on Ubuntu Maverick

2010-10-12T00:00:00+01:00

Continuing the trend of accidentally disabling hacks that people rely on, it now no longer appears to be possible to set your network card's MAC by adding a line like the following to /etc/rc.local:

ifconfig eth0 hw ether 00:0C:xx:xx:xx:xx

It seems that for some reason something (presumably NetworkManager) is resetting the device's MAC address when the computer wakes from suspend. However, at the same time, the "Cloned MAC address" field has been added to NetworkManager's "Edit connection" dialog, and this is where your desired MAC address should be entered.

Batch Re-tabbing Files with Vim

2010-05-23T00:00:00+01:00

So you have your favourite tabbing convention, but you've ended up with some files in some other convention, for example tabs instead of 4 spaces. If you use Vim and already have it set up to your liking, the :retab command will replace indents in the current buffer with those matching your convention. It's a bit tedious to do this manually for each file, so if you trust Vim not to make a mistake (or you're using version control) you can use a bit of scripting-fu to make it easier:

for F in *.{c,h}pp ; do vim -c ":retab" -c ":wq" "$F" ; done

Remapping Mouse Buttons on Ubuntu Lucid

2010-04-23T00:00:00+01:00

For ergonomic reasons (discussed previously) I like a side button on my mouse to act as a middle button instead. Unfortunately there still doesn't seem to be an "easy" way to remap mouse buttons in Linux, and what's more the method of changing such settings seems to keep changing. To achieve the same results in Ubuntu Lucid, I added an Xorg configuration fragment at /usr/lib/X11/xorg.conf.d/20-logitech-mx1100.conf:

Section "InputClass"
        Identifier "Logitech MX1100 button remap"
        MatchProduct "Logitech USB Receiver"
        MatchDevicePath "/dev/input/event*"
        Option "ButtonMapping" "1 2 3 4 5 6 7 2 9 10"
EndSection

As with the previous method, xinput list should give you the product string to use for MatchProduct.

Note

In Ubuntu 10.10 /usr/lib/X11/xorg.conf.d/ became /usr/share/X11/xorg.conf.d/. Since at least 11.04 /etc/X11/xorg.conf.d/ is read in the same way, if it exists, and preferably this configuration snippet should live there.

Fixing ugly Qt fonts in GNOME

2010-04-17T00:00:00+01:00

So far every time I've used Qt applications under GNOME, especially VirtualBox, I've found the Qt font rendering to be appalling. See this screenshot of VirtualBox alongside the GNOME appearance dialog:

As you can see, most of my font settings are at the default "Sans" font. The "Sans" and "Sans serif" fonts, at least in recent Ubuntu releases, refer to the corresponding DejaVu font variants (previously it was Bitstream Vera). However as I found out through experimentation, whilst Qt is obeying my font settings, the generic "Sans" font seems to mean something completely different (at least to Qt4). If I instead specify the DejaVu fonts directly in my font settings, the correct font rendering is used:

So that's it: if you're using GNOME and you seem to get a Qt font rendering that doesn't match your GTK font rendering, try using a more specific font. (I haven't got a clue what "Sans" actually means to GNOME, but it doesn't appear to be a font in it's own right.)

When Dynamic Typing Goes Wrong

2009-07-08T00:00:00+01:00

Yesterday I found out first-hand how using a dynamically typed language can get you into trouble in unexpected ways whilst writing unit tests for CSF (my PHP framework).

Take a look at this code---what do you think it should do?

$foo = 'bar';
var_dump(isset($foo['xyz']));
var_dump($foo['xyz']);

Most people would assume that indexing a string on another string is nonsensical, and they would be right---in PHP you can only index a string with an integer. The logical extension would be that isset($foo['xyz']) would be false and/or any attempt to index using a string would raise an error. And that is where they would be wrong. In fact, isset($foo['xyz']) is true, and $foo['xyz'] is "b".

PHP is weakly typed, which generally means there is a lot of implicit casting between types going on. In this case, while indexing a string can only be done with an integer, there is an implicit cast from a string to an integer, so this cast is performed and the result used as the index. However, the usual way of converting from a string to an integer is to use digits (0--9) up until the first non-digit character (after taking the minus sign into account if there is one). This means the integer value of 'xyz' is 0, since there are no digits to add up.

When explained, this may seem like sensible behaviour, but let's think about the effect this has when coding and debugging: if you accidentally index a string with another string somewhere, you won't know about it straight away. Instead, you'll just be getting "wrong" values and scratching your head over where they could be coming from. I'm of the opinion that if you can only index a string on an integer, then indexing it on a string should be at the very least a warning. Silently doing something completely different isn't very helpful.

And so that this doesn't seem like a nonsensical contrived example, here's a reduced version of the code that raised the issue:

function get_item($array, $path, $default)
{
    $item = $array;
    foreach (explode('.', $path) as $p)
    {
        if (!isset($item[$p]))
        {
            if (func_num_args() < 3)
                throw new ItemNotFound($path);
            else
                return $default;
        }
        else
        {
            $item = $item[$p];
        }
    }
    return $item;
}

What this function does is traverse a multi-dimensional array with a "key path", i.e. "key1.key2" gets $array['key1']['key2']. Since isset() is the most common way people check the existence of an array, this is what I used here. However, things break down in this example:

$array = array('foo' => 'bar');
$x = get_item($array, 'foo.xyz');

The expected behaviour is that the exception should be thrown, since $array['foo'] is not an array with a "xyz" key, but the issue described above instead leads this to return "b".

In this case, there is a way around it: what I really should have been using is the following check:

if (is_array($item) && array_key_exists($p, $item))

This has more interesting consequences---it means that a very very large number of people are using isset() where they mean is_array() && array_key_exists(). As for me, I just fixed a bug in my code exposed by a unit test, so I'm happy (even if a little bewildered by PHP).

Why .NET Won't Beat Java (Yet)

2009-06-23T00:00:00+01:00

It's been no real secret that the .NET CLR (Common Language Runtime) has been Microsoft's answer to Java. Garbage collection, bytecode compilation, large set of core libraries, it's all there. But there is a problem that I've encountered recently: distribution size and install base.

A fairly clean Windows XP machine is fairly certain to not have anything higher than .NET 1.x installed. Anything really compelling in the .NET framework requires 3.5. This means somehow you need to get version 3.5 onto the machine somehow. This isn't a problem in Windows Vista and later, which include the framework. Microsoft's answer to simple deployment is it's "ClickOnce" system, where an application automatically installs .NET from the Internet (if necessary) before installing itself. Sure, it's a 60MB download, but it only needs to be done once.

The real issue is when you can't guarantee an Internet connection or a working .NET 3.5 installation. At this point you must resort to the offline installation, and this is where .NET and Java are very different. For Java 1.6 SE, the offline installer is 16MB for Windows 32-bit. For .NET 3.5, the offline installer is a 200MB universal package, with no way to cut out the parts you don't need. Java in fact is so small relatively speaking that many applications actually include the JRE in their package (for example OpenOffice - 148MB with JRE vs. 134MB without), whereas .NET can turn a 10MB application into a 210MB monstrosity.

Now in my particular situation, I'm embarassed to have chosen to use C#/.NET/WPF for a simple tool at work. For programming the tool itself, it was certainly the fastest option - other kinds of Windows programming, e.g. MFC or Forms, just look painful, and I thought the barrier to entry would be lower than Java. However this 1MB tool requires the 200MB .NET offline installer to be carted around with it because the network it's used on is completely separate from the Internet.

.NET will only be really appealing once it's ubiquitous, but then "critical mass" is one of the big problems for lots of software. For now, I think I'm going to try Java next time...

Mouse Button Remapping with HAL

2009-06-13T00:00:00+01:00

I've had a Logitech MX1000 mouse for a few years now, and the two most important features for me have been the ergonomic build and the few extra buttons. Something I've always found with many-buttoned mice is that the side button closest to the thumb is a much more ergonomic way to "middle click" than the actual middle mouse button---it's a much more natural motion. Middle clicks are quite useful these days, especially with them being a standard way of closing tabs (and opening them in browsers), and having such a popular button perched on a rocking and rolling peak is far from ideal.

Since I'm primarily a Linux user, I don't have Logitech's own SetPoint software at my disposal, so I've always had to find a way to get this functionality in some other way. When I first got the mouse, this method involved deliberately using a "basic" mouse driver (referred to in xorg.conf as "IMPS/2"), which didn't support many mouse buttons. The effect was that the button mappings wrapped around, leaving button 8, my preferred "middle click", mapped to button 2 (8 mod 3), the real middle button.

Unfortunately, newer Xorg versions became smarter and better capable of handling more buttons, and this workaround ceased to function. For the next while, I used something even more hackish: xbindkeys combined with xmacroplay to simulate a middle click with the following part of my .xbindkeysrc:

"echo ButtonRelease 8 ButtonPress 2 ButtonRelease 2 | xmacroplay -d 0 :0.0 &"
    b:8

The downside to this solution is that there are some cases where the button events don't work correctly, one of them being open-in-tab from a bookmark menu in Firefox. It seemed the best solution would be to get Xorg to remap the buttons in such a way that button 8 really was just an extra button 2. The "xinput" utility lets you set button maps in this way---this wiki entry shows how to remap mouse buttons (even if for a different purpose).

This method worked fine, and I put it in my startup programs for GNOME, but it didn't persist after suspend/resume. It appears that when resuming, USB devices get "reattached", and therefore don't keep the settings applied to them the last time they were attached. The workaround for this is to set a policy using a HAL (Hardware Abstraction Layer) .fdi file. These files live in /etc/hal/fdi/policy (at least they do on Ubuntu) and allow you to set various properties on input devices. This page on the Ubuntu wiki gave me the recipe I needed to remap buttons based on the device name. I ended up with the following .fdi file (which I saved at /etc/hal/fdi/policy/logitech-mx1000.fdi):

<deviceinfo version="0.2">

<!-- Remap Logitech MX1000 buttons so that the most accessible side button
     acts as a middle button -->

  <device>
    <match key="info.product" string="Logitech USB RECEIVER">
      <merge key="input.x11_options.ButtonMapping" type="string">1 2 3 4 5 6 7 2</merge>
    </match>
  </device>

</deviceinfo>

Now, whenever my Logitech mouse is connected, it gets the buttons remapped---this includes when resuming from suspend. Problem solved... until things are changed again of course!

The Rise and Fail of Facebook

2009-06-10T00:00:00+01:00

A recent Lifehacker story about a Greasemonkey script to remove quiz stuff from Facebook made me realise something: you know your site is too "web 2.0" when users are going out of their way to remove features. A quick search on Userscripts.org reveals that a substantial proportion of the Facebook-related scripts are either for removing features or auto-playing game applications.

I feel that Facebook started off pretty well. The initial target audience seemed to be students, and that's what most of the users were. It was friendly and uncluttered, compared to other social networking sites like MySpace. This spread to a wider audience, first sucking in kids and then adults. This was less than ideal for some, the idea of their parents on a social networking site being horrifying, but still not a disaster.

But in the midst of all this, the apocalypse happened: a powerful API which allowed other people to extend the functionality of Facebook by creating applications that people could add to their account. Gradually users became inundated with an unstoppable torrent of application requests, caused by application developers making the "tell all your friends" step seem (or be) necessary to using their application. The average user's profile became a mess of applications competing for screen space. The target audience became those who have the time to play endless Flash games, answer endless quizzes, and generally clutter the "requests" page of their friends.

And so Facebook became MySpace 2.0---gaudy and cluttered, with infinite ability to add more clutter. Maybe the lesson learned is that the moment you allow anything "shiny" in a social networking site, your demographic will degenerate to schoolkids who want to fill their profiles with as much of it as possible.

Personally, I've reduced my Facebook interaction for the past 2 years to having the site email me when anybody says something directly to me either via message or on my "wall". There is just too much clutter now for me to wade through the endless application requests, friend requests from people I don't know, and news feed items generated by the latest viral quiz. And maybe pictures of friends getting drunk have lost their novelty value...

Spectacular IE7 CSS box model failure

2009-05-17T00:00:00+01:00

When re-theming DokuWiki to fit in with the general look-and-feel of my blog, I thought it would look good to have buttons with images relevant to what they did. Having created the necessary, CSS, I subjected it to my multi-browser test bed (actually a Windows XP virtual machine with the latest versions of the top 5 browsers running in it). The CSS I used was along the lines of this:

div.dokuwiki form.btn_show .button {
    padding: 1px 0 1px 16px;
    background: transparent url(images/page.png) 0px 1px no-repeat;
    border: none;
}

One thing that became apparent is that IE7 has absolutely no idea how to render this. The correct rendering (as produced by Firefox 3 and all the other browsers) is this:

IE7's take on rendering this looked more like this:

After a bit more experimentation, I came up with the following illustration which roughly shows what is happening:

The nice thing is that this is fixed in the upcoming IE8, so I've taken the stress-free approach; since it's a relatively "minor" visual bug, there's no sense it tearing my hair out trying to make it render correctly everywhere. On this occasion I've decided that people can either deal with the fact their browser is behind the times, or upgrade, or wait for the upgrade to happen for them.

Quick and dirty screen scraping of UK MP expenses data

2009-05-13T00:00:00+01:00

A dominating story in the news recently has been that of UK Members of Parliament "abusing" the expenses system. As part of this, expense claims data has been released to the public, but unsurprisingly not in a simple CSV format that just anybody can play with. All I could find were PDF files and news sites representing the data in their own way.

This led me to wonder how easy it would be to "scrape" the data from one of these sites. Having heard about BeautifulSoup, a Python HTML "tag soup" parser, I opened up an interactive Python session and started to play with the data from this BBC News page. Luckily for me, the BBC page's HTML isn't too ugly, so figuring out how to get the data rows wasn't that hard.

The end result is the following Python script which scrapes the data from the BBC News page and saves it in both CSV and JSON formats

Mozilla/1.22 (compatible; MSIE 2.0; Windows 95)

2009-05-11T00:00:00+01:00

Browsing the visitor stats today, I found this little gem:

"Mozilla/1.22 (compatible; MSIE 2.0; Windows 95)"? Who uses such a browser? My bet is on it being a zombie machine that somebody has completely forgotten about, which would explain the several other hits I have from versions of IE before 5.0... These visitors turn up, hit the same page repeatedly, and then leave. Unfortunately I deleted my spam before seeing this, otherwise I'd be able to verify this theory. Maybe I'll look more closely next time.

Tpl - A tiny PHP template engine

2009-04-30T00:00:00+01:00

While developing CodeScape Framework I came up with a simple template engine that instead of implementing it's own language just provided some structure-related hooks, and I've now got around to separating out and releasing Tpl, a simple, self-contained, hierarchical template engine for PHP.

Tpl's role is centred around template structure, not content---it's not a huge library of helper functions for creating links, displaying images, etc. Like any template engine, the aim is to separate presentation logic from application logic, but it doesn't attempt this by creating a template language. PHP is designed to be interleaved with "static" output, which means it's already a template language. Trying to replace it for the purpose of templating rarely adds anything useful, incurs hefty performance overheads (which you then have to mitigate with caching) and usually is more restrictive than PHP. To me, that doesn't seem like a sensible application of effort. Instead Tpl provides its functionality in the form of hooks---functions which are called from within the template to specify the structure. Instead of PHP being replaced, it is extended.

The main motivating factor for creating Tpl is that all PHP template engines seem to be centred around inclusion rather than inheritance, something I only noticed after spending time working with Django. For example, a typical PHP template might look like this:

<!-- header.php -->
<html>
    <head><title><?=$title?></title></head>
    <body>
        <div id="header">Website Name</div>

<!-- footer.php -->
        <div id="footer">Some footer text</div>
    </body>
</html>

<!-- content.php -->
<? include('header.php'); ?>

<div id="content">
    Some content
</div>

<? include('footer.php'); ?>

The main problem with this is that it bears no real relation to the logical structure of the page, just the order in which stuff should appear in the HTML. If you modify something in the header, you need to make sure you've updated the footer too. The equivalent Django template would look like this:

{# base.html #}
<html>
    <head><title>{{ title }}</title></head>
    <body>
        <div id="header">Website Name</div>

        {% block content %}{% endblock %}

        <div id="footer">Some footer text</div>
    </body>
</html>

{# content.html #}
{% extends "base.html" %}

{% block content %}
<div id="content">
    Some content
</div>
{% endblock %}

The point to note is that the structure is coherent and in one place, and the parts that change are defined as logical blocks which can be replaced. This is the method I've adopted in Tpl---the equivalent template would look like this:

<!-- base.php -->
<html>
    <head><title><?=$C['title']?></title></head>
    <body>
        <div id="header">Website Name</div>

        <? Tpl::block('content'); ?><? Tpl::endblock(); ?>

        <div id="footer">Some footer text</div>
    </body>
</html>

<!-- content.php -->
<? Tpl::inherit('base.php'); ?>

<? Tpl::block('content'); ?>
<div id="content">
    Some content
</div>
<? Tpl::endblock(); ?>

More information about Tpl and how to download it can be found at the Tpl project page.

Enable Bitmap Fonts on Ubuntu Jaunty

2009-04-11T00:00:00+01:00

I like to use tiny bitmap fonts like MonteCarlo for programming, but by default Ubuntu has bitmap font support turned off. From (at least) Gutsy through to Intrepid, this method worked for enabling bitmap font support, but after installing the Jaunty beta I found this no longer works.

Luckily, after a brief look in /etc/fonts, I found that font configuration now follows the nice pattern of a conf.avail directory containing all the available configuration parts, and conf.d containing symlinks to these parts. This makes enabling bitmap fonts even simpler now:

# "Un-disable" bitmap fonts
sudo rm /etc/fonts/conf.d/70-no-bitmaps.conf
# Clear the font cache
sudo fc-cache -f -v

Now you should be able to drop bitmap (i.e. PCF) fonts into ~/.fonts as you would with TTF fonts and be able to use them with no extra hassle.

Update: Andreas asked in the comments if there was a way to do this that doesn't require root access. In fact there is! If you copy /etc/fonts/conf.avail/70-force-bitmaps.conf to ~/.fonts.conf then it has the same effect. (If you already have a ~/.fonts.conf then you'll want to merge them together; copy everything inside <fontconfig>...</fontconfig> from 70-force-bitmaps.conf and put it just before </fontconfig> in ~/.fonts.conf.)

VDB 0.1.0 (alpha) Released

2009-04-01T00:00:00+01:00

Recently I've been working on a program that will let me do a full-text search of my 1500-file digital video library. The result is something I've unimaginatively called "VDB", short for "Video Database".

This release doesn't do much; you can add your library of video files, and it will let you do a full-text search on the filename. You can double-click an item in the list to launch the video with gnome-open (which should honour user settings for preferred applications).

Feel free to download it, mess around with it, and send me any feedback you have. I have plenty of future plans for it, including "watch folders", tagging, metadata, fetching information from IMDB, and being able to search on all of this information. (Think queries like actor:'johnny depp' tag:crime.)

Visit the VDB project page.

C is not MATLAB

2009-03-30T00:00:00+01:00

Refactoring some code at work today, I came across a classic example of somebody attempting to use one language like another. The project in question is a port from MATLAB to C, and the purpose of the function is to see if a square matrix is symmetrical within a certain tolerance.

The implementation I replaced worked somewhat like this:

int check_symmetry(double *mtx, int n, double tolerance)
{
    /* Allocate memory to work in */
    double *mtx_trans = (double *) malloc(n * n * sizeof(double));
    double *mtx_diff = (double *) malloc(n * n * sizeof(double));

    /* Matrix transpose */
    matxtrans(mtx, n, n, mtx_trans);
    /* Matrix subtract */
    matxsub(mtx, n, n, mtx_trans, n, n, mtx_sub);

    /* Assume matrix is symmetric to start with */
    int symmetric = TRUE;

    /* Iterate through rows */
    for (int i = 0; i < n; ++i)
    {
        /* Iterate through columns */
        for (int j = 0; j < n; ++j)
        {
            /* Was the difference greater than the tolerance? */
            double val = mtx_sub[i * n + j];
            if (fabs(val) > tolerance)
            {
                /* Exception found: matrix not symmetric
                 * (but notice that we keep looping...) */
                symmetric = FALSE;
            }
        }
    }

    /* Clean up memory allocations */
    free(mtx_trans);
    free(mtx_diff);

    /* ... and return the result */
    return symmetric;
}

Now, what this is really doing is the equivalent to the following line of MATLAB:

symmetric = max(abs(mtx - mtx')) > tol;

This in itself highlights the massive differences between MATLAB, where matrix operations are fundamental, and C, where they are not. While the above C code gets the correct answer, it's far from efficient. What should be obvious is that the programmer translated the code, not the meaning.

To analyse this simple case: a symmetric matrix is reflected in the diagonal---the lower triangle is the same as the upper. What we really are trying to find out is if any elements violate this constraint. More verbosely: "does any element in the upper triangle differ from it's corresponding element in the lower triangle (by more than a certain value)?" Once we've figured this out, the operation in C is much, much shorter:

int check_symmetry(double *mtx, int n, double tolerance)
{
    /* Iterate through upper triangle only */
    for (int i = 0; i < n; ++i)
        for (int j = i + 1; j < n; ++j)
            /* Compare to lower triangle */
            if (fabs(mtx[i * n + j] - m[j * n + i]) > tolerance)
                /* Exception found: matrix not symmetric */
                return FALSE;
    /* No exceptions found: matrix is symmetric */
    return TRUE;
}

This achieves the minimum number of iterations and element accesses by ignoring the diagonal, only iterating through the upper triangle and reversing the indices to access the lower triangle. The only potential downside is that it's not as obvious what's going on, but that's what comments are for!

The lesson is that you should always make sure you're using the correct method for the tool, and not trying to hammer a screw in with a spanner.

Persistent Remote irssi Session

2009-03-28T00:00:00+00:00

I've recently moved back to using irssi as my IRC client, because when combined with GNU screen it can be kept independent of a graphical session, or even better, of a particular client machine (if you have a server somewhere). Since I have a home fileserver at the moment, I want my IRC client to always be running there. However, my usual workflow to do this is to open a terminal, SSH to the server, resume screen, move the window and resize it. This is far from optimal for something I usually want open...

Instead, I pieced together a command that I can use as a panel shortcut to do it all for me:

gnome-terminal --geometry 100x30-0-0 -x \
        ssh <hostname> -t "screen -D -RR -S irssi irssi"

gnome-terminal --geometry 100x30-0-0 -x: Open "gnome-terminal" at the bottom right of the screen with a size of 100 characters by 30. Run the rest of the command inside the terminal.
ssh <hostname> -t: Connect to <hostname> and execute the supplied command in an interactive pseudo-TTY.
screen -D -RR -S irssi irssi: Do whatever necessary to gain control of the "irssi" screen session if it exists, otherwise create it and run irssi inside it.

MPD + PulseAudio + Ubuntu Intrepid (8.10)

2009-03-21T00:00:00+00:00

Update (2009/04/12)

It appears that as of Ubuntu Jaunty everything now obeys the system-wide setting properly, so you only need to edit /etc/default/pulseaudio and add the user to pulse-access.

I'm a big fan of MPD, and up until the last few months I used it almost exclusively. It's lightweight, it's good at what it does, and there are a lot of different frontends available. One of the things I like most is that it's not tied to a graphical session in any way---you don't need a GUI open for it to be playing music. Normal usage of MPD involves it being started and stopped as a system service.

Unfortunately, this doesn't play too nice with the concept of how PulseAudio should be used. PulseAudio is very user- and session-centric, and the recommended setup means than when nobody is logged in, no sound is going to be playing. While MPD is capable of playing to a PulseAudio server just fine, having the sound server tied to the graphical session means it doesn't have a sound device to attach to at boot, and the sound device suddenly disappears if the user logs off.

Ubuntu has used PulseAudio for as many applications as possible by default since Hardy (8.04), and while it can be removed, doing so can make life difficult. This led me to pursue the option of a system-wide instance which everything uses, and which is always running. But no matter how hard I tried, I got stuck with the same bug: starting GNOME would always result in a per-session instance being spawned in addition to the system-wide one, the two would fight over the sound card, and I would have to manually kill the new one every time I logged in. Eventually I'd had enough of this and started using something else.

Today, I decided to take another stab at getting this set up properly and finding out where that second PulseAudio instance was coming from. Thanks to the help of a couple of people in #pulseaudio on FreeNode IRC, I was able to figure out how to do this, so here is the method I used for the benefit of anybody else in the same position.

Stop PulseAudio spawning per-session

By default PulseAudio puts a script in /etc/X11/Xsession.d which causes a pulseaudio instance to be spawned with every X session. The first step is to move this script somewhere else so it's never run:

sudo mv /etc/X11/Xsession.d/70pulseaudio /root/

The second cause of pulseaudio being spawned is less obvious. There is an Esound compatibility layer for PulseAudio so that applications depending on it still work, transparently:

$ ls -l /usr/bin/esd
lrwxrwxrwx 1 root root 9 2009-03-04 11:03 /usr/bin/esd -> esdcompat

However, as of Ubuntu Intrepid (8.10) there is pretty much nothing which depends solely on Esound, but it still gets spawned when GNOME starts. This causes a PulseAudio session to be spawned for the current user if one can't be found. The simple but ugly fix is to move this link to somewhere else:

sudo mv /usr/bin/esd /usr/bin/esd.bak

Set up a system-wide PulseAudio instance

To get PulseAudio to actually start on boot, you need to set a variable to tell the init script you want this to happen. Make sure you have the following line in /etc/default/pulseaudio:

PULSEAUDIO_SYSTEM_START=1

Any users that will be using the system-wide PulseAudio instance need to be members of the correct groups. This will most likely be your own user, plus whatever user MPD is running as (mpd by default). For each of these users, do:

sudo usermod -aG pulse-access,pulse,pulse-rt <user>

Getting it up and running

First thing's first, you need to tell MPD to use PulseAudio. This can be achieved by adding the following to /etc/mpd.conf:

audio_output {
    type    "pulse"
    name    "My MPD PulseAudio Output"
}

If you're feeling lazy, the easiest thing to do right now would be reboot. Otherwise, you'll need to kill all pulseaudio instances, restart the system-wide instance, and restart MPD:

$ sudo /etc/init.d/mpd stop
 * Stopping Music Player Daemon mpd                   [ OK ]
$ sudo killall -KILL pulseaudio
$ sudo /etc/init.d/pulseaudio restart
 * Stopping PulseAudio Daemon                         [ OK ]
 * Starting PulseAudio Daemon                         [ OK ]
$ sudo /etc/init.d/mpd start
 * Starting Music Player Daemon mpd                   [ OK ]

Finally, log out and log back in again to allow your per-session pulseaudio instance to die and for your new group memberships to take effect.

Caveats

You get "real-time" processing without any extra effort; by default the system-wide instance will run with priority -11.
Some apps are not PulseAudio-aware, but they can usually use an ALSA output; this guide shows you how to redirect ALSA through PulseAudio if you have any problems with the default setup.
Updates may undo the moving-a-file type of changes.

Universities push for higher fees

2009-03-17T00:00:00+00:00

My comment on a recent BBC News article "Universities push for higher fees":

I'm a university student from a low-income background, and having started my course just after the last raise in fees took effect, I knew from the beginning that I'd be finishing with £32k of debt (including maintenance loan). While expensive, it's a risk I was willing to take, since it's the best way to equip myself for the field I enjoy. If this number were to be more in the region of £85k, I think a lot of people would shy away from taking on such a staggering debt.

The government needs to realise that if they allow such increases, and also increase student loans to follow suit, then they will be footing the bill for allowing the fees to be raised (and therefore passing it on to the taxpayers). The vital safety net for most students is that student loans are written off 25 years after finishing/leaving the course, and since the university already has the money, the gap has to be filled from somewhere. This alone explains the observations leading to "Two thirds believed that fees had not deterred applications from students from poorer families". If the government wants to subsidise the universities, then they should propose an increase in per-student subsidies as an alternative, presenting the situation as it really is.

Any attempt to raise or un-cap fees that isn't backed completely by student loans will be disasterous. This would change the university admissions process to be less "those who are most able" and more "those who are most able to pay". For a government that campaigned on equal availability of higher education, this would be a step backwards. In a country with a shrinking manufacturing base, intelligent well-educated individuals are a vital resource for staying competitive in a global market.

As it stands, I've not heard of universities being severely underfunded, and where they claim to be it can usually be attributed to mismanagement of funds. If they could survive when we were paying £1150, why can't they survive when we're paying £3150? It seems to me this may be a case of "they did it once, maybe they'll do it again" on the part of universities, hopeful that the government will grant them more revenue without having to show anything for it.

My painless Linux "upgrade" process

2009-01-07T00:00:00+00:00

I like to keep up to date with the latest version of any Linux distro I use (naturally), but for some reason I don't trust the upgrade process to work properly and/or not leave a load of cruft behind. I've actually attempted an upgrade once just to try and get over my upgrade fear, but instead it went horribly wrong and convinced me to not try again for a long time.

There is a simple way to make reinstallations painless enough that you don't need to offer yourself up to the upgrade gods, and it takes only a small amount of forward planning. Always always always use a separate home partition! Keeping your home partition separate allows you to go as far as reformatting your main operating system partition without losing any of the data you really care about.

Here is my typical upgrade/reinstall process. If you're planning on using this method, please read the whole thing before starting and make sure you understand what you're doing and why. A lot of this is done as root and mistakes can be painful.

Note

Just in case it's not obvious, my username is alan - replace it with your own username! If you have multiple users, repeat operations on each user's home directory.

1 - Prepare

Your /home partition isn't going to be reformatted if everything is done correctly, so if you know what you're doing you shouldn't even need to do a full backup. However, you've probably got some other things on the root partition that have been changed that you want to keep, at least for future reference.

# Backup /etc
sudo tar czvpf /home/etc.tar.gz /etc
# Backup root's home directory
sudo tar czvpf /home/root.tar.gz /root
# Backup anything else you might want to /home, for example MySQL databases
# (maybe backup /var the same way as above?)
# ...

Next you need to make a note of which partition is which so you don't accidentally format the wrong one.

$ df
/dev/sda5             11718932   2419420   9299512  21% /
...
/dev/sda7             44133352    219500  41671984   1% /home

So in this case, /dev/sda5 is what you'll be reformatting and installing your new operating system on, and /dev/sda7 is the one you don't want to touch in any way whatsoever.

Next you'll want to write the fstab line for your home partition somewhere so you can use it on your new installation. The following command will get that from /etc/fstab for you:

$ grep "/home" /etc/fstab
/dev/sda7      /home      ext3     defaults,relatime      0     2

The final preparation step is to move your old home directory somewhere. To do this you'll need to be logged in directly as root. If you're running an Ubuntu-based distribution and haven't set the root password before, do so:

sudo passwd

Log out of your graphical session, switch to a virtual terminal (e.g. press Ctrl-Alt-F2), and log in as root. Move your old home directory:

mv /home/alan /home/alan-old

Everything should be ready now - time to reboot into the installer of your distro of choice!

2 - Installation

Just install as usual, feel free to obliterate your old root partition but make sure you don't touch your home partition! Install as if your home directory will be on your root partition along with everything else. (It's possible to just use your old home partition at this point, but most installers will format any partition you choose to use by default. This way, there is no risk of that happening.) Reboot into your new fresh Linux installation.

3 - Juggling

Now all that's left to do is get your home directory onto your home partition, get your home partition mounted and move all your files back to their rightful places.

Yet again, this process will require you to log in directly as root to move the home directory around, so set the root password as before if necessary for your distribution.

Now log in as root on a virtual terminal (e.g. Ctrl-Alt-F2) to mount your home partition somewhere, copy your new home directory across, and unmount it again:

mount /dev/sda7 /mnt
cp -a /home/alan /mnt/
umount /mnt

Now we want to make sure the home partition get's mounted at /home as it should. Edit /etc/fstab with your editor of choice and add the fstab line from earlier. To make sure everything is correct, mount the home partition in a way that'll use fstab, and check that both the old and new home directories are there:

$ mount /home
$ ls /home/
alan   alan-old

Just in case your user ID has changed, make sure your old home directory is owned by your new user:

chown -R alan:alan /home/alan-old

At this point, you should be finished with using the root account, and can log in as normal. If you want to lock the root account (in the way Ubuntu does by default), run:

sudo usermod -L root

4 - Get your files

Now you should be able to browse both your new and old home directories using your file manager of choice. Open one for each, show hidden files, and start dragging stuff around!

This step might take some time, but has the added advantage that you get to learn where every application stores its user files, and also gives you the opportunity to clear out some junk you didn't need, clean up your directory structure, or whatever else.

Final notes

This process is probably slower than an upgrade, especially considering the step of sorting your old files into your new home directory. However, it gives you very direct control over the process, lets you start with a nice clean copy of your operating system, and if done correctly doesn't require a full backup to be made.

I much prefer this method to doing an upgrade because it leaves you free of the "old version cruft" that accumulates when you upgrade---an operating system upgrade is a complex process and is never going to handle every edge case.

If you trust all of your applications to handle old configuration files etc. correctly and you want to save some time, you can skip all of the steps relating to moving/copying of your user's home directory, so that when you mount your home partition your old home directory is used. If reinstalling rather than upgrading, I tend to do this.

Caution!

While I mention that this can be done without a backup, doing so is at your own risk. Generally speaking it is prudent to backup anything important to somewhere outside of your machine before doing any operating system installation to protect against mistakes. Otherwise, be extremely careful! I do things this way because my home directory is always too large to backup anywhere, but I also run an automated nightly backup of the small, important stuff.

C linked list macro

2008-12-19T00:00:00+00:00

Here's a simple macro I came up with a few weeks ago for easily defining linked list types in C:

#define LINKED_LIST(type, name)     \
    struct name ## _ {              \
        struct name ## _ * next;    \
        type data;                  \
    };                              \
    typedef struct name ## _ name;

Usage is simple:

LINKED_LIST(int, int_ll);

gives the same type as:

struct int_ll_ {
    struct int_ll_ * next;
    int data;
};
typedef int_ll_ int_ll;

Why I dislike PulseAudio (for now)

2008-12-17T00:00:00+00:00

Before this sounds like a rant, I'd like to first say that I think PulseAudio is a very nice concept, from a technical perspective. It's about time Linux got a manageable sound subsystem, and some of the features---such as network streaming and synchronised playback---are quite impressive.

However, one thing that really irks me is that, as deployed by major distributions, it appears to enforce the narrow-minded view that a desktop is used by only one person, and that sound is only ever going to play when somebody is logged in to a graphical session. Anybody who uses MPD has probably experienced the pain caused by this already. Unfortunately my experience so far with Ubuntu Intrepid is that trying to change this results in even more pain. Having set up a system-wide instance I'm now in the position where the first time I attempt to play sound works correctly, but any subsequent attempts fail until I kill a per-session PulseAudio instance that has magically spawned (even though I've set it not to). Once the per-session instance is dead, I have no more problems and everything works perfectly until the next time I log in.

I'm currently trialling Fedora 10 on my desktop, so I'll update this post if I have any more luck there. Feel free to correct me in the comments if I'm just being stupid or missing something!

Update: I've since fixed this problem.

The curse of Firefox extensions

2008-02-24T00:00:00+00:00

Today is my first day of Firefox usage after a three-week break - a break which taught me something...

First, a little background. I've been using Firefox since 0.9, and generally its always been the best browser for me. However, recently I lost my taste for it due to the monumental lack of speed it can display sometimes. I mean things like a half-second delay between hitting Ctrl-T and being able to type in the address bar of the new tab, really long page rendering times, etc. I know that Firefox slowness is generally due to extensions, but I relied on most of my extensions. Here is a list of the extensions I can remember having:

AdBlock Plus
AdBlock Filterset.G updater
CustomiseGoogle
Download statusbar
Firebug
Flashblock
Foxmarks
NoScript
QuickRestart
Stop Autoplay
Stylish
Tab Mix Plus
Tiny Menu
Web developer toolbar

Then I remembered I had tried the Epiphany browser (for GNOME) at some point in the past, and that it had been quite snappy. To put it simply, Epiphany is a browser that uses the Gecko rendering engine, but doesn't include all the other stuff that slows down Firefox (which I can only assume to be the Extension framework and XUL). The upside is that its blindingly fast. The downside is that you no longer have all the amazing Firefox extensions you've become used to. One other advantage over Firefox is that it already has working tag-based bookmarking (slated for Firefox 3.0).

After forcing myself to cope without my extensions for a day, I started to really enjoy the speed and lack of wait-induced stress - Epiphany stays out of my way and gets stuff done really fast. Once I had the momentum, I just kept going. I discovered nice stuff like the fact typing in the address bar also searches your bookmarks as well as your history (something also in Firefox 3.0). Epiphany's UI is reasonably customisable, but not to the extent that Firefox's is.

Eventually I had an epiphany (terrible pun, I know...) - I had just survived 3 weeks without the huge number of extensions I came to think I needed in Firefox. I now realised I could go back to Firefox and use it more sensibly. I threw out my old Firefox profile and started again. I now have only the following well-justified extensions:

Flashblock - on a laptop, unexpected Flash ads = bye-bye battery life
Foxmarks - because I want my bookmarks on at least 3 different machines
QuickRestart - no real effect on performance, and sometimes you want to restart Firefox quickly!

I'm happier with this setup than I was with Epiphany, and it's a good trade-off between fancy features and responsiveness. One thing I really like about Firefox is the ability to put anything anywhere on the toolbars, and I use a fairly condensed layout, so I was also really happy to go back to the layout I liked.

In my opinion, the best thing to do when deciding if an extension is worth installing is ask yourself the following questions:

Does it analyse and/or change what I am viewing?
Does it make comparisons against a large whitelist/blacklist?

Extensions doing either of the above will almost certainly slow your browser down to some extent. You will notice that the only content-sensitive extension I have is Flashblock, which is really worth it on a laptop.

Another option is to have multiple Firefox profiles, one of which is clean (like my current setup), and the other having big slow extensions that you sometimes need, like Firebug. Using the -P option to Firefox you can set up a launcher that starts your second profile instead of the default.