August 20, 2008

Framing protocols

I've been playing with framing protocols.

I need a protocol with:

  • low overhead: it will be used for mobile communications and some of us pay for each byte;
  • multiplex streams: you should be able to open channels inside the same TCP connection;
  • restartable: each session has an ID and I need to restart all of the channels inside it very quickly if my TCP connection dies.

I've looked at AMQP wire protocol, BLIP, BEEP and some others. I plan to look over SSH too.

I'm looking for further suggestions of protocols to reuse. Anyone?

Parallax

I've talked a bit about Freebase over the last few months. I understand that a rich, semantic, structured database of knowledge is not something other people get excited about, but that happens to be my thing.

In the hope to excite others with Freebase, I recommend that you spend 8 minutes watching the Parallax browser in action. That kind of research tool is only possible with something like Freebase on the bottom.

Update: the code for the Parallax browser is open source now.

August 11, 2008

Updated Search In Project with Ack

I've updated my own Search In Project with ack TextMate command.

Changes:

  • search in selected files is optional now: start your query with a : to search only in the selected files;
  • fixed search for words with a initial dash in them.

You can find the current version at GitHub or download the tmCommand file directly.

This is a very simple ack interface. For a more complex and feature-full, see the ack TextMate bundle by Trevor Squires.

August 08, 2008

This cannot be natural

Really, see this...

In her own words, this cannot be natural.

August 05, 2008

XML::LibXML braindead, or is it just me?

I spent the last 30 minutes chasing down this bug. The following tests should all pass but the last one doesn't:

use strict;
use warnings;
use Test::More 'no_plan';

use XML::LibXML;
use XML::LibXML::XPathContext;

my $parser = XML::LibXML->new;
ok($parser, 'XML::LibXML parser created');

my $xml = '<x:me xmlns:x="some_namespace" />';
my $xdoc = $parser->parse_string($xml);
ok($xdoc, 'Valid XML parsed');

$xdoc = XML::LibXML::XPathContext->new($xdoc);
ok($xdoc, 'Converted to XPathContext');

my ($node) = $xdoc->findnodes('/me');
ok(!$node, 'Not found because no prefix means NULL namespace');

($node) = $xdoc->findnodes('/x:me');
ok($node, 'Found because using document prefix');

$xdoc->registerNs( new => 'some_namespace' );
($node) = $xdoc->findnodes('/new:me');
ok($node, 'Found because using new specific prefix');

# Notice the change of namespace
my $uri = $xdoc->lookupNs('x');
is($uri, 'some_namespace', 'x prefix is some_namespace');
$xdoc->registerNs( x => 'other_namespace' );
$uri = $xdoc->lookupNs('x');
is($uri, 'some_namespace', 'x prefix is still some_namespace');

($node) = $xdoc->findnodes('/x:me');
ok(!$node, 'Not found because using document prefix was changed to diff namespace');

The basic problem is this: you get a XML document in which a namespace N is tied to a prefix P. I was expecting that if I set in my own parser prefix P to something else, that my local prefix P would take precedence over the document P. Apparently it does not.

Right now, what I did was this: before using XPath to match anything, I always used registerNs method of XML::LibXML::XPathContext to make sure I was getting the right elements. Now, I use this new function instead:

sub _safe_ns_register {
  my ($xpc, $prefix, $ns) = @_;

  while ($xpc->lookupNs($prefix)) {
    $prefix++;
  }
  $xpc->registerNs($prefix => $ns);

  return $prefix;
}

This function makes sure that the prefix I choose does not clash with any prefixes on the document. It returns the prefix I must use on my XPath expressions.

It solved my problem, but I still think that my local prefix should take precedence over the document one.

August 04, 2008

The DNS wars

The recent DNS bugs sent me reading the cr.yp.to site for old stories, long forgotten. djbdns is one of the DNS server softwares that was immune to this problem from the start.

Some pearls:

I still receive a diff of changes to the cr.yp.to site, almost 5 years running now, I wonder if anybody has an archive of them. It would be interesting to assemble all of those diffs into a repository to track back the changes.

August 01, 2008

STFU

So you have a friend who is going to spend some time traveling and that journey includes the United States. Usually you would buying him a travel guide.

But our times require something different. I recommend that you send your fiend the link to Professor James Duane talk entitled "Don't talk to the Police".

In a country where habeas corpus is no longer controlled by the court system, those 26 minutes could save your friend a lot of trouble (as confirmed in the follow up presentation by a police officer).

(via Schneier)

Back...

The latest weeks where a bit crazy but normality is returning.

July 21, 2008

Tweaks to MySQL installation

After you install one of the MySQL packages available for the Mac, there are some steps that you should do.

First, make sure your MySQL installation knows about time zones. This is important if you want to run your MySQL in the UTC time zone.

To update the mysql database time zone tables, do:

mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root -p mysql

Type the password at the prompt (hit enter if you don't have one yet).

And then make sure all your date/datetime fields use the Highlander-timezone. Edit my.cnf and add:

[mysqld]
default-time-zone=utc

Second, make sure your server is using UTF-8 everywhere. Add to my.cnf:

[mysql]
default-character-set=utf8

[mysqld]
character-set-server=utf8

Third, set mysql to strict SQL:

[mysqld]
sql-mode="TRADITIONAL,NO_ENGINE_SUBSTITUTION,ONLY_FULL_GROUP_BY"

Finally, make sure you use InnoDB by default:

[mysqld]
default-storage-engine=InnoDB

There are probably more tweaks to make your MySQL saner. This are the one I feel comfortable recommending.

DBD::mysql and db_imp errors

I'm installing all my MySQL stuff in a Leopard 10.5.4 desktop and I made some mistakes along the way that I though about documenting here for future reference.

First, although the hardware and OS are 64-bit in a lot of places, the standard perl installed is not one of those. So stick with the i386 MySQL package (or try a 64bit server, but use the 32bit client...). I'm using the Proven Scaling MySQL packages mentioned earlier, and I'm happy so far.

Second, make sure your regular user has all privileges to the test databases. I just do:

melo@DogsHouse:lib $ mysql -uroot -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 108
Server version: 5.0.62-enterprise-gpl MySQL Enterprise Server (GPL)

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> grant all privileges on test.* to 'melo'@'localhost';
Query OK, 0 rows affected (0.00 sec)

mysql> quit
Bye

This will make your DBD::mysql tests much happier.

Third, in case you see failing DBD::mysql tests with:

Can't use dbi_imp_data of wrong size (127 not 124) at ...

Upgrade your DBI. I'm now with 1.605 and no dbi_imp_data errors anymore. Clean DBD::mysql install.

$chromatic++

Nothing to add, really. Maybe another day.

QOD

In response to the FSF article on the iPhone, let me point you to the balanced view of Stu Charlton.

Quote of the day:

The OpenMoko counter-argument is "give it time, in the long run, it will win". And look, in a way, I hope so. Using the iPhone is a great case of following Keynes' adage, in the long run, we are all dead., where we optimize for short term gratification at the expense of our future.

July 18, 2008

Creating a new user using dscl

In case you ever need to create a new user in Leopard using the command line, the required steps are documented in the Porting UNIX/Linux Applications to Mac OS X document.

Disqus

Heads up: I'm switching my comments to Disqus. I had several people complaining about the current HaloScan-based system.

Expect minor disturbances in the force.

Update: we are now Disqus-powered. They still don't have an importer for old comments, so for a while, old comments are orphaned. Hope they fix that soon.

Subversion 1.5.0

I missed the release of the new 1.5.0 version of Subversion. Its funny, actually. This was somewhere around June (the blog post lacks a proper date), and I didn't see it mentioned on any of the blogs I follow.

Anyway, its out and I was expecting it to see the new merge tracking stuff. One of my gripes with Subversion is that yes, branching is cheap, but the merge part is awful hard to do and puts much of the load into your lap.

From my superficial first look, all I can say is this: what a mess!

I need to read the documentation more thoroughly but it seems full of special cases and workarounds for common scenarios that should just work.

For example, the "Reflective/Cyclic merges" that are mentioned, are the most common ones for me when I'm using feature branches.

I think this phrase is telling:

The only way to truly solve this is to invent a new merge algorithm in Subversion that does not rely simply on revisions

Of course you can think that branches are not good for you, and that they slow you down. I think its a matter of personal organization. Multiple active branches fit my brain and my workflow just fine, and I love to organize stuff with them.