Console control with readline

The GNU Readline Library is a really useful library for creating single-line input style programs at the console. From the introduction:

The GNU Readline library provides a set of functions for use by applications that allow users to edit command lines as they are typed in.

There’s some really nice stuff added on top though. Being able to use emacs or vi editing modes are just some of them. This allows you to provide your users with consistent key bindings to what they use in their text editor.

Modules

Taking a look on your file system, you can find readline’s header files under /usr/include/readline/.

chardefs.h  keymaps.h   rlconf.h  rltypedefs.h
history.h   readline.h  rlstdc.h  tilde.h

Taking a really brief tour of these headers, history.h provides the library with the ability to recount previously entered text blocks. This sort of behavior can be observed when using bash; pressing the up and down buttons allow you to scroll through previous commands. keymaps.h provides the library with the mapping abstraction to make your readline-based application behave like vi or emacs or any custom implemented map you wish.

Reading lines

The library’s purpose is to read lines of text. The function that you’ll use to do this is called readline. A simple example of this would be:

#include <stdlib.h>
#include <stdio.h>
#include <readline/readline.h>

int main() {
  char *name;

  name = readline("What is your name? ");

  printf("Hello %s!\n", name);
  free(name);

  return 0;
}

Note! The string that you’re given back from readline is allocated using malloc, so it’s your responsibility to ensure that the memory is freed using free.

Resulting with the following execution:

$ ./hello
What is your name? Joe
Hello Joe!

Historical and auto complete

You can quickly give your readline program a historical account of previously entered lines with the following sort of structure:

while (running) {
  filename = readline("Enter a filename: ");

  // add the data to the input history
  add_history(filename);

  // perform some work on "filename"

  free(filename);
}

The add_history method is available from the readline/history.h header.

Allowing your line edit to include file path auto completion is pretty easy also. You can bind this behavior to any key that you’d like; we’re all pretty used to the tab character doing this work for us though:

rl_bind_key('\t', rl_complete);

Using libxml2

libxml2 is a C library that provides XML handling capabilities to your applications. It supports an extensive range of standards and specifications common in the XML community. It’s also a highly ported library between platforms and architectures making it a good choice when your application needs to move.

For a deeper-dive into the libxml2 library, I suggest a read through on their tutorial.

Building applications

The xml2-config program allows your Makefiles to remain relatively noise-free. To build any applications against libxml2, you just need the --cflags and --libs switches to compile and link respectively:

$ gcc `xml2-config --cflags --libs` prog.c -o prog

Writing programs

Rather than re-produce them all, I’ve just got a link to the set of examples on the libxml2 site.

File encodings and iconv

There are a handful of really useful tools for dealing with character encoding. In today’s post, I’ll take you through identifying this characteristic and changing it.

What is Character Encoding?

Wikipedia has the most comprehensive breakdown on the topic. The simplest way to look at it though is that a character encoding assigns a code to each character in an alphabet.

The unicode and latin4 characters don’t exist within the ASCII character space, therefore those characters simple don’t translate and can’t be encoded by ASCII.

Querying files

To determine what encoding is being used with a file, you can use the file unix utility.

$ echo "Here is some text" > a-test-file
$ file a-test-file
a-test-file: ASCII text

Using the -i switch, we can turn the ASCII text output into a mime string which can yield some more information:

$ file -i a-test-file
a-test-file: text/plain; charset=us-ascii

To make this encoding representation change a little clearer, below I’ve pasted in the output of hexdump on that test file that we’d created earlier.

0000000 6548 6572 6920 2073 6f73 656d 7420 7865
0000010 0a74                                   
0000012

Remember, these bytes are not only in hex; they’re also flipped according to how the string is written. Let’s take the first two bytes 6548:

0x65 = e
0x48 = H

We’re using an 8-bit encoding, our string has 17 characters plus a newline (18). Easy.

Changing the encoding of a file

We can use iconv to transition our text file from one encoding to another. We specify its current encoding with the -f switch and the encoding that we want to convert it to using the -t switch.

$ iconv -f ascii -t unicode a-test-file > a-test-file.unicode

This is changing our test file into an encoding that uses more data-space per character. Taking a look at the type of file we’ve just created:

$ file a-test-file.unicode a-test-file.unicode: Little-endian UTF-16 Unicode text, with no line terminators

If we take a look at the hexdump of this file, you can see that every byte is now padded with an extra zeroed-out byte.

0000000 feff 0048 0065 0072 0065 0020 0069 0073
0000010 0020 0073 006f 006d 0065 0020 0074 0065
0000020 0078 0074 000a                         
0000026

The file also starts with a BOM of feff which was unseen in the ASCII counterpart.

What encodings are supported

You can list the known coded character sets with iconv as well with the --list switch. This will dump a massive list of encodings (and aliases) that you can use.

More!

A really good article was written about the The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!). It’s certainly well written article that’ll give you some schooling in encodings, quick smart.

Doing math with bash

bash can be used to perform simple arithmatic when needed. It is only limited to integer mathematics though. You can see this by using the expr.

An excerpt about expr from the link above:

All-purpose expression evaluator: Concatenates and evaluates the arguments according to the operation given (arguments must be separated by spaces). Operations may be arithmetic, comparison, string, or logical.

Some trivial example usage shows that you can get some quick results if you only need integer math.

$ expr 1 + 1 
2
$ expr 1 - 7
-6
$ expr 6 \* 7
42
$ expr 6.5 \* 7
expr: non-integer argument

At this point, you could probably go for the full-nuclear option and get perl or python to perform floating point calculations; but to keep things a little more shell oriented, you can go with a lighter-weight option, bc.

bc is an arbitrary precision calculator language. Much like every other good shell tool, you can invoke it so that it’ll take its input from STDIN and return its output to STDOUT. Here are some example invocations:

$ echo "1+1" | bc
2
$ echo "1.9+1" | bc
2.9
$ echo "76/5" | bc
15
$ echo "scale=2; 76/5" | bc
15.20

You can see that if you want precision on your answers from integer inputs, you’ll need to set the scale variable to suit. Only feeding in static values is a bit basic though. To put this to work, you just need some variable data at hand.

What’s the percentage battery left on this notebook?

$ echo "scale=2;" $(cat /sys/class/power_supply/BAT0/charge_now) / $(cat /sys/class/power_supply/BAT0/charge_full) | bc
.30

Not much!

node.js module patterns

In today’s post, I’ll walk through some of the more common node.js module patterns that you can use when writing modules.

Exporting a function

Exporting a function from your module is a very procedural way to go about things. This allows you to treat your loaded module as a function itself.

You would define your function in your module like so:

module.exports = function (name) {
  console.log('Hello, ' + name);
};

You can then use your module as if it were a function:

var greeter = require('./greeter');
greeter('John');

Exporting an object

Next up, you can pre-assemble an object and export it as the module itself.

var Greeter = function () { };

Greeter.prototype.greet = function (name) {
  console.log('Hello, ' + name);
}

module.exports = new Greeter();

You can now start to interact with your module as if it were an object:

var greeter = require('./greeter');
greeter.greet('John');

Exporting a prototype

Finally, you can export an object definition (or prototype) as the module itself.

var Greeter = function () { };

Greeter.prototype.greet = function (name) {
  console.log('Hello, ' + name);
}

module.exports = Greeter;

You can now create instances from this module:

var Greeter = require('./greeter');
var greeter = new Greeter();
greeter.greet('John');