Working with lambdas in python

23 Feb 2015

Python provides a simple way to define anonymous functions through the use of the lambda keyword. Today’s post will be a brief introduction to using lambdas in python along with some of the supported higher-order functions.

Declaration

For today’s useless example, I’m going to create a “greeter” function. This function will take in a name and give you back a greeting. This would be defined using a python function like so:

def greet(name):
	return "Hello, %s." % (name,)

Invoking this function gives you endless greetings:

>>> greet("Joe")
'Hello, Joe.'
>>> greet("Paul")
'Hello, Paul.'
>>> greet("Sally")
'Hello, Sally.'

We can transform this function into a lambda with a simple re-structure:

greeter = lambda name: "Hello %s" % (name,)

Just to show you a more complex definition (i.e. one that uses more than one parameter), I’ve prepared a lambda that will execute the quadratic formula.

from math import sqrt

quadratic = lambda a, b, c: ((-b + sqrt((b * b) - (4 * a * c))) / (2 * a), (-b - sqrt((b * b) - (4 * a * c))) / (2 * a))

This is invoked just like any other function:

>>> quadratic(100, 45, 2)
(-0.05, -0.4)
>>> quadratic(100, 41, 2)
(-0.0565917792034417, -0.3534082207965583)
>>> quadratic(100, 41, 4)
(-0.16, -0.25)

Higher order functions

Now that we’re able to define some anonymous functions, they really come into their own when used in conjunction with higher-order functions. The primary functions here are filter, map and reduce.

We can filter a list of numbers to only include the even numbers.

filter(lambda x: x%2 == 0, range(1, 10))

Of course it’s the lambda x: x%2 == 0 performing the even-number test for us.

We can reduce a list of numbers to produce the accumulation of all of those values:

reduce(lambda x, y: x + y, range(1, 10))

Finally, we can transform a list or map a function over a list of numbers and turn them into their inverses:

map(lambda x: 1.0/x, range(1, 10))

Virtual buffer for liquid crystal displays

22 Feb 2015

Today’s post will be about a double buffer implementation that I’m working on for the Liquid Crystal display module with the Arduino micro-controller.

What I’m using?

To write this article, I’m using the following hardware:

For all of the software development that I’ve done here, I’m using the standard Arduino IDE available from their site.

Using the module “normally”

Having a look at the documentation for the Liquid Crystal library, there’s already a comprehensive implementation of filling, scrolling and blinking functions available.

The idea of this article isn’t to re-implement this library, it’s to build on it to add functionality.

The usual setup of the library would looks something similar to this:

#include <LiquidCrystal.h>

LiquidCrystal lcd( 8, 9, 4, 5, 6, 7 );

void setup() {
  lcd.begin(16, 2);
}

This initializes our LCD module, ready for us to start writing to it:

lcd.print("Hello, world!");

Nothing really special here, and rather than trying to piece together the “Hello, World” example from this code here, you’d be better off checking out the samples section for this module.

What’s the idea?

Given that we have a display of 16 characters in width by two characters in height, we’re rather limited in what we can do. What I want to do is expand the 16x2 interface to have a “virtual canvas” to work with of any arbitrary size that you can just use the 16x2 LCD as a window into the canvas.

To begin with, we’ll create some constants to assert our assumptions!

#define LCD_WIDTH      16
#define LCD_HEIGHT     2
#define LCD_SIZE       (LCD_WIDTH * LCD_HEIGHT)

These are pretty straight forward. All of the constants come in handy. Bounds checking is where they really shine, but we also specifically use LCD_HEIGHT as a pitch-type variable that helps us calculate how far into a flat-memory array we need to be to start writing. More on this later.

What do we want to do?

I think initially it’d be nice to:

Define a virtual area (that’s larger than our physical display)
Move the “window” to any point in the virtual area
Render the view at the window co-ordinates to the physical display
Print to arbitrary points of the buffer

I’ve created a class, LCDDoubleBuffer that should wrap all of this functionality up. I’m going to start going through the code from here on.

Define the virtual area

Give a LiquidCrystal reference and buffer dimensions, we can start to initialize our double buffer:

LCDDoubleBuffer(LiquidCrystal *lcd, const int width, const int height) {
	this->_width = width;
	this->_height = height;
	this->_size = width * height;
	this->_lcd = lcd;

	this->_win_x = 0;
	this->_win_y = 0;

	this->_buffer = new char[this->_size];
}

_width, _height and _size all manage the internal data structure for us so that we remember how big our virtual area is.

_lcd is our link back to the “real-world” for us to perform some physical side-effects on our display.

_win_x and _win_y are to co-ordinates to the top left-hand corner of the window that we’ll display on the LCD.

Finally, _buffer is our virtual canvas. Be careful here! Looking at the memory specs, you only get a whopping 2k SRAM to play with. With a char taking up 1 byte, you can see that you’ll quickly go over this if you’re not careful.

Working with the buffer

Well, it’s pretty simple. These are all string operations, and they can be reduced even further than this to say that they’re just memory operations.

To clear the buffer out, we use memset.

void clear() {
	memset(this->_buffer, ' ', sizeof(char) * this->_size);
}

To write a string at a given location, we use memcpy. Be careful here! The code that I present below doesn’t perform any bounds checking on where we are in the buffer. This is buffer under-and-overrun city, right here.

void print(const int x, const int y, const char *s) {
	char *o = this->_buffer + (x + (y * this->_width));
	memcpy(o, s, strlen(s));
}

o which is our offset pointer into the buffer is adjusted by calculating the linear address from x and y which is simply:

linear = x + (y * width)

Rendering the window

This is pretty brute-force. We enumerate over every valid cell defined for our LCD and write out the corresponding character.

void render() {
	int buf_y = this->_win_y;
	for (int y = 0; y < LCD_HEIGHT; y ++) {
	  
		int buf_x = this->_win_x;

		for (int x = 0; x < LCD_WIDTH; x ++) {

			this->_lcd->setCursor(x, y);

			if (buf_y >= 0 && buf_y < this->_height &&
			    buf_x >= 0 && buf_x < this->_width) {
				int ofs = buf_x + (buf_y * this->_width);
				this->_lcd->print(this->_buffer[ofs]);
			} else {
				this->_lcd->print(" ");
			}

			buf_x ++;
		}

		buf_y ++;
	}

}

That big-fat-if-statement in the middle is protecting us from colouring outside the lines. When we are outside the bounds, we’ll write a clearing character (in this case a whitespace).

Actually using the code

We’ve got a double-buffer defined, ready to go. To test it out, I’ve decided to spice-up the “Hello, World” example and get it scrolling on the display using cosine values so it should ease-out and ease-in; back and forth.

Don’t get too excited though. It’s not brilliant.

// create the LCD reference and double buffer
LiquidCrystal lcd( 8, 9, 4, 5, 6, 7 );
LCDDoubleBuffer dbuf(&lcd, 32, 32);

void setup() {
	Serial.begin(9600);

	lcd.begin(16, 2);

	// put "hello world" in the top corner
	dbuf.clear();  
	dbuf.print(0, 0, "Hello, world!");
}

float degs = 0.0f;

void loop() {
	// calculate and flatten to an int
	float a = -3 + (cos(degs) * 5);
	int x = (int)a;

	// move in position and render
	dbuf.moveTo(x, 0);
	dbuf.render();

	delay(50);

	// move along the cos curve
	degs += 0.1f;

	// 360 degrees == 0 degrees
	if (degs > 360.0f) {
		degs = 0.0f;
	}  
}

And there you have it. The full source of this demo is available here as a gist in my GitHub repository.

Textfile processing with bash

10 Feb 2015

Today’s post will be a bash on liner that allows you to process text files, line-by-line.

while read line; do wget $line; done < urls.txt

In this case, urls.txt is a list of urls that we want to process using wget. $line is the value that we’re currently processing (the line of text).

This is a pretty heavy handed way of downloading a handful of URLs. You could much easier do it with xargs which I’d mentioned in a previous post. Anyway, the while way gives you a bit more room to breathe in your loops.

Developing with Tomcat user instances

10 Feb 2015

In today’s post, I’ll take you through installing Apache Tomcat for the purposes of development.

Installing a user instance

To get started, install tomcat8 as you normally would:

$ sudo apt-get install tomcat8

For the purposes of development, it makes sense to have your own instance of tomcat which is away from the system installation. The tomcat8-user package allows you to do just that.

$ sudo apt-get install tomcat8-user

After this install is complete, you can create yourself a local tomcat instance that you can blow up without hurting the system’s version. You do this with the tomcat8-instance-create command:

$ tomcat7-instance-create -p 10080 -c 10005 tomcat

The switches -p puts this instance listening for application requests on port 10080 and the -c switch puts the control port on 10005.

After you’ve done this, you’ll be notified by the console.

You are about to create a Tomcat instance in directory 'tomcat'
* New Tomcat instance created in tomcat
* You might want to edit default configuration in tomcat/conf
* Run tomcat/bin/startup.sh to start your Tomcat instance

The directory that has been setup for you now looks like this:

.
├── bin
│   ├── setenv.sh
│   ├── shutdown.sh
│   └── startup.sh
├── conf
│   ├── catalina.properties
│   ├── context.xml
│   ├── logging.properties
│   ├── server.xml
│   ├── tomcat-users.xml
│   └── web.xml
├── logs
├── temp
├── webapps
└── work

Integrating with Eclipse

To get Eclipse to play nicely with your user-local version of tomcat, you’ll still need to add a few components. This tip is largly based off of the information in this stack overflow question.

$ ln -s /usr/share/tomcat8/lib
$ cp /etc/tomcat8/policy.d/03catalina.policy conf/catalina.policy
$ ln -s /usr/share/tomcat8/bin/bootstrap.jar bin/bootstrap.jar
$ ln -s /usr/share/tomcat8/bin/tomcat-juli.jar bin/tomcat-juli.jar
$ mkdir -p common/classes;
$ mkdir -p server/classes;
$ mkdir -p shared/classes;

You can now add your local user instance of tomcat to Eclipse.

Start logging with log4j2

08 Feb 2015

Probably the easiest way to get logging into your java application is with log4j from Apache. In today’s post I’m going to setup some basic logging with the 2.x series of this library.

Getting installed

The first thing to do is to download log4j from Apache’s downloads page. Once you have the binary distribution, extract it out into your preferable location. I personally put all of my libraries under ~/src/lib that I use for my projects.

Add the following jars to your lib folder in your project that you’re going to add logging to:

log4j-api-2.1.jar
log4j-core-2.1.jar

The versions may be differ, but as long as you’ve referenced the api file and the core file, you’ll be fine.

First logs

In order to get started, you’ll need a concrete implementation of the Logger interface. This is pretty easy with the help of the LogManager class:

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

public class LoggerTest {

static Logger log = LogManager.getLogger(
LoggerTest.class.getName()
);

public static void main(String[] args) {
}

}

We give getLogger the name of the class that we’re in so that the resulting log can reference this name.

From here on, you’ll reference the variable log to send anything into the log. Running this application as it is results in the following error from the log4j2 framework:

ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.

That’s fine, we can still take a look at what an error looks like then by adding the following line to main:

log.log(Level.ERROR, "Error message");

The result is that our line gets logged out:

16:25:16.656 [main] ERROR org.keystoreplay.LoggerTest - Error message

Configuration

The log4j manual has a whole section dedicated to configuring the framework. If you’re looking to do more complex things, that’s the best place to look.

Quoted directly from the manual are the following bullet points regarding the process that the framework will go through to perform automatic configuration:

Log4j will inspect the “log4j.configurationFile” system property and, if set, will attempt to load the configuration using the ConfigurationFactory that matches the file extension.
If no system property is set the YAML ConfigurationFactory will look for log4j2-test.yaml or log4j2-test.yml in the classpath.
If no such file is found the JSON ConfigurationFactory will look for log4j2-test.json or log4j2-test.jsn in the classpath.
If no such file is found the XML ConfigurationFactory will look for log4j2-test.xml in the classpath.
If a test file cannot be located the YAML ConfigurationFactory will look for log4j2.yaml or log4j2.yml on the classpath.
If a YAML file cannot be located the JSON ConfigurationFactory will look for log4j2.json or log4j2.jsn on the classpath.
If a JSON file cannot be located the XML ConfigurationFactory will try to locate log4j2.xml on the classpath.
If no configuration file could be located the DefaultConfiguration will be used. This will cause logging output to go to the console.

What I like to do is contain all of my configuration files under a directory; normally called etc or conf. It doesn’t matter where your configuration files go, just as long as the classpath references them.

A simple configuration for this system is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
  <Appenders>
    <Console name="Console" target="SYSTEM_OUT">
      <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
    </Console>
  </Appenders>
  <Loggers>
    <Root level="trace">
      <AppenderRef ref="Console"/>
    </Root>
  </Loggers>
</Configuration>

This was based off a config block found in the configuration manual. Note that the Level attribute has been set to trace. This will throw everything to the console for us. Adding the following line:

log.log(Level.WARN, "A simple warning");

We get the following output (without an error message telling us that we haven’t configured to logger):

16:40:14.207 [main] WARN  org.keystoreplay.LoggerTest - A simple warning
16:40:14.210 [main] ERROR org.keystoreplay.LoggerTest - Error message

Older Newer

Cogs and Levers A blog full of technical stuff