Cogs and Levers A blog full of technical stuff

Standard object protocols in Python

To give your objects a more baked-in feel, you can use python’s standard object protocol functions so that native operators start to operate on your object.

By implementing the following items on your custom objects, infix operators start to work executing your custom code as per defined.

General

By overriding __bool__ in your objects, you can define how your object will respond in conditional scenarios. __bool__ effectively allows you to use your object as a condition in an if or while statement.

class Boolable:
  def __init__(self, n):
    self.num = n

  def __bool__(self):
    return self.num % 2 == 0    

The method __call__ will allow your object to openly accept function calls:

class Callable:
  def __call__(self, *args, *kwargs):
    # implementation here

c = Callable()
c()

Array

The following overrides allow you to make your objects appear like containers (arrays, etc.):

Operation Method Description
Length __len__ Allows the len function to operate on the object
Iterator __iter__ Gets an object to start the iteration process
Next __next__ Gets an object to continue the iteration process
Reverse __reversed__ Reverses the internal sequence

Dictionary

The following overrides allow you to make your object respond like a dict:

Operation Method Description
Set item __setitem__ Sets an item in the dictionary
Get item __getitem__ Retrieves an item from the dictionary
Delete item __delitem__ Removes an item from the dictionary

Mathematic

The following table lists out all of the methods that you can override on a class that will give you access to arithmetic operators.

Operation Operator LHS RHS Inline
Addition + __add__ __radd__ __iadd__
Subtraction - __sub__ __rsub__ __isub__
Multiplication * __mul__ __rmul__ __imul__
True Division / __truediv__ __rtruediv__ __itruediv__
Floor Division // __floordiv__ __rfloordiv__ __ifloordiv__
Modulo % __mod__ __rmod__ __imod__
Division and Modulo divmod __divmod__ __rdivmod__  
Exponentiation ** __pow__ __rpow__ __ipow__
Shift left << __lshift__ __rlshift__ __ilshift__
Shift right >> __rshift__ __rrshift__ __irshift__
Bitwise AND & __and__ __rand__ __iand__
Bitwise OR | __or__ __ror__ __ior__
Bitwise XOR ^ __xor__ __rxor__ __ixor__
Bitwise NOT ~ __invert__    
Function Method
Floor __floor__
Ceil __ceil__
Round __round__
Negate __neg__
Positive __pos__
Absolute __abs__

Comparison

The following table lists all of the comparison operators

Operation Operator Method
Equals == __eq__
Not Equal != __ne__
Greater than, equal to >= __gte__
Greater than > __gt__
Lesser than, equal to <= __lte__
Lesser than < __lt__

Type conversions

Type Method Description
int __int__  
float __float__  
complex __complex__  
index __index__ Allows python to use your object as an array index

Context

The following override allow your objects to measure contexts:

Operation Method Description
Enter __enter__ Measures when a context enters
Exit __exit__ Measures when a context exits

These functions are useful when your object is supplied to a with statement.

class ContextMeasurement:
  def __enter__(self):
    print("Entering context")
      
  def __exit__(self, exc_class, exc_instance, traceback):
    print("Exiting context")
        
with ContextMeasurement():
  print("Inside the context right now")

Getting started with Akka

Akka is a library designed for building applications using the actor model. From their site:

Akka is a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM.

In today’s post, I’m going to start with some of the primitives to using this framework.

Messages

Actors process messages that you’ll define in your modules. For today’s example, I’m going to implement a very basic logging application. Messages sent into this system are expected to be logged out to the console. To start off, we define the messages for this system:

case object Log
case class LogMessage(when: Date, level: String, text: String)
case class LogString(message: String)
case class LogException(e: Exception)

Using scala’s case classes we can clean up the definition of these log messages. We have a message that will do general logging LogMessage, one that will log a string in LogString and one that will dissect and log out an exception object LogException.

Actor Logic

We now focus on the logic required to log information out from our actor. This is really quite simple; we’re just going to push everything out to the console:

class LogActor extends Actor {

  def receive = {
    case LogMessage(when, level, text) => println(String.format("%s [%s] %s", when.toString(), level, text))
    case LogString(message) => self ! LogMessage(new Date, "info", message)
    case LogException(e) => self ! LogMessage(new Date, "error", e.toString())
  }

}

The receive method is just a big pattern matching statement. Each of the message types are handled in here. Note how LogString and LogException send messages to self. self is a built-in, given to us representing this actor. All we’re doing is just on-forwarding the message in the string and exception cases.

Creating a system

We have actors; we have messages to pass between the actors; we now need a system that the actors will participate in.

// create the system
val system = ActorSystem("myLoggingSystem")

// create an actor
val logger = system.actorOf(Props[LogActor], "logger")

Using the tell and ask methods, we can send and send/receive messages to/from this actor. We also can create a logic-less actor that just acts as a message sender/receiver:

val inbox = Inbox.create(system)

Mailboxes are an important abstraction; they hold messages for actors. Each actor has its own mailbox, but we’ve created one above attached to a system that we can pipe messages into:

inbox.send(logger, LogString("This is the first line of log"))
inbox.send(logger, LogException(new Exception("DOH!")))

Lots of Actors

A slightly more complex topic is to create a pool of actors. In this next snippet, we’ll create a RoundRobinPool.

val actors = system.actorOf(Props[LogActor].withRouter(RoundRobinPool(5)), name = "LoggingActors")

Now that we’ve created a pool, it’s time to smash!

Range(1, 1000000).map(i => actors ! LogString(String.format("Message number %s", i.toString())))

Scheduled

Finally, we can schedule these messages to be sent . . as if they were sent from no where using the actor system that we’d created earlier:

system.scheduler.schedule(0.seconds, 1.second, actors, LogString("Yerrr!"))(system.dispatcher, Actor.noSender)

This will send a LogString message to the actor system actors after zero seconds and then another message every second there after.

Bash Gems

The Bourne Again SHell is one of the most widely deployed shell for Linux that I use all the time. In today’s post, I’m going to collate a lot of the gems that I’d discovered in my travels of using this software.

Finding Help

Nothing can substitute the reference manual materials distributed with this software when it’s installed. At the console, you can read documentation in info format on bash using the following:

info bash

You’re able to deduce executing this command by doing some research at the console, by yourself. Using apropos (which searches the manual pages) you can look for key words.

If you wanted to find any command that begins with the characters ‘ls’ in an attempt to find the command ls, you can perform the following search:

apropos ls | grep '^ls.*'

On my system here, this function emits the following result:

ls (1)               - list directory contents
lsattr (1)           - list file attributes on a Linux second extended file s...
lsb_release (1)      - print distribution-specific information
lsblk (8)            - list block devices
lscpu (1)            - display information about the CPU architecture
lsdiff (1)           - show which files are modified by a patch
lsearch (3)          - linear search of an array
lseek (2)            - reposition read/write file offset
lseek64 (3)          - reposition 64-bit read/write file offset
lshw (1)             - list hardware
lsinitramfs (8)      - list content of an initramfs image
lslocks (8)          - list local system locks
lsmod (8)            - Show the status of modules in the Linux Kernel
lsof (8)             - list open files
lspci (8)            - list all PCI devices
lspcmcia (8)         - display extended PCMCIA debugging information
lspgpot (1)          - extracts the ownertrust values from PGP keyrings and l...
lstat (2)            - get file status
lstat64 (2)          - get file status
lsusb (8)            - list USB devices

We’re only interested in the first item there, but we’re given all of the options. We can now display the manual page with the following:

man ls 1

Variables

Variable creation is fairly straight forward:

# stores the string "John" in var1
var1="John"
# stores the text output of the command 'ls' into var2
var2=`ls -al`

# simple string replacement
var3=${var1/h/a}
# sub-string (turns "John" into "Jo")
var4=${var1:0:2}
# default string substitution (where null)
var6=${var5:-"Value for var5 was not supplied"}

# string interpolation is achieved with $
echo "His name is $var1"

Special variables exist to tell the developer a little bit about their environment:

Variable Description
$? Return code from the last program that just ran
$$ Currently executing script’s PID
$# Number of arguments passed to this script (argc)
$@ All arguments passed to this script
$1 $2 Each argument passed to the script ($3, $4, etc.)

Functions

# define a function
function syntax() {
  echo "usage: prog.sh [options]"
  return 0
}

function print_name() {
  echo "Hello $1"
  return 0
}

# call the function
syntax
print_name "John"

Control Flow Constructs

# Conditionals
if [ var1 == 10 ] then
  echo "It was 10"
else
  echo "It was not 10"
fi

case "$var1" in
  0) echo "Value was zero";;
  1) echo "Value was one";;
  *) echo "Anything but null";;
esac 

# Repetition
for var1 in {1..10} do
done

for ((x=1; x <= 10; x++)) do
done

while [ var1 == 10 ] do
done

Redirection

Special file descriptors of 0 as /dev/stdin, 1 as /dev/stdout and 2 as /dev/stderr.

From the redirections section in the bash manual:

Note that the order of redirections is significant. For example, the command

ls > dirlist 2>&1

directs both standard output (file descriptor 1) and standard error (file descriptor 2) to the file dirlist, while the command

ls 2>&1 > dirlist

directs only the standard output to file dirlist, because the standard error was made a copy of the standard output before the standard output was redirected to dirlist.

JDBC

JDBC (Java Database Connectivity) is a general purpose data access library baked into the Java development and runtime. This library’s purpose is to lower the level of complexity in connecting to different database vendors providing a consistent interface no matter what database you’re connecting to.

In today’s post, I’ll go through the basics of using this library. This blog post will be in context of connecting to a PostgreSQL database.

Drivers

JDBC is based on the premise of drivers. The driver code itself is what fills in the architecture with an implementation that your applications will use. To enumerate all of the drivers, currently in context of your application you can use the following:

Enumeration drivers = DriverManager.getDrivers();

while (drivers.hasMoreElements()) {
  Driver driver = (Driver) drivers.nextElement();
  System.out.println(driver.getClass());
}

I use the term “in context” because whilst you may have the required JAR installed on your system which will be a particular database vendor’s connection library for JDBC, you’ll need to make sure that it’s available on your class path.

For my example, I only have Postgres available to me:

class org.postgresql.Driver

The driver string that you saw in the section above plays an important role in establishing a connection to your database. Before you can start to work with Connection, Statement and ResultSet objects you first need to load in your vendor’s library implementation.

Class.forName("org.postgresql.Driver");

This will reflect your driver into your application ready for use.

Making a connection

To establish a connection with a database, you’ll need to specify a connection string with all of the attributes required to direct your application to the database.

JDBC has a uniform format for specifying its connections with each vendor. Postgres conncetions are no different.

A connection is established using the DriverManager class like so.

Connection connection = DriverManager.getConnection("jdbc:postgresql://localhost:5432/crumbs?user=postgres&password=password");

Running queries

Running retrieves on your database normally comprises of three processes:

  • Preparing a statement to run
  • Executing the statement
  • Enumerating the results

The preparation of the statement is fairly straight forward. The createStatement method on the Connection object will allow you to create an empty statement, whereas prepareStatement will allow you to provide some SQL directly.

// prepare the statement 
Statement retrieveStatement = connection.createStatement();

// execute the statement
ResultSet streetTypes = retrieveStatement.executeQuery("SELECT * FROM \"StreetType\"");

// enumerate the result
while (streetTypes.next()) {
  int id = streetTypes.getInt(streetTypes.findColumn("ID"));
  String name = streetTypes.getString(streetTypes.findColumn("Name"));
  
  System.out.println(String.format("ID: %d, Name: %s\n", id, name));
}

A slightly more complex example where you’d pass in some parameters into your statement might look like this:

PreparedStatement retrieveStatement = 
  connection.prepareStatement("SELECT * FROM \"StreetType\" WHERE \"ID\" > ?");

retrieveStatement.setInt(1, 10);
ResultSet streetTypes = retrieveStatement.executeQuery();

Enumerating a ResultSet object can be achieved with a simple while loop:

while (streetTypes.next()) {
  int id = streetTypes.getInt(streetTypes.findColumn("ID"));
  String name = streetTypes.getString(streetTypes.findColumn("Name"));
  
  System.out.println(String.format("ID: %d, Name: %s\n", id, name));
}

Cleaning up

Finally, all objects should be cleaned up afterwards by using the close functions provided.

streetTypes.close();
retrieveStatement.close();
connection.close();

Other topics

This blog post is just enough to get up and running. There are plenty more complex topics inside of JDBC to be learned:

H2

H2 is a relational database written entirely in Java. It has an extremely small footprint and has an in-memory mode making it an excellent choice for embedded applications.

In today’s post, I’ll take you through using the H2 shell.

Shell

Once you’ve downloaded H2 from their site, you can get a database created and running using the shell. You can invoke the shell with the following command:

java -cp h2-1.4.190.jar org.h2.tools.Shell -url jdbc:h2:~/testdb

I’m using version 1.4.190 here. The -url command line directs us to the file of the database that we’ll create/open.

Once the shell is running, you’re presented with a sql> prompt. You can start creating your table definitions. The documentation on the website is quite extensive with the supported sql grammar, functions and data types.

Further development

Now that you’ve created a database, you can write java applications using JDBC to run queries against your H2 database.