Cogs and Levers A blog full of technical stuff

RAII for C++

Introduction

In programming, RAII stands for “Resource Acquisition is Initialization” and it’s an idiom or technique established by Bjarne Stroustrup to ease resource allocation and deallocation in C++.

Common problems have been when an exception is thrown during initialization, any memory associated during construction (or underlying resources) aren’t released, creating memory leaks in applications.

The Idea

The basic premise is that resource allocation is to be performed in the constructor of your class. Release of the resources occurs in your destructor. The example given on the Wikipedia page deals with holding a lock/mutex for a given file. When execution leaves the scope of the code (whether it be from premature termination of an exception or from the code naturally exiting), the destructors run to release the file handle and lock.

The concept is a great way to not only clean up your code (as all of the “if !null” code is now redundant) but it’s a great safe-guard that you can almost be absent minded about.

It’s important to note that this idiom doesn’t allow you to ignore good exception handling practice. You’re still expected to use exception handling in your code, this will just ensure that your cleanup/release code is executed as expected.

An Implementation

Implementing this idea into your own code is really quite simple. If you have a resource (handle) that you’re managing manually, wrap it in a class.

  • Ensure the constructor takes the handle in
  • Release the handle in the destructor

When working with OpenGL textures, I use a very small class that allows me to handle the resource cleanup, it just managed the generated texture id. When the class falls out of scope or there’s a failure during initialization, the texture is cleaned up.

class texture {
  public:
    // manage the generated texture id
    texture(const GLuint t) : _reference(t) { }
    
    // cleanup of the allocated resource
    virtual ~texture(void);

    // provide access to the reference
    const GLuint reference() const { return _reference; }

  private:
    GLuint _reference;
};

texture::~texture(void) {
  // only run if we have something to clean up
  if (this->_reference != 0) {
    // clear out the texture  
    glDeleteTextures(1, &this->_reference);
    this->_reference = 0;
  }
}

Strictly speaking, the constructor should probably do the generation of the texture itself. Where I’m loading the texture is in another managed object of itself. Most importantly, if an exception is thrown during initialization, this class will remove anything allocated to it (if it did allocate).

It should be mentioned that there are lots of extra attributes we can pile into RAII style classes. There’s a really good write up (in depth) here.

Conclusion

RAII is a great idea to implement into your own classes. The more of this you can practice, the more exception-safe your code will become . . . from car accidents.

Custom Installs with Checkinstall

Introduction

Sometimes it’s necessary to install a piece of software from source. This is normally an awkward process when you’ve had a package manager taking care of all your software needs when you’re faced with the proposition of installing something that the package manager is unaware of. Another concern is that some software developers don’t do the simple things well - some makefiles won’t even offer you the ability to uninstall a piece of software leaving you to try to remove the files that have been peppered into your system directories.

In today’s post, I’ll walk through a sample usage of the application CheckInstall for Debian based Linux distributions.

What is it?

From the CheckInstall page on the Debian Wiki:

CheckInstall keeps track of all the files created or modified by your installation script (“make” “make install” “make install_modules”, “setup”, etc), builds a standard binary package and installs it in your system giving you the ability to uninstall it with your distribution’s standard package management utilities.

The thing I love about the Debian distribution is the stability of the packages in its repositories. Sometimes, it’s also what I’m not to fond of as software vendors are bringing out new versions of their software and they don’t make it into the stable repositories until they’re deemed stable (which takes ages!) or they may never make it into the repositories.

Using CheckInstall

I’ve used CheckInstall for quite a few packages in the past. Just recently, I’ve used it to manage the installation of SDL2 onto my system.

# extract your source package
$ tar -zxvf SDL2-2.0.1.tar.gz

# configure and build as usual
$ cd SDL2-2.0.1
$ ./configure
$ make

# use checkinstall on the installation step
$ sudo checkinstall make install

After this process had finished, a deb file was created for me which represented the files that had been installed on the system and the deb itself had been applied to the system.

The idea here is that it simplifies re-installation and removal by proxy of the generated deb package.

Hadoop Links

Apache Hadoop NextGen MapReduce (YARN)

File System Shell Guide

YAHOO! Hadoop Tutorial

Hadoop 2 (2.2.0) setup on Debian

Today’s post will just be a walk through of the steps required to install Hadoop 2 on Debian Linux. Please note that this is for a single node installation only. This guide is heavily based on the Ubuntu instructions found here.

Install Java

	
# install the java jdk
$ sudo apt-get install openjdk-7-jdk
 
# make a jdk symlink
$ cd /usr/lib/jvm
$ ln -s java-7-openjdk-amd64 jdk
 
# make sure that ssh server is installed
$ sudo apt-get install openssh-server

Add Hadoop Users and Groups

	
# create a new group for hadoop
$ sudo addgroup hadoop
 
# create the hduser and put them in the hadoop group
$ sudo adduser --ingroup hadoop hduser
 
# add them to the sudo group also
$ sudo adduser hduser sudo

Now login as “hduser”.

SSH Certificates

	
# generate your key
$ ssh-keygen -t rsa -P ''
 
# set your public key as authorized
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
 
# test out ssh
$ ssh localhost

Download Hadoop

	
# downoad the package
$ cd ~
$ wget http://mirror.rackcentral.com.au/apache/hadoop/common/hadoop-2.2.0/hadoop-2.2.0.tar.gz
 
# extract the package
$ sudo tar vxzf hadoop-2.2.0.tar.gz -C /usr/local
$ cd /usr/local
$ sudo mv hadoop-2.2.0 hadoop
 
# get the hduser to take ownership
$ sudo chown -R hduser:hadoop hadoop

Setup Environment Variables

Add the following lines to your ~/.bashrc

# Hadoop variables
export JAVA_HOME=/usr/lib/jvm/jdk/
export HADOOP_INSTALL=/usr/local/hadoop
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL

Add the following lines to /usr/local/hadoop/etc/hadoop/hadoop-env.sh

# modify JAVA_HOME
export JAVA_HOME=/usr/lib/jvm/jdk/

Re-login to your machine as hduser, and check the hadoop version.

$ hadoop version

Configure Hadoop

Add the following lines into the <configuration> node within /usr/local/hadoop/etc/hadoop/core-site.xml

<property>
   <name>fs.default.name</name>
   <value>hdfs://localhost:9000</value>
</property>

Add the following lines into the <configuration> node within /usr/local/hadoop/etc/hadoop/yarn-site.xml

<property>
   <name>yarn.nodemanager.aux-services</name>
   <value>mapreduce_shuffle</value>
</property>
<property>
   <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
   <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>

Make a copy of the mapred-site template file

	
$ mv mapred-site.xml.template mapred-site.xml
$ vi mapred-site.xml

Add the following lines into the <configuration> node within /usr/local/hadoop/etc/hadoop/mapred-site.xml

<property>
   <name>mapreduce.framework.name</name>
   <value>yarn</value>
</property>

Prepare the Filesystem

	
# create the physical directories
$ cd ~
$ mkdir -p mydata/hdfs/namenode
$ mkdir -p mydata/hdfs/datanode

Add the following lines into the <configuration> node /usr/local/hadoop/etc/hadoop/hdfs-site.xml

<property>
   <name>dfs.replication</name>
   <value>1</value>
 </property>
 <property>
   <name>dfs.namenode.name.dir</name>
   <value>file:/home/hduser/mydata/hdfs/namenode</value>
 </property>
 <property>
   <name>dfs.datanode.data.dir</name>
   <value>file:/home/hduser/mydata/hdfs/datanode</value>
 </property>

Format the namenode

	
$ hdfs namenode -format

Start Hadoop

	
$ start-dfs.sh
$ start-yarn.sh
 
# check that services are running
$ jps

Run the Example

	
$ cd /usr/local/hadoop
$ hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar pi 2 5

Getting started with Lua using C++

Introduction

Lua is a programming language that has seen increased popularity from the game development industry. It’s put to use in the disciplines of providing configuration data all the way through to scripting automated character actions.

In today’s post, I’ll walk you through the setup process of the libraries up to writing your first testing application within a Linux environment.

Library Installation

Before you get downloading and building this library, you’ll just need to ensure that you have a build environment installed and are able to compile.

At the time of writing this article, the Lua website had 5.2.3 as their latest release. From their downloads page, grab the latest tar.gz archive and extract it onto your machine. Following along with their building instructions, issuing “make linux test” whilst in the extracted folder did the trick. It built Lua ready for me to use. A nice convenience of the make file was the “make local” option. Upon issuing this request, the make system will prepare an installation folder that is suitable for you to use locally (i.e. not installed to the system).

When it comes to downloading the latest versions of libraries, I’ll rarely install these to my system. Rather, I drag them around for each project that needs them so that the project determines its dependencies as opposed to my system.

From here, I prepare a distributable directory of development files that I know that each of my project needs. In the case of Lua, I create the following structure:

.
└── lua-5.2.3
    ├── include
    │   ├── lauxlib.h
    │   ├── luaconf.h
    │   ├── lua.h
    │   ├── lua.hpp
    │	  └── lualib.h
    └── lib
    	├── liblua.a
    	└── lua
    		└── 5.2

I have all of the development headers available (under the “include” folder) and a static version of the Lua library (under lib).

Building applications

When building Lua applications, you’ll need to specify the libraries and include folders to your compiler so it knows where to find them. For a test application that I’d written, the following command line compiled an application for me without any trouble.

$ g++ -Ilib/lua-5.2.3/include -Llib/lua-5.2.3/lib/ -llua -ldl

You can see at the end there, mention of both the “lua” and “dl” libraries.

Test application

A very simple test to start will be creating a program that will execute a Lua script, pull out on of its global variables and display it to screen.

Here’s our Lua script:

x = 10

Pretty simple. We have one variable x set to 10. Now here’s the C++ code that we use to read that one variable out and present it to screen.

#include <iostream>
#include <lua.hpp>

int main(int argc, char *argv[]) {
  // create a new lua context to work with
  lua_State *L = luaL_newstate();

  // open any library we may use
  luaL_openlibs(L);

  // read the Lua script off disk and execute it
  if ((luaL_dofile(L, "test.lua")) != 0) {
    
    // handle any errors 
    std::cout << "unable to load test.lua" << std::endl;
    return 1;
    
  }

  // put the value of X at the top of the stack
  lua_getglobal(L, "x");
  
  // interpret the value at the top of the stack 
  // as an integer and put it in the variable "val"
  int val = (int)lua_tointeger(L, -1);
  
  // pop the value of X off the stack
  lua_pop(L, 1);

  // write the value out
  std::cout << "Value of X: " << val << std::endl;

  // finish up with the Lua context
  lua_close(L);

  return 0;

}

I think that the source code above (paired with the documentation on the Lua website) should make things pretty straight forward.

That’s it for today. This is only scratching the surface on what Lua can do. For my purposes right now, I just need to feed configuration values into programs, this fits the purpose really well.