Loading dynamic libraries in C

21 Nov 2016

Today’s post is going to be a quick demonstration of the dynamic library loading available through Glibc.

Some really important links that shouldn’t be glossed over if you’re serious about some dynamic library development are:

Simple library

To start, we’re going to write a tiny library. It’ll have one function it it called greet that will send out a string:

char *greeting = "Hello";

char *greet(void) {
  return greeting;
}

We can make libtest.so out of this with the following:

gcc -c -Wall -fPIC greet.c -o greet.o
gcc --shared greet.o -o libtest.so

We now have libtest.so as our shared library, ready to be loaded by our host program.

Host program

The executable that takes care of loading this shared library, engaging the functions within it and executing the code will be called the host in this instance. First up, we’ll use dlopen to load the shared library off of disk:

void *test_lib = dlopen(LIBTEST_SO, RTLD_LAZY);

if (!test_lib) {
  fprintf(stderr, "%s\n", dlerror());
  exit(EXIT_FAILURE);
}

Now that we’ve opened the library up, we’ll use dlsym to bury into the library and extract the greet function:

char* (*greet)(void);

greet = (char * (*)(void)) dlsym(test_lib, "greet");

if ((error = dlerror()) != NULL) {
  fprintf(stderr, "%s\n", error);
  exit(EXIT_FAILURE);
}

We’re referencing the function now. Notice the goofy cast: (char * (*)(void)). Here’s a blurb from the manpage:

/* According to the ISO C standard, casting between function pointers and ‘void *’, as done above, produces undefined results. POSIX.1-2003 and POSIX.1-2008 accepted this state of affairs and proposed the following workaround:
  *(void **) (&cosine) = dlsym(handle, "cos");
This (clumsy) cast conforms with the ISO C standard and will avoid any compiler warnings.

The 2013 Technical Corrigendum to POSIX.1-2008 (a.k.a. POSIX.1-2013) improved matters by requiring that conforming implementations support casting ‘void *’ to a function pointer. Nevertheless, some compilers (e.g., gcc with the ‘-pedantic’ option) may complain about the cast used in this program. */

Now we can call the greeter, and clean up with dlclose!

printf("%s\n", greet());

dlclose(test_lib);
exit(EXIT_SUCCESS);

Because we do the dynamic loading of the library inside of our application, we don’t need to tell the compiler of the library’s existence. The host application will need to know about Glibc’s dl library though:

gcc -Wall host.c -ldl -o host

In closing

This has been a really quick lap around the dl library. The working prototype is crude, but forms the skeletal basis of a plugin-architecture should you be able to establish a strong contract between the pieces of library code and the host!

dblink

21 Nov 2016

There are a few tools at a developers disposal to perform queries that go cross-database. In today’s post, I’ll quickly go over using dblink to establish links between Postgres databases.

Example Usage

First up, we need to make sure that the dblink extension is available to our server. CREATE EXTENSION is what we’ll use to do this:

CREATE EXTENSION dblink;

Prior to being able to query against a remote database, we need to use dblink_connect to establish a link from the local context.

-- create the crumbs link
select  dblink_connect(
    'remotedb',
    'host=127.0.0.1 port=5432 dbname=remotedb user=postgres password=password'
);

The connection string that you supply are fairly straight forward details to connect to a server with given credentials.

Using dblink, you can now invoke a query on the remote server and have the result mixed into your local code.

select  *
from    dblink('remotedb', 'SELECT "ID", "Name" FROM "People"')
as      people("ID" int4, "Name" character varying);

When you’re done with the connection, you use dblink_disconnect.

select dblink_disconnect('dbl-crumbs');

Async Queries

dblink also gives you the opportunity to perform async queries which is really handy. You kick the query off, do something and then start fetching the results later on in your code.

/* start the query off */
select  *
from    dblink_send_query('remotedb', 'SELECT "ID", "Name" FROM "People"')
as      people;

/* Do some other work here */

/* start drawing the results */
select  *
from    dblink_get_result('remotedb')
as      people("ID" int4, "Name" character varying);

That’s a bit fancy.

Using nginx as a proxy

17 Nov 2016

When running applications in docker containers, it can make sense to put a proxy server in front. It’s relatively simple to setup an nginx server to sit in front of any application which I’ll demonstrate in this article.

Configuration

In order to get started, we’ll use the nginx image hosted up on dockerhub. This particular image allows us to specify a configuration file to a web server relatively simply.

To setup the scenario, we have a node.js application running on port 3000 of the host machine that we’d look to proxy through the nginx proxy. Here’s how the configuration would look, over port 80:

server {
  listen 80;
  index index.html;

  server_name localhost;

  error_log /var/log/nginx/error.log;
  access_log /var/log/nginx/access.log;
  root /var/www/public;

  location ~* /my-api {
    rewrite /my-api(.*) /$1 break;
    proxy_pass https://172.17.0.1:4010;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
  }

}

There’s even a rewrite here that takes the my-api part of the original request URI out of the forwarded request, so that the node.js application can be treated directly off the root.

Start me up!

To now get this started, we need to sub-in this configuration file as if it were part of the running container.

docker run -ti --rm -v $(pwd)/default.conf:/etc/nginx/conf.d/default.conf -p 80:80 nginx

Security

Yep. Now we need to use SSL and put the application over 443! First up, let’s create a self-signed certificate using OpenSSL.

openssl req -x509 -nodes -days 3652 -newkey rsa:2048 -keyout nginx.key -out nginx.crt

Now that we’ve got our certificate nginx.crt and key nginx.key, we can change the configuration now to proxy our application securely:

server {
  listen 80;
  listen 443 ssl;
  index index.html;

  server_name localhost;
  ssl_certificate /etc/nginx/ssl/nginx.crt;
  ssl_certificate_key /etc/nginx/ssl/nginx.key;

  error_log /var/log/nginx/error.log;
  access_log /var/log/nginx/access.log;
  root /var/www/public;

  location ~* /my-api {
    rewrite /my-api(.*) /$1 break;
    proxy_pass https://172.17.0.1:4010;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
  }

}

Now when we start up the container, we not only need to expose 443 for SSL, but we’ll also volume-in our certificate and key:

docker run    \
       -ti    \
       --rm   \
       -v $(pwd)/default.conf:/etc/nginx/conf.d/default.conf \
       -v $(pwd)/ssl:/etc/nginx/ssl \
       -p 443:443 \
       nginx

Now you can proxy your other dockerized web-applications through nginx without much hassle at all.

Compojure

17 Nov 2016

In a previous post, we setup a really simple route and server executing some Clojure code for us. In today’s post, we’re going to use a library called Compojure to fancy-up a little bit of that route definition.

This should make defining our web application a bit more fun, anyway.

Getting started

Again, we’ll use Leiningen to kick our project off:

lein new webapp-1

We’re going to add some dependencies to the project.clj folder for compojure and http-kit. http-kit is the server that we’ll be using today.

(defproject webapp-1 "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.8.0"]
           [compojure "1.1.8"]
           [http-kit "2.1.16"]])

And then, installation.

lein deps

Hello!

To get started, we’ll define a root route to greet us.

(ns webapp-1.core
  (:require [compojure.core :refer :all]
        [org.httpkit.server :refer [run-server]]))

(defroutes greeter-app
  (GET "/" [] "Hello!"))

(defn -main []
  (run-server greeter-app {:port 3000}))

A quick hit through curl lets us know that we’re up and running:

curl --verbose localhost:3000
* Rebuilt URL to: localhost:3000/
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 3000 (#0)
> GET / HTTP/1.1
> Host: localhost:3000
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
< Content-Length: 6
< Server: http-kit
< Date: Thu, 17 Nov 2016 02:42:48 GMT
< 
* Connection #0 to host localhost left intact
Hello!

Further links

Check out the following:

Good fun.

Simple web server with Clojure

15 Nov 2016

In today’s post, we’re going to use the Clojure HTTP server abstraction called ring to stand a web server up, and attach some some routes. This allows us to expose our Clojure functions over the web in a relatively simple fashion.

Getting started

This blog post is mainly centered around the getting started guide from the ring documentation pages, found here.

We’re going to get started by creating a project using lein.

lein new jetty-test

After this process finishes, you’ll end up with a directory called jetty-test that has a project structure something like this:

.
├── CHANGELOG.md
├── doc
│   └── intro.md
├── LICENSE
├── project.clj
├── README.md
├── resources
├── src
│   └── jetty_test
│       └── core.clj
└── test
    └── jetty_test
        └── core_test.clj

Dependencies

Now we need to make our newly created project depend on ring. We need to add references to ring-core and ring-jetty-adapter in the project.clj file. So it should read something like this:

(defproject jetty-test "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.8.0"]
           [ring/ring-core "1.5.0"]
           [ring/ring-jetty-adapter "1.5.0"]])

We can now install these dependencies into the project.

lein deps

Server code

We can start writing our route code now that the server will respond to. We’ll define a function that simply returns the current date and time:

(defn now [] (java.util.Date.))

We’ll also create a route that will use this function, and send back the text each time the route is requested:

(defn current-time [request]
  {:status 200
   :headers {"Content-Type" "text/plain"}
   :body (str (now))})

That’s it for the server code. We still need to fire up Jetty and attach the handler to it. We need to import ring.adapter.jetty as it contains run-jetty for us:

(use 'ring.adapter.jetty)

(run-jetty current-time {:port 3000})

Running

We run our project using lein:

lein run

Our output now looks something like this:

. . .
. . .
. . .

Retrieving clj-time/clj-time/0.11.0/clj-time-0.11.0.jar from clojars
Retrieving ring/ring-core/1.5.0/ring-core-1.5.0.jar from clojars
Retrieving ring/ring-servlet/1.5.0/ring-servlet-1.5.0.jar from clojars
Retrieving clojure-complete/clojure-complete/0.2.4/clojure-complete-0.2.4.jar from clojars
2016-11-15 22:34:11.551:INFO::main: Logging initialized @877ms
2016-11-15 22:34:11.620:INFO:oejs.Server:main: jetty-9.2.10.v20150310
2016-11-15 22:34:11.646:INFO:oejs.ServerConnector:main: Started ServerConnector@795f253{HTTP/1.1}{0.0.0.0:3000}
2016-11-15 22:34:11.647:INFO:oejs.Server:main: Started @973ms

. . . suggesting that our server is ready to take requests. We can use curl to test it out for us:

$ curl -v http://localhost:3000/

*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 3000 (#0)
> GET / HTTP/1.1
> Host: localhost:3000
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Tue, 15 Nov 2016 22:36:34 GMT
< Content-Type: text/plain; charset=ISO-8859-1
< Content-Length: 28
< Server: Jetty(9.2.10.v20150310)
< 
* Connection #0 to host localhost left intact
Tue Nov 15 22:36:34 UTC 2016

That’s it. Pretty simple.

Older Newer

Cogs and Levers A blog full of technical stuff

Loading dynamic libraries in C

Simple library

Host program

In closing

dblink

Example Usage

Async Queries

Using nginx as a proxy

Configuration

Start me up!

Security

Compojure

Getting started

Hello!

Further links

Simple web server with Clojure

Getting started

Dependencies

Server code

Running