Create a systemd daemon

Like it or hate it, you’ll find systemd on any modern linux system these days. This management subsystem is the newer replacement for the older init scripts-based systems.

In today’s article, I’m going to show you how to create a daemon that will sit under the management of systemd.

.service file

A service file instructs systemd how to interact with the deployed application. It’s referred to as the service unit configuration. The following is an example:

[Unit]
Description=My Service
After=network.target

[Service]
ExecStart=/usr/bin/my-service
Restart=on-failure

[Install]
WantedBy=multi-user.target

Installation

This file, once you’ve created it gets deployed to systemd with a cp. You also need to notify systemd.

sudo cp my-service.service /lib/systemd/system

sudo systemctl daemon-reload
sudo systemctl enable my-service

Interacting with the service

Once the service is in place, you can start to interact with the daemon using systemctl.

sudo systemctl start my-service
sudo systemctl status my-service

You can also get a hold of any log pushed out to the standard system file descriptors:

journalctl --unit my-service --follow

Removal

Once you no longer need your service, you can remove it simply my rm‘ing the .service file. Of course, you’ll need to disable your service first.

That’s it for creating a systemd service.

vim Tips

Getting the most out of vim is really about committing commonly used key stokes/patterns into muscle memory. Once you have the basics of these movements, your text editing experience becomes a lot more optimised and efficient.

This blog post is just an index of useful keystrokes that I use.

Misc. stuff

Macros

Adding text

Buffer management

Navigation

Split management

NERDTree

Executing shellcode in C

A fairly common technique when welding chunks of executable code together is to have the ability to flexibly execute that code. In today’s article, I’ll take you through extracting the shellcode for a function that you write along with the hosting C code that will execute the shellcode for you.

Starting with a C function

First up, we’ll create some shellcode to execute. This is just going to be a very simple function and we’ll compile-only; no linking. Once we have an object file, we’ll use objdump to view the dissassembly and extract the shell code.

Here’s the function.

int foo() {
  return 10;
}

Not the most exciting; but we should take note of the function’s signature at this point. We’re going to create a pointer that points to a function of this signature when we go to execute this code.

For reference, a function pointer that points to a function of this signature would be int (*f)().

Compile, and dissassemble.

gcc -c -03 ex.c
objdump ex.o -d

The shellcode

The output of this function will look something similar to this:

ex.o:     file format elf64-x86-64


Disassembly of section .text:

0000000000000000 <foo>:
   0:	b8 0a 00 00 00       	mov    $0xa,%eax
   5:	c3                   	retq

objdump gives us the dissassembly to the right-hand side; but the hex breakdown to the left is what we’re interested in. Concatenating all of these bytes together, we’re given b80a000000c3. This is our shell code!

Execution

This is all pretty simple.

• Setup the shellcode in an array
• Create a function pointer that points to the shellcode
• Execute that function

#include <stdio.h>

unsigned char code[] = "\xb8\x0a\x00\x00\x00\xc3";

int main(int argc, char **argv) {
  int foo_value = 0;

  int (*foo)() = (int(*)())code;
  foo_value = foo();

  printf("%d\n", foo_value);
}

The variable code holds the shellcode, and you can see it in string form there. We’ve just taken the objdump output and prepared it for a unsigned char string.

We create the function pointer called foo to point to that block of code. Execute the code, grabbing the return value and then print it to screen.

The only tricky part about this is compiling the host program. We need to specify two switches to allow our binary to run.

-z execstack and --fno-stack-protector. Without these our code will just segfault.

gcc -fno-stack-protector -z execstack test.c -o test

This code runs as expected now, pushing a 10 out the STDOUT through printf.

RPC for your Python code

gRPC is an RPC framework from Google that simplifies standing your application up for remote access.

In today’s article, we’ll build a remote calculator.

Prepare your system

Before we begin, you’ll need a couple of packages to assist in creating this project.

Both grpcio and grpcio-tools can be installed with the following:

pip install grpcio
pip install grpcio-tools

Create your definition

Before we begin, we really need a clear idea on how our service will look. This involves creating a contract which will detail the data structures and service definitions that will be utilised between system actors.

To do this, we’ll use a proto file (in the protobuf format) which we’ll use to generate our contract code.

In our application we can add, subtract, multiply and divide. This is a stateful service, so we’ll be creating sessions to conduct calculations in. A create method will create a session, where as the answer method will tear our session down, emitting the result.

syntax = "proto3";

message Number {
  float value = 1;
}

message SessionOperation {
  string token = 1;
  float value = 2;
}

service Calculator {
  rpc Create(Number) returns (SessionOperation) { }
  rpc Answer(SessionOperation) returns (Number) { }

  rpc Add(SessionOperation) returns (Number) { }
  rpc Subtract(SessionOperation) returns (Number) { }
  rpc Multiply(SessionOperation) returns (Number) { }
  rpc Divide(SessionOperation) returns (Number) { }
}

Running this file through grpc_tools with the following command:

python -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. calc.proto

We’re now left with two automatically generated files, calc_pb2_grpc.py and calc_pb2.py. These files hold the foundations of value mashalling and service definition for us.

Implementing the server

Now that we’ve generated some stubs to get our server running, we need to supply the implementation itself. A class CalculatorServicer amongst other artifacts were generated for us. We derive this class to supply our functions out.

class CalculatorServicer(calc_pb2_grpc.CalculatorServicer):

    def Create(self, request, context):
        serial = str(uuid.uuid4())
        calc_db[serial] = request.value

        response = calc_pb2.SessionOperation()
        response.token = serial
        response.value = calc_db[serial]

        return response

Here’s the Create implementation. You can see that it’s just reserving a piece of the calc_db dictionary, and storing the initial value.

request is in the shape of the message that we defined for this service. In the case of Create the input message is in the type of Number. You can see that the value attribute is being accessed.

The remainder of the implementation are the arithmetic operations along with the session closure:

    def Answer(self, request, context):
        serial = request.token

        response = calc_pb2.Number()
        response.value = calc_db[serial]

        calc_db[serial] = None

        return response

    def Add(self, request, context):
        value = request.value
        serial = request.token

        calc_db[serial] = calc_db[serial] + value        

        response = calc_pb2.Number()
        response.value = calc_db[serial]
        return response

    def Subtract(self, request, context):
        value = request.value
        serial = request.token

        calc_db[serial] = calc_db[serial] - value        

        response = calc_pb2.Number()
        response.value = calc_db[serial]
        return response

    def Multiply(self, request, context):
        value = request.value
        serial = request.token

        calc_db[serial] = calc_db[serial] * value        

        response = calc_pb2.Number()
        response.value = calc_db[serial]
        return response

    def Divide(self, request, context):
        value = request.value
        serial = request.token

        calc_db[serial] = calc_db[serial] / value        

        response = calc_pb2.Number()
        response.value = calc_db[serial]
        return response

Finally, we need to start accepting connections.

Standing the server up

The following code sets up the calculator.

server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
calc_pb2_grpc.add_CalculatorServicer_to_server(CalculatorServicer(), server)

print('Starting server. Listening on port 3000.')
server.add_insecure_port('[::]:3000')
server.start()

try:
    while True:
        time.sleep(10000)
except KeyboardInterrupt:
    server.stop(0)

Invoking the code

Now, we’ll create a client to invoke these services.

import grpc

import calc_pb2
import calc_pb2_grpc

channel = grpc.insecure_channel('localhost:3000')

stub = calc_pb2_grpc.CalculatorStub(channel)
initial = calc_pb2.Number(value=0)

session = stub.Create(initial)
print 'Session is ' + session.token

stub.Add(calc_pb2.SessionOperation(token=session.token, value=5))
stub.Subtract(calc_pb2.SessionOperation(token=session.token, value=3))
stub.Multiply(calc_pb2.SessionOperation(token=session.token, value=10))
stub.Divide(calc_pb2.SessionOperation(token=session.token, value=2))

answer = stub.Answer(calc_pb2.SessionOperation(token=session.token, value=0))
print 'Answer is ' + str(answer.value)

So, we’re setting up a session with a value of 0. We then . .

• Add 5
• Subtract 3
• Multiply by 10
• Divide by 2

We should end up with 10.

➜  remote-calc python calc_client.py
Session is 167aa460-6d14-4ecc-a729-3afb1b99714e
Answer is 10.0

Wrapping up

This is a really simple, trivial, well studied (contrived) example of how you’d use this technology. It does demonstrate the ability to offer your python code remotely.

Clojure threading macros

A Threading Macro in Clojure is a utility for representing nested function calls in a linear fashion.

Simple transformations

Meet mick.

user=> (def mick {:name "Mick" :age 25})
#'user/mick

He’s our subject for today.

If we wanted to give mick an :occupation, we could simply do this using assoc; like so:

user=> (assoc mick :occupation "Painter")
{:name "Mick", :age 25, :occupation "Painter"}

At the same time, we also want to take note of his earning for the year:

user=> (assoc mick :occupation "Painter" :ytd 0)
{:name "Mick", :age 25, :occupation "Painter", :ytd 0}

Keeping in mind that this isn’t actually changing mick at all. It’s just associating new pairs to him, and returning the new object.

mick got paid, $100 the other week, so we increment his :ytd by 100. We do this by performing the transformation after we’ve given him the attribute.

user=> (update (assoc mick :occupation "Painter" :ytd 0) :ytd + 100)
{:name "Mick", :age 25, :occupation "Painter", :ytd 100}

He earned another $32 as well, in another job.

user=> (update (update (assoc mick :occupation "Painter" :ytd 0) :ytd + 100) :ytd + 32)
{:name "Mick", :age 25, :occupation "Painter", :ytd 132}

He also got a dog.

user=>  (assoc (update (update (assoc mick :occupation "Painter" :ytd 0) :ytd + 100) :ytd + 32) :pets [:dog])
{:name "Mick", :age 25, :occupation "Painter", :ytd 132, :pets [:dog]}

So, the nesting gets out of control. Quickly.

Thread first macro

We’ll use -> (The thread-first macro) to perform all of these actions in one form (must as we’ve done above), but in a much more readable manner.

user=> (-> mick
  #_=>   (assoc :occupation "Painter" :ytd 0)
  #_=>   (update :ytd + 100)
  #_=>   (update :ytd + 32)
  #_=>   (assoc :pets [:dog]))
{:name "Mick", :age 25, :occupation "Painter", :ytd 132, :pets [:dog]}  

So, it’s the same result; but with a much cleaner and easier to read interface.

Thread last macro

We saw above that the -> threading macro works well for bare values being passed to forms. When the problem changes to the value not being supplied in the initial position, we use thread last ->>. The value that we’re threading appears as the last item in each of the transformations, rather than the mick example where they were the first.

user=> (filter #(> % 12) (map #(* % 5) [1 2 3 4 5]))
(15 20 25)

We multiply the elements of the vector [1 2 3 4 5] by 5 and then filter out those items that are greater than 12.

Again, nesting quickly takes over here; but we can express this with ->>:

user=> (->> [1 2 3 4 5]
  #_=>   (map #(* % 5) ,,,)
  #_=>   (filter #(> % 12) ,,,))
(15 20 25)

Again, this is a much more readable form.

as

If the insertion point of the threaded value varies, we can use as-> to alias the value.

user=> (as-> "Mick" n
  #_=>   (clojure.string/upper-case n)
  #_=>   (clojure.string/reverse n)
  #_=>   (.substring n 1))
"CIM"

Take the name “Mick”

• Convert it to upper case
• Reverse it
• Substring, skipping the first character

It’s the substring call, which takes the string in the initial position that’s interesting here; as it’s the only call that does that. upper-case and reverse take it in as the only (or last).

some

The two macros some-> and some->> work like their -> and ->> counterparts; only they do it on Java interop methods.

cond

cond-> and cond->> will evaluate a set of conditions, applying the threaded value to the front to back of any expression associated to a condition that evaulates true.

The following example has been taken from here.

(defn describe-number [n]
  (cond-> []
    (odd? n) (conj "odd")
    (even? n) (conj "even")
    (zero? n) (conj "zero")
    (pos? n) (conj "positive")))

So you can describe a number as you go:

user=> (describe-number 1)
["odd" "positive"]
user=> (describe-number 5)
["odd" "positive"]
user=> (describe-number 4)
["even" "positive"]
user=> (describe-number -4)
["even"]