RSocket for streaming applications

Introduction

In a world of changing software requirements, and more demand for results and analytics in realtime, framework and system creators have needed to become smarter with the way that they reason about information ingestion. Streaming frameworks have started to make good ground in establishing themselves as accessible software platforms for many developers.

In today’s article, we’ll explore RSocket very briefly. RSocket’s place is to provide an application protocol that is directly designed for reactive streaming applications.

Core Feature

The core features of implementing this specification are as follows:

• Metadata and Payload frames
• All 4 interaction models : Fire-and-forget, request/response, requestStream, requestChannel
• Request-N frame : application level flow control
• Fragmentation/Reassembly : Drivers are assumed to fully encode/decode the expected user data type
• Keep-Alive frame : A responder receiving a keep-alive frame must reply a keep-alive frame
• Error Frame : in order to fully support connection lifecycle
• Handling the unexpected : If Resumption, Leasing or an extension is not supported, rejected error frames must be used

Example

The following are two code snippets take from the RSocket website.

The sample server:

RSocketFactory.receive()
    .frameDecoder(Frame::retain)
    .acceptor(new PingHandler())
    .transport(TcpServerTransport.create(7878))
    .start()
    .block()
    .onClose();

The sample client:

Mono<RSocket> client =
    RSocketFactory.connect()
        .frameDecoder(Frame::retain)
        .transport(TcpClientTransport.create(7878))
        .start();

PingClient pingClient = new PingClient(client);

Recorder recorder = pingClient.startTracker(Duration.ofSeconds(1));

int count = 1_000;

pingClient
    .startPingPong(count, recorder)
    .doOnTerminate(() -> System.out.println("Sent " + count + " messages."))
    .blockLast();

Perform your shell work in parallel

Introduction

In some cases, breaking your larger programming problems into smaller, parallelizable units makes sense from a time complexity problem. If the work you are trying to perform exhibits some of these parallelizable characteristics, you should only need to wait for the longest of your jobs to finish.

In today’s post, we’ll be talking about GNU Parallel.

A summary from their website:

GNU parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU parallel can then split the input and pipe it into commands in parallel.

Input

The input system is quite complex. Delimiting the inputs with the ::: operator, parallel will make a catersian product out of the input values.

parallel echo ::: John Paul Sally
John
Paul
Sally

parallel echo ::: John Paul Sally ::: Smith
John Smith
Paul Smith
Sally Smith

parallel echo ::: John Paul Sally ::: Smith Jones Brown
John Smith
John Jones
John Brown
Paul Smith
Paul Jones
Paul Brown
Sally Smith
Sally Jones
Sally Brown

Linkage is possible using :::+, should this flexibility be required.

parallel echo ::: John Paul Sally :::+ Smith Jones Brown
John Smith
Paul Jones
Sally Brown

See more about input sources in the tutorial.

curl

For some examples, I’ll use curl.

Let’s get three web pages downloaded:

• google.com
• yahoo.com
• zombo.com

Getting these down, one at a time times in at nearlly 1.2 seconds.

( curl www.google.com && curl www.yahoo.com && curl www.zombo.com; )  0.02s user 0.05s system 5% cpu 1.195 total

Running these downloads in parallel, we take half a second off the time:

parallel curl {1} ::: www.google.com www.yahoo.com www.zombo.com  0.21s user 0.04s system 33% cpu 0.774 total

Summing up

GNU Parallel is a great utility to get multiple things done at once at the shell. Take a look at the tutorial and immediately become more productive.

Mounting windows filesystems in Linux

A quick note on mounting windows filesystems on linux.

Your platform will require cifs-utils.

sudo apt install cifs-utils

From here, you can connect and mount a remote file system.

sudo mount -t cifs //ip.address.of.windows/location/to/mount /mnt -o user=john,password=mypassword,domain=mydomain

Done. You can now access the remote filesystem.

HDF5

In today’s article, we’ll talk a bit about the HDF5 format. From the HDF5 website:

An HDF5 file is a container for two kinds of objects: datasets, which are array-like collections of data, and groups, which are folder-like containers that hold datasets and other groups.

Useful links:

• HDF5 for Python

Text search with grep

GNU Grep is an amazingly powerful tool that will allow you to search input files containing specified patterns.

The following command performs a recursive search for a string, inside a directory of files:

grep -rnw '/path/to/search' -e 'pattern-to-find'

Breaking this command down:

• -r makes the execution recursive
• -n will give you the line number
• -w matches the whole word

You can use the --exclude switch to remove file patterns from the set of files included in your search. Removing js and html files from your search might look like this:

--exclude *.{js,html}

The opposite of this will be the --include switch.

For further details on this really useful tool, check out the man page.