Streams in Java 8

04 Feb 2017

The functional programming paradigm has certainly seen a lot of attention of late, where some of the features that can be exploited from it have properties that assist in scale programming.

In today’s post, I’ll walk through Java 8 streams which allow you to treat collection data structures in a functional way.

What is a stream?

A stream prepares a collection of elements in such a way that you can operate on it functionally. The higher-order functions that you’d commonly see in a functional arrangement are:

An example

To start with, we need to define some test data. We’re going to work with artists and songs:

/** Song.java */
public class Song {
  public String name;
  public int year;
  public int sales;

  public Song(String name, int year, int sales) {
    this.name = name;
    this.year = year;
    this.sales = sales;
  }

  public String getName() {
    return name;
  }

  public int getYear() {
    return year;
  }

  public int getSales() {
    return sales;
  }
}

/* Artist.java */
public class Artist {
  private String name;
  private int age;
  private List<Song> songs;

  public Artist(String name, int age, List<Song> songs) {
    this.name = name;
    this.age = age;
  }

  public String getName() {
    return name;
  }

  public int getAge() {
    return age;
  }

  public List<Song> getSongs() {
    return songs;
  }
}

Now that we have a basic structure of artists and songs, we can define some test data to work with.

Don’t judge me.

Artist rickAstley = new Artist(
  "Rick Astley",
  50,
  Arrays.asList(
    new Song(
      "Never Gonna Give You Up",
      1987,
      500000
    )
  )
);

Artist bonJovi = new Artist(
  "Jon Bon Jovi",
  54,
  Arrays.asList(
    new Song(
      "Livin' on a Prayer",
      1986,
      3400000
    ),
    new Song(
      "Wanted Dead or Alive",
      1987,
      4000000
    )
  )
);

List<Artist> artists = Arrays.asList(bonJovi, rickAstley);

Ok, now that the boilerplate is out of the way; we can start the fun stuff.

Map

map is a function that allows you to apply a function to each element of a list; transforming and returning it. We can grab just the artist’s names with the following:

artists.stream()
  .map(Artist::getName)
  .collect(toList())

toList comes out of the java.util.stream.Collectors class. So you can import static:

import static java.util.stream.Collectors.toList;

Mapping deeper

flatMap allows you to perform the map process on arrays of arrays. Each artist has an array of songs, so we can flat map (at the artist level) to emit a flat list of songs:

artists.stream()
  .flatMap(a -> a.getSongs().stream())
  .map(Song::getName)
  .collect(toList())

Filter

We only want artists over the age of 52.

artists.stream()
  .filter(a -> a.getAge() > 52)
  .map(Artist::getName)
  .collect(toList())

Reduce

We can aggregate all of the sales figures into a single value:

artists.stream()
  .flatMap(a -> a.getSongs().stream())
  .mapToInt(Song::getSales)
  .sum()

Ok, so this has been a whirlwind tour of the Java 8 functional interface.

Implicit values in Scala

16 Jan 2017

Scala has the concept of implicit parameters which allows the developer to implicitly apply a value that has been previously defined. According to the documentation

A method with implicit parameters can be applied to arguments just like a normal method. In this case the implicit label has no effect. However, if such a method misses arguments for its implicit parameters, such arguments will be automatically provided.

An example

We’ll create a Person class that expects a first and last name. There’s also a manager field to be supplied:

case class Person(firstName: String, lastName: String)
                 (implicit manager: Person = null)

Immediately, you can see that the manager parameter has been decorated with the implicit keyword. What this says is:

First, any Person that can be accessed (at the point of construction) that has been declared implicit.
Second, any Person also declared implicit in companion modules.

For demonstration purposes, a showHierarchy function has been created to show management:

def showHierarchy() = {

  print(s"$firstName $lastName is ")

  if (manager == null) {
    println("not managed by anybody")
  } else {
    val managerFirstName = manager.firstName
    val managerLastName = manager.lastName

    println(s"managed by $managerFirstName $managerLastName")
  }
}

When the Person has a manager, you’ll see their name; otherwise this function will show that they are “not managed by anybody”.

Using the class

With all of this definition, we now take a look at usage. Note that the manager parameter has a default value of null. So, any invocation where an implicit can not be supplied still doesn’t need to be specified as it’s been defaulted.

val sam = Person("Sam", "Brown")
val joe = Person("Joe", "Smith")

sam.showHierarchy()
joe.showHierarchy()

This program shows the following output:

Sam Brown is not managed by anybody
Joe Smith is not managed by anybody

Makes sense. sam wasn’t declared as implicit and as such, wouldn’t be offered in the construction of joe. So, we modify the construction of sam and add the keyword:

implicit val sam = Person("Sam", "Brown")

This has an immediate impact on the output of the program:

Sam Brown is not managed by anybody
Joe Smith is managed by Sam Brown

As you can see, sam has now been implicitly offered to the construction of joe and as such, the manager parameter gets filled (in the construction of joe).

Handy.

Create a DSL using implicits in Scala

09 Jan 2017

Scala provides some simple options to get up and running the development of domain specific languages. In today’s post, I’ll take you through the usage of the implicit keyword that easily allows for this to happen.

End goal

For the purposes of today, I want to create a language that defines simple, directional instructions. “Move left”, “move forward”, etc. The end goal, I want a language that looks like this:

var jump = 5.0 units up
var shuffle = -2.0 units left
var fall = -6000.0 units down

Implicit conversion

The implicit conversion will allow us to get this process started. It’ll provide the ability for us to take that literal, double value (at the start of each of those statements), and convert them into a class instance that we can do more interesting things with.

implicit def convertDoubleToDirectionUtil(amount: Double) = new DirectionUtil(amount)

Ok, so we don’t know what a DirectionUtil class is just yet. That’s coming. Right now, it’s important to focus on this function which is an implicit conversion; allows us to express a double-value like normal and have Scala treat it as a DirectionUtil instance.

Implicit classes

The implicit class allows us to offer a class up to an implicit conversion when it’s in scope.

The definition of our DirectionUtil class here, allows us the units definition function.

implicit class DirectionUtil(val amount: Double) {

  def units(dir: String) = {
    dir match {
      case "forward" => scale((0.0, 0.0, 1.0), amount)
      case "backward" => scale((0.0, 0.0, -1.0), amount)
      case "left" => scale((-1.0, 0.0, 0.0), amount)
      case "right" => scale((1.0, 0.0, 0.0), amount)
      case "up" => scale((0.0, -1.0, 0.0), amount)
      case "down" => scale((0.0, 1.0, 0.0), amount)
      case _ => (0.0, 0.0, 0.0)
    }
  }

}

Really simply, this is just a vector-scaling function. Takes in a tuple, and adjusts it by a magnitude depending on the direction passed in.

Wrapping it all up

To finish up, we’ll put this class and implicit conversion into one neat singleton:

object DirectionUtil {
  val forward = "forward"
  val backward = "backward"
  val left = "left"
  val right = "right"
  val up = "up"
  val down = "down"

  def scale(vec: (Double, Double, Double), mag: Double) = {
    val (x, y, z) = vec
    (x * mag, y * mag, z * mag)
  }

  implicit def convertDoubleToDirectionUtil(amount: Double) = new DirectionUtil(amount)

  implicit class DirectionUtil(val amount: Double) {

    def units(dir: String) = {
      dir match {
        case "forward" => scale((0.0, 0.0, 1.0), amount)
        case "backward" => scale((0.0, 0.0, -1.0), amount)
        case "left" => scale((-1.0, 0.0, 0.0), amount)
        case "right" => scale((1.0, 0.0, 0.0), amount)
        case "up" => scale((0.0, -1.0, 0.0), amount)
        case "down" => scale((0.0, 1.0, 0.0), amount)
        case _ => (0.0, 0.0, 0.0)
      }
    }

  }
}

Now that we have this defined, we can start writing code that looks like this:

object Main extends App {
  import DirectionUtil._

  val jump = 5.0 units up
  println(jump)
}

. . which, as expected gives us an output like this:

(0.0,-5.0,0.0)

In closing

This is a really simple example, but you can see immediately how it can be applied to much more complex scenarios; and how you can be a lot more expressive in your scala source code.

Lists, vectors, sets and maps

15 Dec 2016

In a previous article, we’d spent a little bit of time working with maps in clojure; we also looked at formalising the structure of a map into records. In today’s post, I’ll discuss some of the collection types that will allows us to carry around many values at once.

General

Before we start looking at the collection types, there are some general-purpose functions that you can use with these types. These are very helpful functions that really form the basis of your collection processing:

first will return you the head of your collection; with rest providing you with everything else but the head. last will return you the final item in your collection and cons will provide you with the ability to construct a new list with a given head and tail.

nth will give you indexed access to you collection.

List

A list is an ordered collection of elements. From the documentation:

Lists are collections. They implement the ISeq interface directly (except for the empty list, which is not a valid seq). count is O(1). conj puts the item at the front of the list.

Because lists are a natural part of the language, they’ll be invoked as a function if they’re not defined correctly or quoted adequately. A list can be defined using the list function, but can also be quoted:

user=> (list :gibson :fender :ibanez)
(:gibson :fender :ibanez)
user=> '(:joe :john :peter)
(:joe :john :peter)
user=> '("a" "b" "c"))
("a" "b" "c")
user=> '(1 2 3)
(1 2 3)

Vector

A vector is an ordered collection of elements, but is optimised for the random-access case. From the documentation:

A Vector is a collection of values indexed by contiguous integers. Vectors support access to items by index in log32N hops. count is O(1). conj puts the item at the end of the vector.

user=> [:gibson :fender :ibanez]
[:gibson :fender :ibanez]

You can use the nth function to get the value at an index, or you can just invoke the vector itself. It’ll take an index, and return you the item:

user=> (nth [:gibson :fender :ibanez] 1)
:fender
user=> ([:gibson :fender :ibanez] 1)
:fender

Set

A set is an un-ordered collection of items. You can use sorted-set if your application requires your set to remain in an ordered way.

Clojure allows you to wrap your values with #{} in order to define a set, literally.

user=> #{:gibson :fender ibanez}
#{:gibson :fender ibanez}

Sets have methods to easily union two sets, or find the difference.

Map

Definition

A map is a collection of key-value-pairs. Declaring maps allows you to define attribute keys and values in an organised or arbitrary way. From the documentation:

A Map is a collection that maps keys to values. Two different map types are provided - hashed and sorted. Hash maps require keys that correctly support hashCode and equals. Sorted maps require keys that implement Comparable, or an instance of Comparator. Hash maps provide faster access (log32N hops) vs (logN hops), but sorted maps are, well, sorted. count is O(1). conj expects another (possibly single entry) map as the item, and returns a new map which is the old map plus the entries from the new, which may overwrite entries of the old.

user=> { :name "Michael" :age 21 }
{:name "Michael", :age 21}
user=> ({ :name "Michael" :age 21 } :name)
"Michael"

user=> (def person { :name "Michael" :age 21 })
#'user/person
user=> (assoc person :age 22)
{:name "Michael", :age 22}

user=> (def person-aux { :hair :brown })
#'user/person-aux
user=> (merge person person-aux)
{:name "Michael", :age 21, :hair :brown}

Finishing up

This has been a quick lap around the collection types in Clojure.

Data definition in Clojure

07 Dec 2016

Introduction

In today’s article, I’ll briefly go over representing record-like information in Clojure.

Maps

First off, we’ll take a quick tour using a Map.

A Map is a collection that maps keys to values

So this is a pretty fundamental data type that we can use. We can pretty easily represent a person using it:

user=> (def person { :first-name "John" :last-name "Smith" :age 21 })
#'user/person
user=> user/person
{:first-name "John", :last-name "Smith", :age 21}

We can use count to count the number of pairs in the map.

user=> (count person)
3

We can use conj to give our person a hair attribute.

user=> (conj person { :hair :brown })
{:first-name "John", :last-name "Smith", :age 21, :hair :brown}

To navigate the map a little bit we can use get, contains?, find, keys, vals.

Using seq you can return a seq over the pairs; on each item in the seq you can use key to get the value of the key and val to get the value.

user=> (map #(println (key %1)) (seq person))
:first-name
:last-name
:age
(nil nil nil)

This gives us a full key/value pairing structure for us to arbitrarily represent data in a organized fashion; but not organized enough.

Records

Using records or defrecord we can turn a list of keys into a type that we can repeatably construct.

user=> (defrecord Person [ first-name last-name age ])
user.Person
user=> (def person-rec (->Person "Mary" "Jane" 25))
#'user/person-rec
user=> user/person-rec
#user.Person{:first-name "Mary", :last-name "Jane", :age 25}

We’re afforded all of the same functions above to work on this value; but we’re given the positional factory function ->Constructor so that we can construct our types in a much more intuitive way.

Domain functions

Now that we’ve spent a little bit of time creating data structures and defining data records, we can create functions that will allow us to perform operations on this data. Domain functions in Clojure perform operations specific to the data structure. In this following example, we’ve created a function to format a person’s full name:

user=> (defn get-full-name [p] (str (get p :first-name) " " (get p :last-name)))
#'user/get-name
user=> (user/get-full-name person)
"John Smith"

Object-oriented programming gives us polymorphism by allowing us to implement a class’s method differently per type that we derive from a common base. Clojure, gives us this multi-dispatch effect (choosing the right function for the given data-type) through multi methods and protocols.

Multi-methods

We’re going to use the defmulti macro to define our multi method and we’ll provide implementations using the defmethod macro. From the documentation:

Clojure eschews the traditional object-oriented approach of creating a new data type for each new situation, instead preferring to build a large library of functions on a small set of types. However, Clojure fully recognizes the value of runtime polymorphism in enabling flexible and extensible system architecture. Clojure supports sophisticated runtime polymorphism through a multimethod system that supports dispatching on types, values, attributes and metadata of, and relationships between, one or more arguments.

We’re going to expand the Person example above, by adding a new record of Company.

user=> (defrecord Person [first-name last-name age])
user=> (defrecord Company [name number])

Then we’ll define our multi method called get-full-name. Its job is to put together the name of our entity. Because we have both a Person and Company type entity, we’re going to need two different implementations:

user=> (defmulti get-full-name (fn [entity] (class entity)))
user=> (defmethod get-full-name Person [person] (str (get person :first-name) " " (get person :last-name)))
user=> (defmethod get-full-name Company [company] (get company :name))

The Person implementation of the get-full-name function concatenates the :first-name and :last-name attributes together, where as the Company implementation need only return the :name attribute.

Something that is interesting and unique to multi-methods is value-based dispatch; we’ve already seen type-based dispatch.

Consider a temperature conversion between Fahrenheit and Celsius. We create our multi method the same way, but this time we need to give the parameter values identity:

(defmulti convert-temp (fn [src-unit dest-unit amount] [src-unit dest-unit]))

We can now give our multi method some implementations based on the source and destination units passed:

;; C to F
(defmethod convert-temp [:c :f] [_ _ c] (+ 32 (* c 1.8)))

;; F to C
(defmethod convert-temp [:f :c] [_ _ f] (/ (- f 32) 1.8))

;; anything else
(defmethod convert-temp :default 
  [su du a] 
  (if (= su du) 
    a 
    (assert false "Conversion between units was not defined")))

We can now test out that the dispatching works:

user=> (convert-temp :f :c 100)
37.77777777777778
user=> (convert-temp :c :f 37.77777777777778)
100.0
user=> (convert-temp :c :c 50)
50
user=> (convert-temp :c :k 50)

AssertionError Assert failed: Conversion between units was not defined
false  user/eval1317/fn--1318 (form-init7827911538193486373.clj:1)

Protocols

Protocols are a little more reminiscent of object-oriented programming in a sense that they are closely related to interfaces. From the documentation:

Clojure is written in terms of abstractions. There are abstractions for sequences, collections, callability, etc. In addition, Clojure supplies many implementations of these abstractions. The abstractions are specified by host interfaces, and the implementations by host classes. While this was sufficient for bootstrapping the language, it left Clojure without similar abstraction and low-level implementation facilities. The protocols and datatypes features add powerful and flexible mechanisms for abstraction and data structure definition with no compromises vs the facilities of the host platform.

So, protocols give us a way to defining abstractions. We can treat our Person and Company scenario as such, by calling them a Party.

We use defprotocol to start our abstraction definition. extend-protocol is then used to supply implementations.

user=> (defprotocol Party
  #_=>   (get-full-name [entity]))

user=> (extend-protocol Party
  #_=>   Person
  #_=>   (get-full-name [person] (str (get person :first-name) " " (get person :last-name))))

user=> (extend-protocol Party
  #_=>   Company
  #_=>   (get-full-name [company] (get company :name)))

Wrapping up

This has been a brief tour on creating map/record data and some domain functions to work with them.

Older Newer

Cogs and Levers A blog full of technical stuff

Streams in Java 8

What is a stream?

An example

Map

Mapping deeper

Filter

Reduce

Implicit values in Scala

An example

Using the class

Create a DSL using implicits in Scala

End goal

Implicit conversion

Implicit classes

Wrapping it all up

In closing

Lists, vectors, sets and maps

General

List

Vector

Set

Map

Definition

Finishing up

Data definition in Clojure

Introduction

Maps

Records

Domain functions

Multi-methods

Protocols

Wrapping up