Testing REST APIs with cURL

Introduction

Testing is a large component of any software development done, sometimes though - you don’t want to go through a full unit test suite just to see what a REST service is doing. I’ve come across some interesting concepts with cURL that will certainly be a shortcut benefit to seeing what responses your REST services are returning.

Requests

You can simulate all of the different HTTP verbs against any URL you’d like using cURL with the following syntax at the console:

# Retrieve person (id: 1)
$ curl -i -X GET http://localhost/service/people/1

# Retrieve all people
$ curl -i -X GET http://localhost/service/people

# Delete person (id: 1)
$ curl -i -X DELETE http://localhost/service/people/1

# Create a new person
$ curl -i -X POST -H 'Content-Type: application/json' -d '{"first_name": "John", "last_name": "Smith"}' http://localhost/service/people

# Modify a person (id: 1)
$ curl -i -X PUT -H 'Content-Type: application/json' -d '{"first_name": "Jane", "last_name": "Smith"}' http://localhost/service/people/1

Haskell Video Links

A small list of video library links on the topic of Haskell are accumulating in my inbox, so I thought I’d compile them here for viewing later.

• Haskell video collection on YouTube
• Happstack foundation example: ControlV paste board
• Happstack plugins
• Simon Peyton Jones: Haskell Tutorial (xmonad)
• Simon Peyton Jones: A History of Haskell
• Video presentation on Haskell.org
• Code Deconstructed Series

MongoDB Aggregation Framework

Introduction

Sometimes having the power of MapReduce at your fingertips and applying this technology to simpler aggregate queries can be more hassle than it needs to be. MongoDB provides a simpler solution (for a simpler class of problems) in the form of the Aggregation framework. This framework allows you to develop queries within the mongo environment that are analogous to GROUP BY, HAVING, COUNT, SUM, etc. that you would normally use in “relational land”.

Today’s post, I want to walk through a couple of simple queries on using this framework to maximise productivity when pivoting data.

Marrying the old with the new

As a bit of a cheat’s reference, the following table provides the some examples of aggregate queries in a relational database and how they transpose over to the Mongo aggregation environment.

The source of this table can be found here.

Counting records

Sql example:

	
SELECT COUNT(*) AS count
FROM items

Mongo example:

	
db.items.aggregate( [
{ $group: { _id: null,
            count: { $sum: 1 } } }
] )

Accumulating values

Sql example:

	
SELECT SUM(price) AS total
FROM items

Mongo example:

	
db.items.aggregate( [
{ $group: { _id: null,
            total: { $sum: "$price" } } }
] )

Aggregation with identifier

Sql example:

	
SELECT category_id, SUM(price) AS total
FROM items
GROUP BY category_id

Mongo example:

	
db.items.aggregate( [
{ $group: { _id: "$category_id",
            total: { $sum: "$price" } } }
] )

Aggregation with identifier and sorting

Sql example:

	
SELECT category_id, SUM(price) AS total
FROM items
GROUP BY category_id
ORDER BY total

Mongo example:

	
db.items.aggregate( [
{ $group: { _id: "$category_id",
            total: { $sum: "$price" } } },
{ $sort: { total: 1 } }
] )

Multiple Aggregation

Sql example:

	
SELECT category_id, when SUM(price) AS total
FROM items
GROUP BY category_id, when

Mongo example:

	
db.items.aggregate( [
{ $group: { _id: { category_id: "$category_id",
                   when: "$when" },
            total: { $sum: "$price" } } }
] )

Aggregate Filtration

Sql example:

	
SELECT category_id, count(*)
FROM items
GROUP BY category_id
HAVING count(*) > 1

Mongo example:

	
db.items.aggregate( [
{ $group: { _id: "$category_id",
            count: { $sum: 1 } } },
{ $match: { count: { $gt: 1 } } } 
] )

Multiple Aggregate Filtration

Sql example:

	
SELECT category_id, when, SUM(price) AS total
FROM items
GROUP BY category_id, when
HAVING total > 100

Mongo example:

	
db.items.aggregate( [
{ $group: { _id: { category_id: "$category_id",
                   when: "$when" },
            total: { $sum: "$price" } } },
{ $match: { total: { $gt: 100 } } } 
] )

Aggregate with Source Filtration

Sql example:

	
SELECT category_id, SUM(price) AS total
FROM items
WHERE active = 1
GROUP BY category_id

Mongo example:

	
db.items.aggregate( [
{ $match: { active: 1 } },
{ $group: { _id: "$category_id",
            total: { $sum: "$price" } } }
] )

Aggregate Filtration with Source Filtration

Sql example:

	
SELECT category_id, SUM(price) AS total
FROM items
WHERE active = 1
GROUP BY category_id
HAVING total > 100

Mongo example:

	
db.items.aggregate( [
{ $match: { active: 1 } },
{ $group: { _id: "$category_id",
            total: { $sum: "$price" } } },
{ $match: { total: { $gt: 100 } } }
] )

Aggregated joins

Sql example:

	
SELECT category_id, SUM(co.weight) AS weight
FROM items i, components co
WHERE co.item_id = i.id
GROUP BY category_id

Mongo example:

	
db.items.aggregate( [
{ $unwind: "$components" },
{ $group: { _id: "$category_id",
            weight: { $sum: "$components.weight" } } }
] )

Sub-query Aggregation

Sql example:

	
SELECT COUNT(*)
FROM (SELECT category_id, when
      FROM items
      GROUP BY category_id, when) AS Table1

Mongo example:

	
db.items.aggregate( [
{ $group: { _id: { category_id: "$category_id",
                   when: "$when" } } },
{ $group: { _id: null, count: { $sum: 1 } } }
] )

Stored Javascripts in MongoDB

Introduction

Coming from a relational database background, technologies such as stored procedures and user defined functions have always helped out when building a database infrastructure. MongoDB provides the same sort of code storage in stored javascripts in the database.

Creating a stored javascript

Creating a stored javascript into a database is a straight forward process of adding an item to the system.js collection.

> db.collection.js.save({_id: "greaterThan10", 
...                      value: function (x) { return x > 10 }});

Ok, this isn’t the most useful of functions. We’re testing if the value passed in the greater than 10. We’re able to use this in queries of our own using $where syntax like so:

> db.people.find({$where: "greaterThan10(this.age)"})

This would get all of the “people” documents out of the database where they were over the age of 10. This is quite verbose of such a simple example, but you can see that by filling out the function in the saved javascript with more complex operations, you could achieve a lot with a little.

Removing a stored javascript

Working with the collection as usual, you can simply remove your stored javascript by id.

> db.collection.js.remove({_id: "greaterThan10"})

Testing

As a final note, once you’ve created your stored javascript you can test it using eval easy enough.

> db.eval("return greaterThan10(9);")
false
> db.eval("return greaterThan10(11);")
true

This is just a short introduction into the world of stored javascripts. The internal workings of MongoDB is all based on javascript, so it’s a good idea to have your skills nice and sharp before going in!

64bit ABI Cheatsheet

Introduction

Using a few different articles around the web, I thought it might be a good idea to aggregate all of the information around calling conventions in 64bit land. This could come in handy when wanting to write a cross OS compliant library at the assembly level. This particular article will directly target the assembly programming layer so that topics like C++ name mangling and caller clean-up are removed from its scope.

References

• x86 calling conventions
• x86-64.org documentation
• Mac OSX 64 bit Assembly System Calls

Microsoft

Windows will use RCX, RDX, R8 and R9 for the first four integer or pointer arguments. XMM0, XMM1, XMM2 and XMM3 are used for floating point arguments. Additional arguments are passed via the stack (right to left).

An integer or pointer return value will be returned in RAX. Floating point return will be in XMM0.

System V

System V operating systems will use RDI, RSI, RDX, RCX, R8 and R9. XMM0, XMM1, XMM2, XMM3, XMM4, XMM5, XMM6 and XMM7 will be used to pass floating point parameters. RAX will hold the syscall number. Additional arguments are passed via the stack (right to left).

Return values are sent back via RAX.

Syscall Numbers

It’s interesting to note the structure of the syscall number when it comes time to execute. Looking at syscall_sw.h, you’ll see that apple machines want a 2 in the higher-order double word such that the write syscall, normally passed as 0x04 would be passed as 0x2000004 in OSX.

That’s it for today. Just a cheat sheet really.