Listing open ports and who owns them
21 Jul 2015To list all of the network ports and users that own them you can use the lsof command.
sudo lsof -iThe netstat command is also available to provide the same sort of information.
sudo netstat -lptuTo list all of the network ports and users that own them you can use the lsof command.
sudo lsof -iThe netstat command is also available to provide the same sort of information.
sudo netstat -lptuA promise is an object that represents the result of a computation; whether it be a positive or negative result. What’s special about promises in concurrent programming is that they allow you to compose your code in such a way that is a little more natural than the callbacks-in-callbacks style.
In today’s post, I’m going to work with the Q library for Node.js to demonstrate how we can use promises to clean up our code into more concise blocks of logic.
From the npm page for the Q library, it even says:
On the first pass, promises can mitigate the “Pyramid of Doom”: the situation where code marches to the right faster than it marches forward.
In the following example, I’m going to simulate some work using setTimeout. This will also give us some asynchronous context. Here are the two function calls we’ll look to sequence:
var getUserByName = function (name, callback) {
setTimeout(function () {
try {
callback(null, {
id: 1,
name: name
});
} catch (e) {
callback(e, null);
}
}, 1000);
};
var getCarsByUser = function (userId, callback) {
setTimeout(function () {
try {
callback(null, ['Toyota', 'Mitsubishi', 'Mazda']);
} catch (e) {
callback(e, null);
}
}, 1000);
};Even though the inputs and outputs of these functions are invalid, I just wanted to show that getCarsByUser is dependent on the output of getUserByName.
As any good-citizen in the node eco-system the last parameter of both of these functions is a callback function that take the signature of (err, data). Sequencing this code normally would look as follows:
getUserByName('joe', function (err, user) {
getCarsByUser(user.id, function (err, cars) {
// do something here
});
});The code starts to move to the right as you get deeper and deeper into the callback tree.
We can convert this into promises with the following code:
var pGetUserByName = Q.denodeify(getUserByName),
pGetCarsByUser = Q.denodeify(getCarsByUser);
pGetUserByName('joe').then(pGetCarsByUser)
.done();Because we’ve structured our callbacks “correctly”, we can use the denodeify function to directly convert our functions into promises. We can then sequence our work together using then. If we wanted to continue to build this promise, we could omit the done call for something else to complete work on.
When error handling gets involved in the callback scenario, the if-trees start to muddy-up the functions a little more:
getUserByName('joe', function (err, user) {
if (err != null) {
console.error(err);
} else {
getCarsByUser(user.id, function (err, cars) {
if (err != null) {
console.error(err);
} else {
// work with the data here
}
});
}
});In the promise version, we can use the fail function to perform our error handling for us like so:
pGetUserByName('joe').then(pGetCarsByUser)
.fail(console.error)
.done();Makes for a very concise set of instructions to work on.
There are a couple of ways to get promises integrated into your existing code base. Of course, it’s always best to implement these things at the start so that you have this model of programming in the front of your mind; as opposed to an after thought.
From synchronous code, you can just use the fcall function to start off a promise:
var getName = Q.fcall(function () {
return 'John';
});In this case, you just supply any parameters that are expected also:
var getGenderName = function (gender) {
if (gender == 'F') {
return 'Mary';
}
return 'John';
}
var getName = Q.fcall(getGenderName, 'F');In asynchronous cases, you can use defer. This will require you to restructure your original code though to include its use.
var getGenderName = function (gender) {
var deferred = Q.defer();
var done = false;
var v = 0;
var prog = function () {
setTimeout(function () {
if (!done) {
v ++;
deferred.notify(v);
prog();
}
}, 1000);
};
prog();
setTimeout(function () {
if (gender == 'F') {
deferred.resolve('Mary');
} else if (gender == 'M') {
deferred.resolve('John');
} else {
deferred.reject(new Error('Invalid gender code'));
}
done = true;
}, 5000);
return deferred.promise;
};We’re able to send progress updates using this method as well. You can see that with the use of the notify function. Here’s the call for this function now:
getGenderName('F')
.then(function (name) {
console.log('Gender name was: ' + name);
})
.progress(function (p) {
console.log('Progress: ' + p);
})
.fail(function (err) {
console.error(err);
})
.done();resolve is our successful case, reject is our error case and notify is the progress updater.
This function can be restructured a little further with the use of promise though:
var getGenderName = function (gender) {
return Q.promise(function (resolve, reject, notify) {
var done = false;
var v = 0;
var prog = function () {
setTimeout(function () {
if (!done) {
v ++;
notify(v);
prog();
}
}, 1000);
};
prog();
setTimeout(function () {
if (gender == 'F') {
resolve('Mary');
} else if (gender == 'M') {
resolve('John');
} else {
reject(new Error('Invalid gender code'));
}
done = true;
}, 5000);
});
};Our client code doesn’t change.
Finally, nfcall and nfapply can be used to ease the integration of promises in your code. These functions are setup deliberately to deal with the Node.js callback style.
As mentioned previously, Node.js runs in a single thread. In order to get it running with concurrency you need to use a library. In today’s post, I’m going to go through the async library really briefly.
It’s important to note that you need to follow conventions in order for this library to be successful for you.
All these functions assume you follow the Node.js convention of providing a single callback as the last argument of your async function.
A great deal of asynchronous work that you’ll do will be conducted on collections/lists. The async module provides the usual processing facilities for these data types, and they’re all simple to use. Here we’ll filter out non-prime numbers:
var async = require('async'),
range = require('node-range');
var candidates = range(3, 1000).toArray();
var isPrime = function (v, callback) {
if ((v % 2) == 0) {
callback(false);
return;
}
var m = 3;
while (m < v) {
if ((v % m) == 0) {
callback(false);
return;
}
m += 2;
}
callback(true);
};
async.filter(candidates, isPrime, function (res) {
console.log(res);
});Note that isPrime uses a callback to send its result back. This allows all items in the array, candidates to participate nicely in the async operation.
There are a few different work strategies you can employ with the async module.
series will execute items one-after-the-other; parallel execute items at the same time (or, in parallel); waterfall operates like series however it’ll automatically supply the return of the previous call as input to the next.
Everything you expect is in this library.
Node.js operates in a single thread. In order to get your program to take advantage of all of the cores in your machine, you’ll need some extra help. Today’s post is about the cluster module which has been created to solve this very problem.
The cluster module gives your Node.js application the ability to create new processes that will execute your code. Any children that you spawn will share your server ports so this is an excellent utility for process resilience in network applications.
Using the cluster library, you’re given a master and worker relationship in your code. The master has the ability to spawn new workers. From there you can use message passing, IPC, network, etc. to communicate between your workers and the master.
In the following sample, I’ll put together a http server that allows a user to kill and create processes. You’ll also see the round-robin approach to requests as well as each of the workers sharing the same port.
var cluster = require('cluster'),
http = require('http'),
os = require('os');
if (cluster.isMaster) {
var nWorkers = os.cpus().length;
console.log('Creating ' + nWorkers + ' workers');
for (var i = 0; i < nWorkers; i ++) {
var w = cluster.fork();
w.on('message', function (msg) {
console.log(msg);
if (msg.cmd == 'life') {
var w = cluster.fork();
console.log('Just spawned ' + w.process.pid);
}
});
}
cluster.on('exit', function (worker, code, signal) {
console.log('Worker ' + worker.process.pid + ' has finished');
});
} else {
http.createServer(function (req, res) {
if (req.url == '/poison') {
cluster.worker.kill();
res.writeHead(200);
res.end('pid is taking poison ' + process.pid);
} else if (req.url == '/life') {
process.send({ cmd: 'life' });
res.writeHead(200);
res.end('new pid was requested');
} else {
res.writeHead(200);
res.end('pid of this worker is ' + process.pid);
}
}).listen(3000);
}Measuring isMaster and in other cases isWorker allows you to place code for both sides of your process. This is like the tradition unix fork process.
We count the number of cpu cores and store that off in nWorkers. This is how many workers we’ll create. Messages are delivered from the worker using the send function. These are then caught and interpreted by the master using the message event.
The master will go through the workers in a round-robin fashion (by default) who are all listening on port 3000.
There is plenty more to this API than what’s in this example. Check out the documentation for more information.
Safely responding to error scenarios can be difficult at times. Changing the context of when exceptions are raised amplifies and complicates the problem somewhat.
In today’s post, I’m going to walk through some simple usage of the Domain module in Node.js and how it can be applied in scenarios to make your software more fault tolerant overall.
The description of a domain given in the API documentation sums it up best, I think:
Domains provide a way to handle multiple different IO operations as a single group. If any of the event emitters or callbacks registered to a domain emit an
errorevent, or throw an error, then the domain object will be notified, rather than losing the context of the error in theprocess.on('uncaughtException')handler, or causing the program to exit immediately with an error code.
Going off the information in my previous post about eventing, the error events generated by EventEmitter objects are going to be registered inside of the domain allowing us a greater level of control and visibility in exception cases, no matter the context.
In this example, we’ll create a generic EventEmitter and domain and we’ll see how the chain of error handling occurs:
var EventEmitter = require('events').EventEmitter;
var domain = require('domain');
var d1 = domain.create();
var emitter = new EventEmitter();
d1.on('error', function (err) {
console.log('Domain: ' + err.stack);
});
d1.add(emitter);
emitter.on('error', function (err) {
console.log('Listener: ' + err.stack);
});We’ve created the domain d1 and have attached an error handler to it. We’ve also created our EventEmitter called emitter and attached a handler to it as well. The following code now starts to raise errors:
// this one gets handled by the emitter listener
emitter.emit('error', new Error('First error'));
// removing the emitter listener should force the exception
// to bubble to the domain
emitter.removeAllListeners('error');
emitter.emit('error', new Error('Second error'));
// removing the emitter from the domain should have us converting
// the error into an unhandled exception
d1.remove(emitter);
emitter.emit('error', new Error('Third error'));As the comments read, we have our error being reported in different places as objects get detached from one another. The output of which looks like this:
Listener: Error: First error
at Object.<anonymous> (/home/michael/event1.js:19:23)
at Module._compile (module.js:460:26)
at Object.Module._extensions..js (module.js:478:10)
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Function.Module.runMain (module.js:501:10)
at startup (node.js:129:16)
at node.js:814:3
Domain: Error: Second error
at Object.<anonymous> (/home/michael/event1.js:24:23)
at Module._compile (module.js:460:26)
at Object.Module._extensions..js (module.js:478:10)
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Function.Module.runMain (module.js:501:10)
at startup (node.js:129:16)
at node.js:814:3
events.js:85
throw er; // Unhandled 'error' event
^
Error: Third error
at Object.<anonymous> (/home/michael/event1.js:29:23)
at Module._compile (module.js:460:26)
at Object.Module._extensions..js (module.js:478:10)
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Function.Module.runMain (module.js:501:10)
at startup (node.js:129:16)
at node.js:814:3Our exceptions are reported to our attached handler on emitter first. Once it’s been removed as a handler, the error is then reported to the domain d1. Once the domain has no knowledge of emitter, the last error manifests as an unhandled error.
An interesting point made in the documentation is about implicit and explicit binding.
If domains are in use, then all new EventEmitter objects (including Stream objects, requests, responses, etc.) will be implicitly bound to the active domain at the time of their creation.
So, if we’re in a scenario where we’re creating EventEmitter objects inside of the domain, there’s no need to add them using the add function.
In a lot of cases you aren’t afforded this luxury. The objects that you want to observe are created at a higher scope or just generally before the domain is constructed; in these cases you need to use the add function.
The api documentation contains a great example usage of the domain module in conjunction with the cluster model. It illustrates the ability to give your application a higher level of resilience against errors thrown so that not all of your uses are effected by a single rogue request.
The following started as an excerpt from the aforementioned documentation, but has been adapted for this article:
var server = require('http').createServer(function(req, res) {
var d = domain.create();
d.on('error', function(er) {
console.error('error', er.stack);
try {
// make sure we close down within 30 seconds
var killtimer = setTimeout(function() {
process.exit(1);
}, 30000);
// But don't keep the process open just for that!
killtimer.unref();
// stop taking new requests.
server.close();
// Let the master know we're dead. This will trigger a
// 'disconnect' in the cluster master, and then it will fork
// a new worker.
cluster.worker.disconnect();
// try to send an error to the request that triggered the problem
res.statusCode = 500;
res.setHeader('content-type', 'text/plain');
res.end('Oops, there was a problem!\n');
} catch (er2) {
// oh well, not much we can do at this point.
console.error('Error sending 500!', er2.stack);
}
});
// explicitly added req and res to the domain
d.add(req);
d.add(res);
// Now run the handler function in the domain.
d.run(function() {
handleRequest(req, res);
});
});
server.listen(PORT);The run method at the end is really the error pillow for our request handler. We don’t really know what went wrong in unhandled exception cases, all we know is that “something” went wrong. Safest course of action in these conditions is to shut down the failing server and start again.
Pretty neat.