Message::Passing(3pm)

Usage

    message-pass --input STDIN --output STDOUT
    {"foo": "bar"}
    {"foo":"bar"}

Overview

A library for building high performance, loosely coupled and reliable/resilient applications, structured as small services which communicate over the network by passing messages.

Basic premise

You have data for discrete events, represented by a hash (and serialized as JSON).

This could be a text log line, an audit record of an API event, a metric emitted from your application that you wish to aggregate and process - anything that can be a simple hash really..

You want to be able to shove these events over the network easily, and aggregate them / filter and rewrite them / split them into worker queues.

This module is designed as a simple framework for writing components that let you do all of these things, in a simple and easily extensible manor.

For a practical example, You generate events from a source (e.g. ZeroMQ output of logs and performance metrics from your Catalyst FCGI or Starman workers) and run one script that will give you a central application log file, or push the logs into Elasticsearch.

There are a growing set of components you can plug together to make your solution.

Getting started is really easy - you can just use the "message-pass" command installed by the distribution. If you have a common config that you want to repeat, or you want to write your own server which does something more flexible than the normal script allows, then see Message::Passing::DSL.

To dive straight in, see the documentation for the command line utility message-pass, and see the examples in Message::Passing::Manual::Cookbook.

For more about how the system works, see Message::Passing::Manual::Concepts.

Components

Below is a non-exhaustive list of components available.

Inputs

Inputs receive data from a source (usually a network protocol).

They are responsible for decoding the data into a hash before passing it onto the next stage.

Inputs include:

Message::Passing::Input::STDIN

Message::Passing::Input::ZeroMQ

Message::Passing::Input::STOMP

Message::Passing::Input::AMQP

Message::Passing::Input::Syslog

Message::Passing::Input::Redis

Message::Passing::Input::Test

You can easily write your own input, just use AnyEvent, and consume Message::Passing::Role::Input.

Filter

Filters can transform a message in any way.

Examples include:

Message::Passing::Filter::Null - Returns the input unchanged.

Message::Passing::Filter::All - Stops any messages it receives from being passed to the output. I.e. literally filters all input out.

Message::Passing::Filter::T - Splits the incoming message to multiple outputs.

You can easily write your own filter, just consume Message::Passing::Role::Filter.

Note that filters can be chained, and a filter can return undef to stop a message being passed to the output.

Outputs

Outputs send data to somewhere, i.e. they consume messages.

Message::Passing::Output::STDOUT

Message::Passing::Output::AMQP

Message::Passing::Output::STOMP

Message::Passing::Output::ZeroMQ

Message::Passing::Output::WebHooks

Message::Passing::Output::Search::Elasticsearch

Message::Passing::Output::Redis

Message::Passing::Output::Test

This module

This is a simple MooX::Options script, with one input, one filter and one output. To build your own similar scripts, see:

Message::Passing::DSL - To declare your message chains.

Message::Passing::Role::CLIComponent - To provide "foo" and "foo_options" attribute pairs.

Message::Passing::Role::Script - To provide daemonization features.

Methods

build_chain

Builds and returns the configured chain of input => filter => output.

start

Class method to call the run_message_server function with the results of having constructed an instance of this class, parsed command line options and constructed a chain.

This is the entry point for the script.

Support

Bugs

Please log bugs at rt.cpan.org. Each distribution has a bug tracker link in its metacpan.org page.

Discussion

#message-passing on irc.perl.org.

Source code

Source code for all modules is available at <http://github.com/suretec> and forks/patches are very welcome.

Sponsorship

This module exists due to the wonderful people at Suretec Systems Ltd. <http://www.suretecsystems.com/> who sponsored its development for its VoIP division called SureVoIP <http://www.surevoip.co.uk/> for use with the SureVoIP API - <http://www.surevoip.co.uk/support/wiki/api_documentation>

Releases

6 releases since Debian 8 (Jessie); the text changed 3 times.