---
title: Fuse MQ Fabric
in_menu: false
sort_info: 10
--- pipeline:markdown
{:toc} 

# Fuse MQ Fabric

This document describes how to create a **Messaging Cloud** using [Fuse MQ](../index.html) and its integration with [Fuse Fabric](../../fabric/index.html)

## Fuse MQ Profiles

In Fabric, everything is defined through profiles. The main profile for Fuse MQ brokers is called **mq-base** and it defines basic options for the broker instances.

It's an abstract profile, so you shouldn't ever try to create an instance using this profile, as it will not give you the expected effect.
For basic broker creation you can use the **mq** profile which inherits most of its properties from **mq-base**.

## Creating new broker instances

A Fuse MQ broker is an Karaf container instance running a message broker profile. The profile defines the broker dependencies (through features) and configuration for the broker.
The simplest option is to use the pre-canned **mq** profile. You can see the display the contents of the **mq** profile and it parent **mq-base**, using the command:

    FuseFabric:karaf@root> profile-display mq
    ...
    FuseFabric:karaf@root> profile-display mq-base
    ...


To create a new **mq** broker instance called **broker1**, use the command:

     FuseFabric:karaf@root>container-create-child --profile mq root broker1
     The following containers have been created successfully:
 	    broker1

This will create a new container called **broker1** and with a broker (of the same name) running on it.
The **--parent** option allows the karaf container to inherit feature and configuration repositories from the current container.


The **mq-create** command provides a short cut to the above with more flexibility because it result in a new profile.
To create a new broker instance called **broker1** with **mq-create**, use the command:

    FuseFabric:karaf@root> mq-create --create-container broker1 broker1
    MQ profile broker1 ready
    Successfully created container broker1

Just like the basic **container-create-child** command, **mq-create** will create a container called **broker1** and run a broker on it.
There are some differences:
* The new **broker1** container has its parent automatically assigned to the current container
* The new broker has its own new **broker1** profile which is based on the **mq-base** template
It is possible to further modify the **broker1** profile to adjust the configuration of this new broker as we will see shortly.


## Starting broker on existing container instances

The **mq-create** command can start brokers on existing containers. Let's take a look at the following example

    FuseFabric:karaf@root> container-create-child --profile karaf root broker1
    The following containers have been created successfully:
	    broker1

    FuseFabric:karaf@root> mq-create --assign-container broker1 broker1
    MQ profile broker1 ready
    Profile successfully assigned to broker1


Here, we first create an "empty" karaf container and later create and start the new **broker1** profile on it with **assign-container** option.
Of course, instead of local child container we can assign a broker to ssh and cloud containers.

## Connecting to the broker

So far we discussed how we can start brokers in Fabric. Now let's discuss for a bit how Fabric discovery works and how clients can connect to the broker.
Every broker started in fabric has its group. If nothing is specified it's **default**.
This means that when the broker starts, it will register itself to the fabric registry under the defined group.
All the client needs to do then is to use connection url like

    discovery:(fabric:default)

where default in this case is the group name we want use. The connection factory will then find available brokers in the group and connect the client to the one of them.
You can test the brokers started with the previously described methods by creating a container with the **example-mq** profile

    FuseFabric:karaf@root> container-create-child --profile example-mq root example

After the example container is successfully provisioned

    FuseFabric:karaf@root> watch container-list

you can connect to it and check its logs to verify the flow of messages, like

    FuseFabric:karaf@root> container-connect -u <user> example
    FuseFabric:karaf@root> log:display

Here's a short clip that demonstrates how to start a broker and run a demo in Fabric

<object width="853" height="480"><param name="movie" value="http://www.youtube.com/v/rua-7asylNw?version=3&amp;hl=en_US&amp;rel=0"></param><param name="allowFullScreen" value="true"></param><param name="allowscriptaccess" value="always"></param><embed src="http://www.youtube.com/v/rua-7asylNw?version=3&amp;hl=en_US&amp;rel=0" type="application/x-shockwave-flash" width="853" height="480" allowscriptaccess="always" allowfullscreen="true"></embed></object>

## Topologies

### Broker clusters

The basic "cluster" of brokers means that you can start multiple unconnected brokers, identified by unique names, in the same group. Each broker will register with that group
and clients using **discovery:(fabric:&lt;group name&gt;)** will naturally load balanced across them.
This type of setup is ideal for load balancing non-persistent messages across brokers and provide high-availability.

### Master-Slave

With Master-Slave, multiple peer brokers provide the same service and all contest to be the master.
Only **one** master can exist at a given time, the rest resume as slaves. When the master stops,
the remaining brokers (slaves) contest to become the new master. When the broker containers
span machines or data centres, the result is a highly available broker.

Master-Slave topologies are simple to setup in Fabric. All you need to do is start multiple brokers with **same** name in a group. The Fabric registry will do the rest for you.
So if you do something like:

    mq-create --create-container broker1,broker2,broker3 hq-broker

This will create three container instances (broker1, broker2 and broker3) and try to start a broker named **hq-broker** in the group **default** on each of them.
The first broker that starts will become the master, while the others will be the slaves.
You can run the same example as before and test what happens when you stop the master (one of the slaves will take over and clients will reconnect).
If brokers are persistent you need to ensure that all use the same store, which is the case by default because they share the value from the mq-base profile.
When running instances on separate machines and using shared storage it will require some additional setup. We'll cover that topic later on.

The benefit of this kind of master-slave setup is that you don't depend on locking strategies of shared storage so it can even be used with non-persistent brokers.
The fabric group uses ZooKeeper to manage a shared distributed lock that controls ownership of the master status.

The following clip demonstrates runnning a master-slave broker pair and example reconnecting when master dies.

<object width="853" height="480"><param name="movie" value="http://www.youtube.com/v/mZHr6uJqTZs&hl=en&fs=1"></param><param name="allowFullScreen" value="true"></param><embed src="http://www.youtube.com/v/mZHr6uJqTZs&hl=en&fs=1" type="application/x-shockwave-flash" allowfullscreen="true" width="853" height="480"></embed></object>

One more feature of Fabric Master-Slave setups is that you can share the same containers between multiple master-slave instances.
Given the three containers running the **hq-broker** above, it is possible to reuse the same containers for another highly available broker, **web-broker**.
 You can do the following:

    mq-create --assign-container broker1,broker2,broker3 web-broker

This will assign the new **web-broker** profile to the same instances already running **hq-broker**.
The internal Fabric mechanism will prevent two masters to run on the same instance, so the master will be elected by one of the two instances running the slave for the **hq-broker**.
The other brokers will run slaves for the **web-broker**. In this scenario we have two masters running and two slaves waiting.
This is just a simple example, but it shows how it can help better utilize the resources.

### Networks

If you want your broker to establish network connections to other brokers from a specific group, you can specify it with the **networks** option to the **mq-create** command. Examine the following scenario

    mq-create --group us-east --networks us-west --create-container us-east1,us-east2 us-east

We created a **us-east** broker (master-slave pair) in the **us-east** group on two container instances in a data center (somewhere on the east coast!).
We also defined we want to network this broker with the broker in **us-west** group. Let's now create that brokers

    mq-create --group us-west --networks us-east --create-container us-west1,us-west2 us-west

Now we have two master-slave pair of brokers mutually networked, like shown on the following diagram. 

![Broker Network](/images/fabric/fabric-network.png)

There's also an example profile that demo this topology. You can run

    container-create-child --profile example-mq-cluster  root example

and see a producer sending messages to east coast cluster, using

    discovery:(fabric:us-east)

and consumer consuming messages from west coast cluster using

    discovery:(fabric:us-west)


## Customizations

So far we've seen how to run and connect to the brokers in Fabric environment. But we never discussed where the configuration is and how we can customize
broker behavior. As with standalone brokers, the most of the behavior is configured through XML configuration and tweaked using properties.
The difference, when you're setting up things in Fabric is that your XML configuration is stored in ZooKeeper registry and properties are kept in
appropriate configuration pids in broker's profile.

For starters let's see what can you do with properties. If you take a look at the example of creating a networked broker with **mq-create** command, like

    mq-create --create-container broker1 --networks default broker1

You can expect new Fabric profile to be created (called **broker1**) and assigned to a newly created container of the same name.
To understand better how broker is configured, let's take a look at the profile

    FuseFabric:karaf@root> profile-display broker1
    Profile id: broker1
    Version   : 1.0
    Parents   : mq-base
    Associated Containers :

    Container settings
    ----------------------------

    Configuration details
    ----------------------------
    PID: org.fusesource.mq.fabric.server-broker1
    standby.pool default
    connectors openwire
    broker-name broker1
    data /opt/fabric8-karaf/data/broker1
    config zk:/fabric/configs/versions/1.0/profiles/mq-base/broker.xml
    group default
    network default

You can see that under the configuration details we have a bunch of properties that defines broker behavior. Some of those, like **network** or **group**
we were explicitly setting using **mq-create** in previous example. You probably won't touch basic things once you created your broker profile but you can
add more properties there to further tune it.

### Networks

For example, one of the differences between Fabric and standalone setup is how networks are created.
As network connectors in this case uses Fabric discovery to find brokers it'll connect to,
they are configured using properties. You can additionally configure network connectors by adding properties prefixed with **network.** to the configuration pid.
For example

    profile-edit -p org.fusesource.mq.fabric.server-broker1/network.bridgeTempDestinations=false broker1


will add

    network.bridgeTempDestinations=false

to the properties and making the broker reconfigure the network connector.

### Configuration

As you could see in the previous example, a default XML configuration template for the brokers is taken from the ZooKeeper path

    /fabric/configs/versions/1.0/profiles/mq-base/broker.xml

and it looks like

{pygmentize:: xml}
    <beans
      xmlns="http://www.springframework.org/schema/beans"
      xmlns:amq="http://activemq.apache.org/schema/core"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
      http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">

        <!-- Allows us to use system properties and fabric as variables in this configuration file -->
        <bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
            <property name="properties">
                <bean class="org.fusesource.mq.fabric.ConfigurationProperties"/>
            </property>
        </bean>

        <broker xmlns="http://activemq.apache.org/schema/core" brokerName="${broker-name}" dataDirectory="${data}" start="false">

            <destinationPolicy>
                <policyMap>
                  <policyEntries>
                    <policyEntry topic=">" producerFlowControl="true" memoryLimit="1mb">
                      <pendingSubscriberPolicy>
                        <vmCursor />
                      </pendingSubscriberPolicy>
                    </policyEntry>
                    <policyEntry queue=">" producerFlowControl="true" memoryLimit="1mb">
                    </policyEntry>
                  </policyEntries>
                </policyMap>
            </destinationPolicy>

            <managementContext>
                <managementContext createConnector="false"/>
            </managementContext>

            <persistenceAdapter>
                <kahaDB directory="${data}/kahadb"/>
            </persistenceAdapter>

            <transportConnectors>
                <transportConnector name="openwire" uri="tcp://0.0.0.0:0"/>
            </transportConnectors>
        </broker>

    </beans>
{pygmentize}

Note that some of the properties from the configuration are used in this template (like **broker-name** and **data**)
and it's important that you reuse them properly. You can edit this configuration with some of our tools like IDE or FMC
but also you might want to use the XML template from some other place. For that you can import the config in the ZooKeeper registry like

    zk:import -v -t /fabric/configs/versions/1.0/profiles/mq-base/mybroker.xml /workspace/configs/mybroker.xml

Now you can create a broker with **data** argument specified and use this customized configuration template, like

    mq-create --config mybroker.xml broker1

Note that **config** option will assume that configuration is stored in the current version of the **mq-base** profile, so you need to specify only file name without path.


### Data

One more important thing when you run master slave configurations with persistent brokers is where your store is located.
That's why it's important that you can define data directory for your broker with **mq-create** command. The example

    mq-create --create-container broker1 --data /var/activemq/broker1 broker1

will create **broker1** broker and use **/var/activemq/broker1** directory for the data (and store) location. Then you can mount some shared storage to this path
and share storage among brokers in the master slave setup.






