Queue Processors

Queues - Solace

Choosing Solace

Why did I choose Solace?

The real reason, I chose Solace because the office did a study on queue processors (I was not part of this study). The results of the study determined that Solace was the best queue processor for our purposes.

But that was a few years ago.

Before installing Solace I did a bit of studying. I did web site searches. I decided that for the most part, for my general purposes I could use any queue processor. It’s when you start your second or third serious project that you start to recognize that you may need to use a specific queue processor for your needs.

I figured I would use Solace, but I wanted to confirm that I wasn’t making a mistake. So I asked AI to compare Solace against a few other processors. Here are the results.

The Server

The Solace server comes in 2 flavors. (at least)

  • Solace Cloud
  • Solace On Premise

I use use the Solace On Premise server. It has a requirement that it runs in a docker container.

Here is the docker script (Compose.yml)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
version: '3.6'

services:
  solace:
    image: solace/solace-pubsub-standard
    shm_size: '1000m'  # Set the shared memory size // required for Solace
    ports:
      - "8007:8080"  # Web UI
      - "55555:55555"  # SMF
      - "8008:8008"  # MQTT
      - "1883:1883"  # MQTT
      - "8000:8000"  # Health check
    environment:
      - USERNAME_ADMIN_GLOBALACCESSLEVEL=admin
      - USERNAME_ADMIN_PASSWORD=admin
    volumes:
      - solace-data:/var/lib/solace
    build:
      context: .
      shm_size: '2gb'

volumes:
  solace-data:

You can access a server administration page at http://localhost:8007

The Clients

STM

This is short for ‘Solace Try Me’ it’s a CLI tool that allows you to test the server. You can use it to do lots, like send and receive messages.

It’s a good starting point.

It runs in a Linux environment. (On my home computer I opened WSL and was able to run STM through WSL) Here’s what I do.

1
2
Start > Powershell > 
Start powershell > Wsl.exe -d Ubuntu

You are now in a Linux environment.

sample commands:

1
stm -h
  • Gives a list of commands
1
Stm –he
  • Gives a list of sample commands
1
Stm config -h
  • Gives help about the config command
1
Stm config init
  • Initializes stuff. Sets p an event broer named SEMP
1
Stm manage queue --create /test/take1

Creates a queue named /test/take1

Website

There is a website that allows you to configure the server.

http://localhost:8007/#/login

You’ll need a username and password (hint, refer to the compose.yml file)

Your Producers / Consumers

One of the things that Solace advertises is that you can connect to the server from any language, using many different protocols.

The quickstart
https://docs.solace.com/API/API-Developer-Guide/Quick-Start.htm
Gives you instruction for creating applications in Java, .Net, Node.JS.

I learned first playing with STM, and then using the .NET tutorials

Architecture

First is the application design.

  • Do you need a queue server in the first place? Will your applications benefit from splitting the workload between multiple processes? Can they be architected to use a queue server?
  • What type of queues will you need?
    • Lifo?
    • Fifo?
    • Does one process broadcast messages to multiple clients? (Similar to Facebook or Twitter)
    • Does one process send messages to a single consumer?
    • Does one process send messages to multiple consumers?
    • If you split to multiple consumers, does the message go to all consumers? or just one consumer?
    • What happens with a new consumer? Does it get all the messages added to the queue?
Setup a Publish / Subscription environment

Solace Topics

Topics

A topic is like a queue name.

One of solaces strengths is that they will support scaling up by pulling out topics out of a queue and directing it to another server.

Here’s how it works.

You define a hierachial based topic name for example /AP/Vendor/Add, and /AP/Vendor/Edit, and /AP/Statement/Add.

Then if you find the consumer slowing down, you can setup a second consumer and direct messages with the /AP/Statement/Add topic to the second consumer.

Topic Naming

When defining a topic name, it is recommended you consider the kinds of messages you will be sending on the queue. You may decide later to pull out some of those tasks for later processing. If you think this will apply then you should define your topic names in a hierachial way using slash (‘/‘) as a separator.

It’s elegant, but you must define your queue names in a hierachial way.

  • Hierarcial definition of the subject of an event message.

    • Example a/b/c.
    • Topic subscription

  • String used to subscribe to topics

    • Ex a/b/c,
    • Ex a/*/b - * is a wildcard to match that level
    • a/> - will match a/b, and a/b/c

  • Endpoints and Queues

    • Queues – an endpoint configured with topic subscriptions that clients can publish messages to and compute messages from.
      • Recommended – or the normal
      • Queues can be durable or non-durable
    • Topic Endpoints –
      • Message Reply
        • Used to resend messages to new or existing clients. Canbe performed on a queue or topic endpoint. Kept until the log is full.
      • Message VPN
        • A security envelope. If you had 2 message VPNS, then publisher A, subscriber 1 and 2 could be in one of those message vpns, while publisher b, subscriber 3 and 3 are in the other message vpn.
        • Those 2 message vpns will keep identically named topics separate.
        • Each client connection is associated with a single Message VPN. Part of the loginis to specify which Message VPN.
        • Out of Box – one is created named default
        • Message VPN Bridge – used to allow multiple VPNS to link up to the same topic.

  • Event Meshes

    • A network of individual event brokers
    • Similar to Message VPN Bridging – but done differently
    • Considered the best way.
      • Dynamic Message Routing