How to tackle 5 common event bus pattern problems

Data overload

With IoT materializing more and more as time passes, the amount of data created by events grows exponentially for many organizations. Using an event bus to pull all of that data together may seem attractive -- but such volumes could bring the whole network to its knees. Instead, it's better to use edge computing devices that can act as an intelligence layer to look at created events and data to decide that the data doesn't need to be transferred to the bus or to any other system. As a result, endpoint and bus architecture is the key.

Standards 

The idea of an event bus is to normalize and standardize incoming data so that other systems can make use of it. But while event bus vendors will probably provide a base set of connectors to the most common endpoints on the market, vendors do not take responsibility for ensuring that connectors work from the event bus to the endpoint.

Therefore, the primary problem that arises is when an endpoint -- particularly if it is a hardware device -- is too specialized in the market and has low uptake. It becomes cost-ineffective for the device manufacturers to try and maintain the currency of connectors -- or even the devices themselves. Over time, the connector's functionality depreciates, lessening the value of the data entering the bus. To prevent this, organizations should prioritize standardized or high-volume devices wherever possible.

Cross-platform support

Another major consideration is whether the bus must operate across more than one type of hardware or operating system -- or even a hybrid cloud platform. Even if a particular bus can operate on various platforms like Azure, AWS and Google Cloud Platform, teams should also make sure it can integrate with the data aggregation platforms like Splunk and Dell Edge Gateways, as well as analysis platforms like Pentaho, Information Builders and IBM Streams. Look for event buses that are either simple to implement across multiple clouds or that already have their services installed on major public clouds.

Finding the right use

Ideally, an event bus should act as a single engine that covers multiple different needs. For example, can the bus aggregate and present IoT data in near real time that can enable issue identification and response triggering, as required? Can it collect data from networking devices and analyze that data to identify abnormal patterns that indicate a potential distributed denial-of-service attack? 

Prioritize event buses that can integrate and operate with multiple different external systems and perform pattern matching. It may also be worthwhile to look for highly componentized event buses, rather than searching for one that builds in all the analysis and new event triggering itself. For example, Apache Kafka is a highly componentized system with lots of additional functions that users can add as required, and it enables users to replace certain functions with other tools of their choice should they prefer a best-of-breed approach.

Longevity

Once implemented, an event bus will become a core part of your system and will be difficult to swap out at a later date. Installing a bus from a vendor that cannot demonstrate long-term plans and commitment could be disastrous to your own platform and those dependent on it.

The vendor should at least be able to talk to you about what functionalities they are already planning to release within the coming year as well as what areas they still expect to need future work. Identify any area  where you believe the vendor is behind the curve. It is also important to carry out sufficient due diligence as to the health of the vendor. For instance, do they have good financial backing and a strong customer base? And what do technical experts have to say about them?

A well-designed, implemented and managed event bus is quickly becoming a necessity, so organizations must put in great time and research before jumping into an event bus acquisition and implementation.