Cloud native solutions are available in abundance and for almost every problem, there is a product that can be used as a managed service. However, sometimes we come across use cases where nothing fits to a T, and/or the cost of implementing the native solution is detrimental.
We came across such a use case, and designed a product which runs at a fraction of the cost and administration required of a native cloud solution. Its been a challenge, but a lot of fun.
The finished product is what we call a “multi-schema-multiple-output-Stream” which has the possibility to apply schemas (not infer schemas, which would have been a bit easier) to raw data and output data in multiple formats with a latency which is 1/100th of what we had (near real time).
The product also saves millions in investment that the client does not have to make to get similar functionality. Sometimes, it’s just better to make your toys, rather than buy them from a shop.
Leave a Reply