rtd.pub is a platform for easily connecting any real-time data source to Microsoft Excel. You can write code in Go and Python, or simply configure pre-built open source connectors.
This is a design and approach I’ve been mulling over for the past couple of months. I have now implemented it.
This, and of course spreadsheets in general, are somewhat boring. However, they’re the original low code/RAD tool: in terms of bang for buck and the control they put into the hands of domain experts, nothing comes close. (No, not even some sketchy code regurgitated by a large language model - yet anyway.)
I appreciate this is all quite niche, but if it saves time and makes lives easier, why not?
History lesson
Over the years, there have been many ways of extending Excel. The original C API (ancient, fast, hard to use - if you look closely you can see evidence that Excel was originally written in Pascal), COM, C++, VBA, C# … all on Windows, of course.
Current versions contain a browser engine (Edge/Chromium on Windows, Safari on Mac) and expose a JavaScript SDK through it. This is technically very cool and has a lot of use cases, but unfortunately is quite slow for very fast moving data.
Needless to say, it’s a bewildering array of choices in unfamiliar territory for many developers who simply want to use their language of choice, particularly for those not from a Windows background. To management, it might appear that developers are making heavy weather of just displaying some numbers on a screen. Exactly how hard can it be?
In simple scenarios, not very, but to do it well, you will need to understand each approach, and which fits best for a particular use case. This is time-consuming.
As we have established, for very high velocity data, the JavaScript bridge is a poor choice. The trusty old =RTD() function that leverages COM automation is still the best choice. But then you need to understand COM, threading and … what language to even write your COM server in. “Hey Google, what even is a COM server?”
How does rtd.pub help?
It shields developers from the above by building in know-how, workarounds and patterns to keep code simple and domain focused. Developers can write or reuse code in any language to reliably stream values into Excel.
How?
rtd.pub boils down to three things:
- a gRPC protocol
- a unified pipe of data (UNIX sockets, H2) into Excel, via a conduit RTD server
- plugins that implement said protocol, bringing in any data from anywhere
An RTD server (written in C++) communicates with tiny processes called plugins over UNIX sockets. This lightweight component runs within Excel, but by design, delegates the interesting work to be done to plugins. A plugin process implements a gRPC service definition and streams protobuf messages. Plugins are started by the plugin host which supervises the process, multiplexing streams from all other plugins to efficiently pass over to Excel.
Plugins
As plugins are so easy to write (usually in fewer than 100 lines of code, depending on language) it is easy to get results quickly. To move even more quickly, there is an SDK for Go (Python coming soon) that makes it very easy indeed. Finally, to move even more quickly than that, NATS is supported out of the box. If you use NATS already, you can hook your subjects up to Excel with a few lines of configuration. Support for additional messaging systems will be added soon.
It is also simple to integrate with external vendors. For example, I wrote a plugin to the Polygon.io stocks websocket API with their official Go client.
Commercials
I will provide support, custom integrations and white label solutions for data vendors, built on rtd.pub. There will also be a small per-seat monthly/yearly subscription for the downloadable Excel add-in for Windows. Mac support will follow.
An ecosystem is important, so the following will be available under a permissive open source licence.
- The protocol / gRPC contract
- All docs, SDKs and tooling
- Core plugins, such as messaging integrations such as
natsandkafka, plus well-known data vendors - which can also be sponsored by said well-known data vendors ✨
GA soon
Pop over to rtd.pub to see it in action, and register your interest via your GitHub account.
Give me a shout in the usual places or email if you want to arrange a demo, ask questions or are just plain curious about the whole thing.