Tim,

I have been developing components using ATL, which communicates with custom hardware. These components are then accessed from ASP scripts running on Personal Web Server. With this setup I am running 20 remote data acquisition systems from a central location.

My question is, what open-source technologies should I be investigating to achieve similar capability. One possibility is staying with Windows and COM, but using some open-source tools. The other possibility is using Linux. I have been thinking of seeing what I can do with Apache, Java Server Pages, and XML. I could use CGI, but I prefer the ASP and component style of developing web applications. For most people, performance and scalability is important, but I am developing single user applications, so ease of development is most important.

Bill,

Since your question started from a set of Microsoft technologies with which I'm not personally all that familiar, I passed it along to Jon Udell, author of our book Practical Internet Groupware, and a really awesome thinker about the future of both windows component-based software and the web. His answer is pure gold, so I'm not going to try to put any additional varnish on it:

--Tim

There are two levels to this issue. First: the openness (or not) of the pieces that talk to your custom hardware and do the data acquisition. This does sound like a good candidate for an open-source project, especially since data acquisition is often appropriately done with el-cheapo low-end hardware. But what component model should an open-source developer target for this stuff? CORBA? XPCOM? There isn't yet a widely-accepted, broadly-deployed open-source alternative to the Microsoft model for this level of what I'd call "microcomponentry."

Second: the openness (or not) of the interface between your data-collecting computers and the outside world. Here, I'd argue that you already are open, because you're using Web protocols that look the same to clients no matter if it's ASP or JSP or CGI or any other method of serving those protocols. That said, I think there is an opportunity to elevate the game from HTML screen-scraping to a more robust method of exchanging well-structured XML packets. Again, this can be accomplished irrespective of the lineage of the underlying HTTP substrate. I call this level "macrocomponentry."

I have lately started to wonder whether the distinction between microcomponents and macrocomponents is as severe as we often imagine. In principle, why couldn't your data-collecting components export an HTTP/XML interface locally, to the application that controls them, which in turn then exports an HTTP/XML interface to the outside world?

To take an example from another realm, I recently wrote about some problems achieving "database dialtone" in the Unix/Linux world. Specifically, I noted that the ODBC model--driver manager and pluggable drivers--has never really taken root in Unix/Linux. (See http://www.byte.com/column/BYT19991104S0001.) In the aftermath of that column, I started to speculate about an HTTP/XML solution that would bump things up a level. Sure enough, somebody's not only thought it, but done it! See this message about ODBC Socket Server in my discussion group: http://www.byte.com/nntp/databases?comment_id=3863#thread.

I'm starting to wonder: what can't you do with macrocomponent-style (e.g. XML over HTTP) technology? Likewise, when must you really use microcomponents (COM, CORBA, XPCOM, beans, etc.)? We say that microcomponents are faster, but I'm not at all sure we're comparing apples to apples when we say that. Sure, COM is faster than DCOM (or HTTP), but how much is network latency, how much is intrinsic overhead, and when and to what extent does the intrinsic overhead really matter? Do we know for sure that local XML/HTTP would not work as a microcomponent architecture, for plugging in parts of an app that run locally?

MS is now pushing SOAP (an XML over HTTP protocol) as the way to extend COM services over the WAN. The assumption is: start with COM, because it's faster/better, and then switch to SOAP as needed. What if the assumption were the reverse? Start with SOAP, because it's simpler/more-flexible/more-universal, then switch to COM in those cases where SOAP's local performance becomes an issue? Maybe that conversion would be from XML-over-HTTP to XML over some hotrod local transport?

To me the benefits of this look like: simplicity, flexibility, universality, and local/remote transparency. Caveat: I'm a total Web head; I just want all software development to be as easy as Web software development. I'm sure people coming from the C++ and COM side of the world think this is ludicrous. But I do think there are lot of folks in your shoes, for whom, as you say, ease of development (and, I would add, the network effects of open-source) matter more than raw performance.

--Jon

Return to: Ask Tim Archive