September 2013 Project Update

Greetings. It’s been several months since my last post here and there’s a lot of new stuff to talk about.

In May I gave an hour-long demo and Q&A on the Encapsule Project Schema HTML 5 application at the Seattle Graph Database Meetup group. I felt like maybe half of the twenty or so people who came that evening understood the whole point of Soft Circuit Description Language (SCDL). Of that ten, perhaps two people were able to relate the concepts I discussed back to the rather boring Schema HTML 5 app demo I brought along to show.

Not a big win but I did get some good questions. The most interesting was if I was going to enable to programmatic creation of SCDL model and specification data or was simply going to provide the Schema SCDL Editor. I hadn’t previously considered programmatic generation of SCDL to be a high priority and was curious about what people would use such a facility for if it did exist.

Programmatic Generation of SCDL and Program Annotation

Why do we need a programmatic way to generate SCDL in addition to the Schema SCDL editor? The answer is fascinating; providing an API for manipulating SCDL programmatically makes it possible to write programs that write programs (i.e. a program synthesizer). Synthesis as you’ll recall involves the combination of two or more things to make something new. A SCDL program synthesizer specifically leverages SCDL as a domain specific annotation language.

For example a SCDL model set of re-usable data visualization component comprises a kit for building entire class of visualization applications (the class of all possible allowed combinations of the various SCDL models). A SCDL program synthesizer program could be written to glean salient information and features of an input data set(s) (e.g. via parsing), and use this information to generate a SCDL specification to visualize the data in terms of the available SCDL data visualization components.

In this example, what we’re annotating is how we want another program to visualize the input data given its structure, features, content… The resultant “annotation”, a SCDL specification, could then be executed to visualize the data, or serialized and passed along with the data to some other processing entity (e.g. to respond to a request).

To take a far less trivial example, the original question was posed by a gentleman whose job it is to figure out how to programmatically port thousands of legacy enterprise applications to the web. Plausibly I can imagine SCDL providing a framework for building up such transformation systems in a predictable way that scales. The viability of this approach of course depends on much infrastructure that doesn’t yet exist so currently this is theoretical. However, the entire exchange convinced me that I must provide a way to programmatically generate SCDL models and specifications.

In the course of this discussion it was mentioned that I’m rather blithely ignoring the semantic web. I countered that I’ve tried but just don’t get it and that perhaps by working through things my way I’ll understand. Or, be able to demonstrate a simpler way to affect the same ends using data flow graphs.

Schema HTML SCDL Editor and SCDL API

I’ve spent this entire year so far essentially trying to figure out how to write HTML 5 apps like Schema. It’s actually quite a difficult task to build an interactive data editing and visualization application when your data model is complicated. There’s a ton of runtime state and keeping the code clean defect free is extremely hard unless you’ve got some solid low-level infrastructure code.

As of the May demo I was still struggling with fundamental architecture questions about how I was going to structure the application and prevent myself from building something so sprawling that I would not longer be able to comprehend it.

Questions about support for a SCDL API got me thinking that the whatever solution I came up with had to both provide a nice and clean API but additionally support the following core scenarios for building interactive data editing / visualization apps in the browser:

  • The app data model must be declared at design-time and then used at runtime to tailor the specific functionality of generic routines to ensure that changes to the data model do not impose huge refactoring penalties.
  • All application data needs to be addressable in order to ensure a clean decoupling between the many routines that operate on it. Further the addressing schema must accommodate local hash URL routing as well as some future URI-based protocol.
  • It needs to be easy to segregate actual application runtime data from metadata.
  • It needs to be easy to write simple routines that operate on both app data and app metadata stores.
  • It must be trivial to save / restore all runtime app state in JSON format.

My summer has been consumed with designing a solution that meets all the above criteria and the result is a new Javscript library I call ONMjs.

ONMjs

ONM stands for Object Namespace Manager. The ‘js’ is tacked on the end to signify that this is a Javascript implementation of the library (others are certainly possible but for now, just Javascript).

The currently published version of Schema is essentially a complete re-write of the demo I gave in May that leverages the new ONMjs library to manage all of its application state.

What you see in the browser, in the little window panes, are actually little plug-ins that observe and react to changes in runtime application data via a sophisticated implementation of the Observer Pattern. Some even change the data and ONMjs takes care of letting everyone else know.

Over the next week I’ll be forking the ONMjs library into its own repo on GitHub and plan a series of small example apps and blog posts to explain what it is and how to use it in your own applications.

ONMjs is completely generic, is dependency-free Javascript, and is intended to be used in conjunction with libraries such as D3js and Knockoutjs (or with any number of other libraries).

If you’re working on this sort of web app ONMjs wants to be your friend. Stay tuned.

  1. No comments yet.

  1. No trackbacks yet.

You must be logged in to post a comment.