This document serves as a brief introduction to the concepts of Substance and Data-orientation together with a Substance programmers guide. Substance is the core of Shared Substance, it is a programming framework built upon Python implementing the Data-Oriented programming model.
Unlike Object-Orientation (OO), Data-Orientation (DO) supports a fundamental separation between data, represented by Nodes, and functionality, represented by Facet. Nodes are organized in a tree, where each node is uniquely identifiable by its path. A node may have zero or more facets associated with it. Nodes and facets can be dynamically added and removed at run-time, i.e. functionality can be associated to and dissociated from data at run-time. Our motivation for introducing DO is that the hierarchical structure and separation between data and functionality provide a flexible foundation for easily sharing any application data in a distributed system. sharing both application state and behavior in a distributed system.
A DO application consists of a root node with a facet containing the main entry point of the application. Facets resemble aspects in aspect-oriented programming. However, while aspects typically address system-wide concerns such as serialization, facets are local to their node. Facets also resemble dynamic mix-ins that can be added and removed at run-time, but they are more cleanly separated from their data node and from each other than mix-ins.
Substance is our reference implementation of the data-oriented programming model. Rather than creating a new language from scratch, we used Python, taking advantage of its reflection and meta-programming capabilities. This also gives us access to a rich set of Python libraries.
Substance implements the basic DO concepts, Nodes and Facets. Nodes contain values, the equivalent of OO fields, while facets contain handlers, the equivalent of OO methods, publishers, to emit events, and errors, for exception handling. Nodes and facets are addressed through paths, similar to Unix paths or Web URIs. Each node is instantiated with a set of default facets. For example, CoreStructural provides functionality for adding and removing children nodes, while CoreValue provides local value manipulation.
Programming an application with Substance consists in programming its facets. In our reference implementation, facets are Python objects and can therefore inherit from any Python class. A renderer facet can, e.g., inherit from a Qt Graphics View on Linux or from a Cocoa View on Mac OS X. When a facet is added to a node, Substance calls its instantiate method and its destroy method when it is removed, allowing the developer to respectively set up and clean up values.
Substance supports both reactive (event-driven) and imperative (message-driven) programming: Facets can listen to some or all the events published by another facet and trigger handlers; Facets can also send events directly to other facets as messages, using the traditional syntax of a method call. A large part of the implementation of Substance uses reactive programming, especially by listening to CoreValue and CoreStructural events. For example, a rendering facet listens to changes in both the structure and values of a scene graph to update the display.
All application logic in Substance is expressed through facets. The following code example shows the minimal Substance environment, with no application logic at all.
from substance.environment.bootstrap import *
class MainFacet(Facet):
def __init__(self):
Facet.__init__(self, "MainFacet")
def instantiate(self, with_state):
#This is where the application logic would go
boot_std_default(name = "MainFacet", description = "An example environment")
boot.add_facet(MainFacet(), environment / "app")
done_boot()
The first line imports the Substance framework, followed by a Python class that is our main facet of the environment. The first of the three last lines will boot up a default Substance environment with a name and a description. In the second to last line we add our facet to the environment/app subtree. environment/app is reserved for application development. Finally we indicated that we are done booting.
In Substance nodes and facets are accessed by paths. The root path of a Substance environment is accessible by the global environment path. To access e.g. the app subtree with a path the following is possible:
path_to_app = environment / "app"
Facets are also accessible by paths, for instance here is the path of the MainFacet above:
path_to_mainfacet = environment / "app#MainFacet"
A facet can access its own path in the following manner:
path_facet = self.get_path()
And a facet can access the path of its node in the following manner:
path_to_facet_node = self.node.get_path()
A shorthand for accessing the local node of a facet is using the local keyword:
child_node = local.new_child(self, "ChildNode", A child node)
Calling a handler on a facet resembles calling a method on a Python object:
path_to_node.SomeFacet.some_handler(self)
A handler can also be called directly on a facet path:
path_to_som_facet.some_handler(self)
(NB: due to internal event propagation mechanism, self must always be passed along in handler calls)
Nodes have values, and values are created and modified through the CoreValue facet.
local.CoreValue.new_value(this, "Foo", "Some value" 42)
local.CoreValue.set_value(this, "Foo", 43)
It is possible to shorthand the usage of any of handlers on the CoreFacets
local.new_value(this, "Foo", "Some value" 42)
local.set_value(this, "Foo", 43)
Manipulation of the trees are performed through the CoreStructural facet e.g.:
child_path = local.CoreStructural.new_child(this, "ChildNode", "A child node")
or:
child_path = local.new_child(this, "ChildNode", "A child node")
To remove a node:
local.remove_child(this, "ChildNode")
Facets are added and removed through the CoreFacet facet.
child_path.add_facet(self, SomeFacet())
child_path.remove_facet(self, "SomeFacet")
Facets can be added and removed dynamically during runtime.
To call a handler on a facet, you must access the facet through its path. E.g. to call some_handler on the SomeFacet above, you would do the following:
result = child_path.SomeFacet.some_handler(self)
Substance supports reactive programming by using listeners. Below is two classes. First a ListeningFacet that will install a listener ValueLister on the node it is installed on. The value listener listens on incoming events to CoreValue that are either new_value or set_value. This means that if some other facet sets a value on the node ListeningFacet is installed on value_updated will be called.
class ListeningFacet(Facet):
def __init__(self):
Facet.__init__(self, "MainFacet")
def instantiate(self, with_state):
local.add_listener(self, ValueListener(self.get_path())))
@Facet.HANDLER("<void>::<void>")
def value_update(self, to_path, name, value):
#handle update
class ValueListener(Listener):
the_name = "Simple Value listener"
the_description = "Listens"
def __init__(self, callback_path):
Listener.__init__(self, self.the_name, self.description, "CoreValue", ("new_value", "set_value"))
self._callback_path = callback_path
def trigger(self, event):
self._callback_path.value_update(self, event.to_path().node_path(), event.payload_element(0), event.payload_element(1))
The constructor for a listener looks like this:
def __init__(self, name, description, facet, commands, is_regexp = False):
"""
<name>: The name of the listener. On a given node, listener names are assumed to be unique
<description>: A text describing the intention and behaviour of the listener
<facet>: The name of the facet on whose events the listener wants to listen
<commands>: A list of the commands of which the listener wants to listen
<is_regexp>: States whether <facet> and <commands> should be treated as regular expressions
"""
...
To support reactive programming it is also possible for a facet to raise an event is self by using publishers_. Publishers must be declared in the facet as shown below:
class SomeFacet(Facet):
...
@Facet.PUBLISHER("<void>::<parameters>")
def publisher_command_name(self, parameters)
self.publish.publisher_command_name(self, user-parameters)
Now calling the handler publisher_command_name will publish an event that can be listened to by listening to publisher_command_name on SomeFacet.
Facets may access dependencies installed on nodes. In our Basic rendering example we install a web based image broker facet that has a dependency to a scene graph that it produces data for. This is expressed something like this:
class ImageBroker(Facet):
@Facet.DEPENDENCY("SceneGraph", "The root node of the scene graph")
def __init__(self, port):
Facet.__init__(self, "ImageProvider")
...
def instantiate(self, with_state):
...
sg = local.get_dependency(self, "SceneGraph")
The dependency is installed on the node the dependent facet resides on, so from a main facet this will look like this:
class Main(Facet):
def __init__(self):
Facet.__init__(self, "Main")
def instantiate(self, with_state):
scenegraph = local.new_child(self, "Scenegraph", "This is the root node of the scenegraph for WILD")
...
self.ib = local.new_child(self, "ImageBroker", "A webserver for adding images")
self.ib.set_dependency(self, "SceneGraph", scenegraph)
self.ib.add_facet(self, ImageBroker(3444))
The Substance programming model is designed to support run-time inspection. Inspection in Substance is very helpful both for debugging purposes, but also for getting an overview of how a Substance environment looks at run-time. It is recommendable to try both of the two techniques for inspection out.
Out of the box a simple tool for easy inspection is provided in the Substance standard library. If the following line is added to the Substance bootstrapping:
boot_std_feature("resource.net.bearers.http")
Now when running the environment it will print something like the following:
2011-10-13 18:54:21,656 - substance.environment.bootstrap - INFO - /environment : Booting std feature : resource.net.bearers.http : {}
2011-10-13 18:54:21,693 - substance.std.resource.net.bearers.http - INFO - /environment/resource/net/bearers/http#HTTP Bearer : HTTP socket instantiated on port : 51909
2011-10-13 18:54:21,695 - substance.environment.Environment - INFO - /environment/local#Environment : Registering capability : http bearer port : The port of the local HTTP bearer. : 51909
In this case it is now possible to inspect and traverse the environment by accessing http://localhost:51909
If the HTML inspection is a bit too mundane for you, it is possible to get an experimental realtime 3D visualization of a Substance environment. It requires a running Ubigraph server. (In /resources there is a runnable binary for Mac). Now just add the following line in the imports of the main source file of your Substance environment:
from substance.visualization.ubigraph import *