ipyclam: interactive python console for CLAM networks manipulation
Xavier Serra (not the MTG head, but an homonym student at the UPF) is about to reach an interesting milestone on his Final Career Project related to CLAM. He is developing an interactive Python console for CLAM named ipyclam. It is a text based interface to explore and manipulate CLAM networks. Hopefully this will end being a dock widget within the Network Editor but it is also useful as interactive text console tool and as programming interface, which are the first useful outputs Xavi is about to get. ipyclam comes to cover the need of manipulating complex networks in cases when point and click is just tedious and also that of generating parametrized networks.
The API
Let’s make clear that it is not a full wrapper for the CLAM API like Hernan’s pyclam, ipyclam just enables high level operations you usually do with Network Editor: instantiating, connecting, configuring and playing modules. Moreover we have intentionally decoupled ipyclam API to any existing CLAM API in order to define the most pythonic, convenient, and clean API we could think. For example the code to build an stereo cable would be:
net = Network() net.processing1 = net.types.AudioSource net.processing1.NOuputs = 2 net.processing2 = net.types.AudioSink net.processing2.NInputs = 2 net.processing1.Output1 > net.processing2.Input1 net.processing1.Output2 > net.processing2.Input2 net.play()
An interesting feature, is that network.code() just returns an string with similar python code, so a network knows how to regenerate itself. This opens the door to python based format for CLAM networks. Bye bye XML!.
Convenience? Yes, please. Connections can be done in batch, broadcasting connections from all the connectors in one processing to all or to a single connection of another:
net.processing1 > net.processing2 # all to all net.processing1.Output1 > net.processing2 # one to all net.processing1 > net.processing2.Input1 # all to one
Convenience is good but all things should be doable. Often most convenient solution is not the best suited for all the cases. Accessing configuration parameters and connectors of any kind directly from proccessing is quite convenient but names can collide, to solve that some special attributes are provided to restrict the name scope.
network.processing1._config.NOutputs network.processing1._inports.InPort1 network.processing1._outports.OutPort1 network.processing1._incontrols.InControl1 network.processing1._outcontrols.OutControl1
Connection names can have spaces, and attributes cannot, so we also provide an alternative subscript syntax.
net.processing1['Output 1']
One of the features I like the more is tab key exploration.
- network.[tab] and you get the list of processings
- network.processing._inports[tab] and you get the list of input ports
- network.processing._config[tab] and you get the list of configuration parameters
- network.processing.[tab] and you get connectors and cofiguration parameters
- network.types.[tab] and you get the list of available processing types
It works for interactive console with tab completion like ipython. We also provide ipyclam_console, an ipython based shell conveniently importing modules and adding a network to start playing with.
For more details on the API just check this document.
The implementation
ipyclam is defined in two layers. The front-end layer, purely written in python, makes the convenient API happen: Exploration, dynamic object attributes, many interfaces to similar operations… This upper layer, relies on the lower, less pythonic and more purpose oriented one, the network proxy layer, which provides access to the state and operations of the actual network with quite fewer entry points.
Xavi defined the front-end layer while evolving, in parallel, a dummy proxy layer simulating network status with basic python structures. That gave us the flexibility to explore and evolve both the frond-end and the proxy interface without bothering about the actual CLAM implementation that would have made such exploration quite rigid.
Two layers approach is also convenient because by defining the ops in the network proxy layer, we are able to reuse the front-end API with any other patching like environment. I am thinking on JACK or environments like ingen.
Status and road map
Xavi just finished most important bits of the API: Inspection, module creation, module type enumeration and module inter-connection. Currently we still rely on a dummy network proxy, but a CLAM based proxy will arrive during next week. Configuration API seems to be controversial and we are holding it until we have something implemented with CLAM.
Once we have a full CLAM based proxy the next big goal is to integrate the console in NetworkEditor. We need a console widget for Qt which is able to execute a python interpret. We found some candidates:
- qconsole which already has a python wrapper
- qtermwidget a more robust console (based on KDE Konsole) but without direct support for python, and
- ipython qt frontend quite experimental and using a weird threading model, but still interesting as ipyclam_console already uses ipython and it is awesome
So either candidate seems to require some work on our side.
Well, I hope you will find this new CLAM feature as amusing as I do. Xavier is really doing quite a good job, congrats and keep it that high.