ipyclam: Embedded IPython console in NetworkEditor

Take a look at the screenshot.

Yes, it is what it seems to be: A Qt-based IPython console embedded into CLAM NetworkEditor. Yes, ipyclam and NetworkEditor share the same network. And yes, sadly, although ipyclam sees anything we do in NetworkEditor. NetworkEditor does not refresh on ipyclam changes yet.

Still it is quite a big step forward because we have been blocked at that point for a while and it has been unblocked in just an afternoon.

In the past months, Xavi and me spent many hours looking for a proper API in IPython to do what we wanted without getting too low level (and postal). Now that IPython provided that new API (and documented examples), it has been quite straight forward to integrate.

A text console but it is not

The new Qt based IPython console is amazing. It is even better in many ways than the standard IPython terminal most of us are used to. It provides:

  • syntax highlighting as you write,
  • graceful multi-line edition,
  • function API hints when writing calls,
  • navigable tab completion,
  • customizable graphical representations of results,
  • many hooks to add our bells and whistles.

So it seems a text console, but it is not. Yep, we can add many bells and whistles there.

Back to the original ipyclam goal

Lately, we clearly realized that ipyclam strength is clearly empowering CLAM prototyping, by adding interactive Python scripting and PyQt/PySide to the QtDesigner/NetworkEditor graphical design tandem. But the original motivation for ipyclam, which was very shortsighted, I must admit, was providing NetworkEditor with a complementary console to deal with big and complex networks where graphical patching was not practical. And that’s a milestone that we have not reached yet.

I’ll try to abstract the problem so that it could be useful beyond CLAM. We had a domain object implemented in C++ and we wanted to provide a Python console (a Qt widget) to manipulate it. So the first goal, which Xavi achieved very succesfully, is to build a Python wrapper, ipyclam to the domain object, a CLAM::Network.

IPython separated quite well the concepts of Python kernel from the Qt Console. Indeed it separated those concepts that much that they were at different processes and that made our goal unfeasible, at least, by using the stable high level API of IPython.

Luckily, IPython 1.3 provides high level interfaces and examples for having both kernel and console in the same process connected via a local pipe. I took that code and changed it to be able to inject the CLAM::Network and I got a Python script that serves as Qt based ipyclam_console which I named ipyclam_qtconsole.py and it is working at the repository. I modified the code from Jarosz Pawel to get the kernel encapsulated inside the console widget itself and I added a function to easily injecting objects into the interpret main namespace, for example the ipyclam network object.

Still the example was a Python program using a PySide base widget. Gladly, we already had taken our quest to get the PySide and PyQt4 working all together with C++, and we already have a few functions to transfer up and down QObjects from and to C++. So if I can abstract the whole thing into a QWidget I could use it as is and I still can’t believe it worked that well.

namespace py = boost::python;
QWidget * GetIPyClamConsole(CLAM::Network & network)
{
    try
    {
        Py_Initialize();
        // Dummy __main__ namespace, to run execs
        py::object _main = py::import("__main__");
        py::object _main_ns = _main.attr("__dict__");
        // Adding working dir to the Python search path
        py::exec("import sys; sys.path.append('.')" , _main_ns, _main_ns);
        // Simulate that we have a working command line (expected by IPython)
        py::exec("sys.argv=['ipyclam']\n", _main_ns, _main_ns);
        // Build an ipyclam network having the CLAM network as backend
        py::object ipyclamModule = py::import("ipyclam");
        py::object proxy = py::object(py::ptr(&network)); // The proxy backend
        py::object net = ipyclamModule.attr("Network")(proxy); // The ipyclam network
        // Creating the IPython based console widget

        py::object consoleModule = py::import("ipyclam.ipyclam_qtconsole");
        py::object console = consoleModule.attr("IPythonConsoleQtWidget")();

        // Injecting the network into the namespace
        console.attr("namespace_inject")("net", net);

        // Unwrapping the PySide based qt console to use it as a abstract QWidget
        QWidget * consoleWidget = (QWidget*) shibokenUnwrap(console.ptr());
        return consoleWidget;
    }
    catch (py::error_already_set & e)
    {
        std::cerr << "Run time Python error!" << std::endl;
        PyErr_Print();
        return 0;
    }

}

TODO’s

Well, still fully not working, what else we have to get done:

  • Clean-up dependencies
    • NetworkEditor depends on ipyclam on run-time
    • NetworkEditor depends on Boost-Python and Shiboken on compile-time, has to be both?
  • Refresh the canvas when Python changes the network, including:
    • Processing creation and deletion
    • Processing connection
    • Configuration affecting connector
    • Renames
    • Changes on the back-end
  • Providing documentation so that the Qt console hints help the user
  • Polishing ipyclam for some nasty things you get when you play a little with it.
  • Controling the banner and the prompt (we could control it on Terminal based but the same interface seems to be gone in Qt)

Comments

0 comments.

Leave a comment

 (optional)
 (optional)