Rendering deep text-based mindmaps with WiseMapping and Python

🔗

In this blog post, I'm going to demonstrate how to reuse WiseMapping HTML+JS rendering engine to easily visualize...

text-based mindmaps
    like this one
    have many benefits
        they are readable as-it-is
        they don't require any tool to be edited
        [they follow the UNIX tenets](http://www.ru.j-npcs.org/usoft/WWW/LJ/Articles/unixtenets.html)

For the impatient ones, here is the result (click & drag to center the mindmap):

And you'll find here all that is required to launch this viewer yourself:

./build_viewer.sh
./wisemapping_txt2xml.py text_based_mindmaps.txt > wise-editor/src/main/webapp/samples/text_based_mindmaps.xml
python3 -m http.server

You only need: git, python3, rsync and sed.

EDIT [2017/08/02]: in the end I put the viewer in a separate repo: https://github.com/Lucas-C/wisemapping-mindmap-viewer

Genesis of the project

Perfect time to slip in one of their songs.

The reason behind this experiment (and this previous one with graphviz) is that I wanted to leave the excellent Freeplane mindmapping software for a text file based solution, in order to store my mindmaps in Gitlab.

Using graphviz has some severe limitations : big mindmaps are hard to read and navigate due to the absence of a folding mechanism, links are not clickable, etc.

Then I realized an Open-Source mindmap software, WiseMapping, was in fact a web interface built on top of a Java REST API. So I planned to write a mock API in Python in order to answer the required HTTP calls to display a mindmap based on a text file.

Poking around

To begin with, I followed the project README instructions:

cd wise-webapp
mvn jetty:run-war -Dmaven.test.skip=true
cd -
mvn assembly:assembly -Dmaven.test.skip=true
python3 -m http.server

The tests systematically failed, hence I passed -Dmaven.test.skip=true to all the Maven commands. I'more Python than Ruby, so I replaced the command they used to serve the static files by a Python one.

Quickly, I realized that it would be even easier than I thought: most of the code logic was in Javascript, the API only serving as a mindmap save/restore mechanism. Zero call to the Java process is made when loading the default welcome.xml mindmap in viewmode.html !

Extracting the HTML+JS mindmap renderer from WiseMapping

If you look at build_viewer.sh you'll see that I use rsync to extract from the repo only a few files required to run the viewer. They are mostly JS & image files, and not even all of them are needed !

Looking at my browser Network tab, I figured out that the mapId = 'welcome' in viewmode.html was the name of an XML file in the [samples/] (https://bitbucket.org/wisemapping/wisemapping-open-source/src/v4.0.3/wise-editor/src/main/webapp/samples/) directory that was loaded and parsed by the app.

Instead of this default value, I just changed the code to load the XML file corresponding to the query string:

mapId = location.search.substr(1) || 'welcome'

WTF is JSPomLoader ???

Strangely, loading a mindmap was really slow. Here is what Firefox Network tab showed when loading the viewser in a new private window:

Firefox screenshot: 287 requests, 1 292,48 KB, 30,26s

I realized that for some crazy reason, when the JS editor component was loading, it retrieved the Maven pom.xml file from disk, parsed it and then loaded those files one per one ! And with jQuery adding a ?_=${timestamp} query parameter each time, to avoid browser caches !! JSPomLoader is the class responsible for this absurdity this in mindplot-min.js.

I decided to build a JS bundle instead. After some experiments with xmlstartlet and browserify, I realized it was doable in just 3 simple commands:

cd mindplot/src/main/javascript
sed -n '/<includes>/,/<\/includes>/{/<includes>/d;/<\/includes>/d;p}' ../../../../wisemapping-open-source/mindplot/pom.xml | sed -e 's~${basedir}~../../..~' -e 's~\s*<include>~~' -e 's~</include>$~~' > js_sources
cat js_sources > ../../../../wise-editor/src/main/webapp/js/mindplot-bundle.js
cd -
sed -i '$d' wise-editor/src/main/webapp/js/editor.js
cat wise-editor/src/main/webapp/js/mindplot-bundle.js >> wise-editor/src/main/webapp/js/editor.js

The second command makes editor.js use our new mindplot-bundle.js instead of mindplot-min.js by replacing its last JS line.

The result:

Firefox screenshot: 170 requests, 808,26 KB, 7,28s

Avoiding unwanted LocalStorage cache

At this stage, I could add XML mindmaps in wise-editor/src/main/webapp/samples/ and visualize them with viewmode.html?file_base_name.

However, when modifying an XML file after rendering it once, changes to the mindmap were not taken into consideration when refreshing the page.

This happened because the WiseMapping editor agressively caches mindmaps in your browser LocalStorage and does not expect changes on those XML files outside its saving mechanism. Hence I had to add this line to disable the cache:

persistence.discardChanges(mapId);

Converting text mindmaps to XML

Last but not least, remained to convert textual mindmaps into WiseMapping XML format.

In order to do so I wrote a Python script named wisemapping_txt2xml.py that uses a pyparsing to parse the pseudo-markdown syntax:

  • Markdown:
Markdown styles
    Font styles
        **bold**
        __italic__
        __**bold italic**__
    Hyperlinks
        [My blog](https://chezsoi.org/lucas/blog)
        ![](https://chezsoi.org/lucas/blog/images/2014/Jul/bw-2.jpg)

./wisemapping_txt2xml.py --images-size 100,100 --no-shrink markdown_example.txt > wise-editor/src/main/webapp/samples/markdown_example.xml

The result:

  • WiseMaping icons (they are all in wise-editor/src/main/webapp/icons/)
Blah blah blah   !icon=${icon_name}
./wisemapping_txt2xml.py icons_example.txt > wise-editor/src/main/webapp/samples/icons_example.xml

The result:

  • and with inline attributes like <!--fontStyle=";;#dfcfe6;;;" bgColor="#0a0a08"-->, full support for all the features in the default welcome.xml defined in an welcome.txt:

Final touches

In the end, I noticed the links preview, when you hover over a node with an anchor, was broken: it uses pagepeeker.com to create website screenshots, but oviously that doesn't work on localhost. Comparing with Framindmap fork of WiseMapping, I realized they also fixed in their own way the JS loader hell. And in their version, they replaced the pagepeeker screenshot by a simple button, which I preferred:

git clone https://framagit.org/framasoft/framindmap.git
cp framindmap/webapps/wisemapping/js/mindplot-min.js wise-editor/src/main/webapp/js/mindplot-bundle.js
# puis regénérer wise-editor/src/main/webapp/js/editor.js
• • • • • • • • • •