Scratch/Blockly language as additional macro language

fiji
imagej
macro

#1

Dear Community,
I have been playing a bit with the Scratch language some times ago. Eventhough it is mostly though to introduce kids to programing, I realised that it has some high potential for more serious projects that involves non-initiated coders.

As an example, the platform TinkerCad Electronics allows to do some electric circuits simulation including Arduino, and the sketch of the Arduino can be written in this Scratch language.

I also used the offline Scratch editor to allow the user to generate some text file using predefined function blocs.
It is even possible to wrap a Scratch project into an mobile application using the MIT App Inventor for instance !

The Idea

My idea now which is probably a bit futuristic, would be to have a Scratch Editor in Fiji that would allow to write macros in this language.
I agree that complex program are more easily written in a hard-coded language, but I think it could be some intuitive alternative to the macro language for instance.
It is possible to write some extensions for Scratch to have your own Scratch blocs, but then you can only use them in the Scratch Editor.

What I dont know is how to use the scratch language into a stand alone application like Fiji…
But if TinkerCad can do it on a website I guess there are some ways :wink:


#2

Cool idea. Having a visual programming language directly available in Fiji would be awesome. I think it would be unlikely that the original Scratch language would be incorporated. If someone had the capacity to take on this project, then I suspect they would end up re-implementing a Scratch-like language instead of trying to make Scratch and Fiji/ImageJ libraries compatible. That said, KNIME Image Processing is one way to do visual programming with ImageJ/Fiji libraries that already exists, it looks like:


#3

I am using Knime intensively too for image processing but for most Fiji users it is still too overwhelming somehow.
I think some core parts of the scratch language could be reused like the loop structures… and then indeed the big part of the work would be to implement the Fiji functions into some Scratch blocs.


#4

I just found out that there is a javascript library provided by google to do this kind of visual programming like Scratch, it is called Blockly


#5

Hi, I think it’s a good idea to use block-style programing interface. And I also did some investigation on those libraries including Scratch and Blockly, they are fun, simple and light-weight.

Along this line, there is also Joy.js which I think is easier to read and write. We started with the idea of creating an easy-to-use image processing tool with Joy.js. And the project quickly evolved into ImJoy – a browser based image processing software which utilises recent web technologies such as webassembly, the entire Python ecosystem, conda and other techniques. Joy.js is used as the core part for composing workflow in ImJoy.

In general, new JS libraries(including Blockly and Scratch) can be easily introduced into ImJoy by set their url as requirements, and then bridged with other plugins in ImJoy. I just made a demo which uses Blockly, and you can try it out by clicking this link: https://imjoy.io/#/app?p=oeway/ImJoy-Demo-Plugins:Fixed-Blockly, and here is the source code of this plugin deployed to our demo plugin repository.

If someone interested in this, potentially, we could also try to make an ImJoy plugin with pyimagej, and use those block-style libraries to call ImageJ functions through the pyimagej plugin.

This is a screenshot of this Blockly demo plugin in ImJoy:


#6

Hi Wei,
Thanks for your message, I mentioned ImJoy in a blog post on the forum among other online/workflow image processing solutions (I just tagged you :wink: )
I think it is very promising, maybe you could take advantage of the Neubias network to start disseminating the tool to the advanced users in the first place, to gradually raise the attention to the platform :smiley:

You might even have the possibility to add ImJoy to the top banner of the forum with the other open source tools. Check the “Your icon here” if ImJoy fulfills the requirements.

I did not know about this one, that’s good to know ! Indeed the interface seems smoother than the dragging and embedding of blocks !

If someone has the time and motivation to lead this project that would be great. But It think it might be a bit premature, for the moment I would expect the users to get familiar with the functionalities of the platform and the possibilities of the Joy language :wink:


#7

Hi Laurent,

I saw your blog post which is quite nice, thanks. And also thanks for your suggestions, indeed we should do that. We have been quite busy on developing the core features of ImJoy, making documentations, and also adding demo plugins. Right now the ImJoy API and functionality more or less stabilized, and we started to accumulate more demo plugins. Hopefully we can do a beta release soon, before that, any feedback would be more than welcome.

I will also do a demo session for ImJoy during the Open-source Software Lounge at NEUBIAS 2019(Luxembourg).