Interface Vision

It's a fascinating discussion. Maybe we can meet half way?

> I definitely believe that there is value in artists learning to think with the rigor that code encourages. It's a fascinating cross-pollination...

I agree!

> Design is remarkably accessible...

But this... Well it depends on the point of view. Design can be as intimidating for geeks as programming is for artists.

I have nothing against making programming tools more user or artist friendly. In a sense Photoshop is such a step - it allows many creators make sophisticated images without learning to draw and without learning to program. And certainly many many amazing works were created with it and similar tools.

The point I am trying to make here is that the visual/artistic power or possibilities of generative art and design are very much driven and dependent on the artist's fluency to program. There were many attempts to create programming tools [1] that let you avoid having to slog through typing the code, but seems like none of them would match the expressive possibilities of 'raw' coding and gain any wider adoption. In other words all of them have significant limitations and this would be a step back from the point of view generative artist.

And just to make it clear, I don't think that using any of these higher level tools is wrong or produces works of lesser value. It's just that these works are outside of unique possibilities of generative art driven by traditional coding.

[1]…

You give two examples (spreadsheets, Autohotkey) that actually contradict your central thesis of no good tools. Seems like those are examples of ordinary people successfully using no-code to make their lives easier?

There are so many free programming languages available to anyone interested, ranging in human comprehensibility from (the incomprehensible) brainfuck [0] to natural language interpreters for writers of interactive fiction [1], and everything in between. Visual programming languages [2], languages for children [3], languages for visual artists, for musicians, for stock traders. There's even a language using power ballad lyrics [4]!

I love your sentiment that programming is for everyone, but with respect I utterly disagree that there are no good tools for ordinary people to use. If anything, there is an overwhelming, embarrassment of riches.

But, maybe you are right? Maybe you have an insight that could be helpful? Share it, man! The world needs you!

> ...we kneecap them with awful tools

Who is this "we", Kemosabe?

If you want to help make interactive narratives easier and less prone to bugs, I have a fallow side project I need help with [5]. These people seem cool [6] and could use your insights, I'm sure (seem to be defunct, but perhaps from lack of interest? Jump start it? Track them down, find out what happened?)

[0]
https://en.wikipedia.org/wiki/Brainfuck
[1] http://inform7.com/

[2]…

Cool stuff. Wish I found this a few years ago when I dug into this topic for a thesis. Back then I analyzed this incredible resource (warning: clicking this link will load looots of images): http://blog.interfacevision.com/design/design-visual-progarmming-languages-snapshots/
and came up with my own provisional taxonomy where major categories were "Line-connected block-based", "Snap-together block-based", "List-based", "Enhanced text", and a few others.

Looking back I still think that the most promising direction when it comes to real-world applications of visual programming is the hybrid approach: make a language which has "real-time" interchangeable text and visual representations -- you get the best of both worlds this way (that was basically the topic of my thesis[1]).

I thought it was an original idea when I started out with my research, but sure enough there were attempts in the past. And towards the end (4+ years ago) I discovered a fresh one which seems to be still going strong, albeit rebranded: https://github.com/enso-org
Last time I checked they were making what I consider a bad design choice by leaving the structuring of the visual representation entirely up to the user, but maybe that's fixed now.

Anyway I think it's a cool topic and worth keeping an eye on.

[1] If anyone is interested it's available here: https://djedr.github.io/masters_thesis.pdf

Here is a comprehensive list of different visual programming languages.

http://blog.interfacevision.com/design/design-visual-progarmming-languages-snapshots/

The issue isn't visual programming but rather that we haven't found many good cases to use it still.

I do think visual programming is going to have a bigger and bigger impact in things like robotics, space exploration etc where you need more tactile access to computation power.

It's kind of like with the idea of a 3d interface. It only make sense when the holistic whole rather than the details are important.

It makes no sense to create a 3d warehouse I can walk around when I could just do a search and a list.

But you wouldn't necessarily want to control a robot doing repairs on your spaceship by having to program it every time you wanted it to do something new.

I've done a lot of research into other VPLs as well, and in theory, all of these ideas are sound.

Unfortunately, it seems like VPLs only work well for well-defined domains (say, a VPL for shaders, a VPL for compositing, a VPL for EE, a VPL for business logic)

Once you try to go general, and add "zooming in" and "zooming out" mappings, the number of ways that people would prefer to visualize each becomes exponentially more complex to display/interact with; so much so that then simply using text seems like a general-enough solution.

I'm still hopeful that a general purpose VPL could exist and work for 'real' programs, but so far my search has come up empty.

A good reference list to look at for existing VPLs:

http://blog.interfacevision.com/design/design-visual-progarmming-languages-snapshots/

Apple's objective-c and associated framework were designed to be coded against (and even though it supports message passing, it still heavily uses parameterized sub-routines).

A lot of 3d modeling software provides frameworks where you hook things up instead of coding out solutions (Foundry Modo, Eyeon Fusion, Dynamo are a few). These are very domain specific and not designed to work as general purpose languages/frameworks.

We are working on some ideas for a general purpose compose-able framework and language that uses messages with behavior as the abstraction for communication between sub-systems (http://blog.interfacevision.com/design/design-composition-based-language/).