# Can we implement a Neural Network interactively?

Implementing a Neural Network manually need some code and need to make some connections between these components.

Can we implementing a Neural Network interactively?If I want to make a net like following topology is just drag some components to specific location.

The following is about machine learning.

or

Can we create a drawing tool for Neural Network like drawing tool for Graphics?

• You definitely could implement some block-programming scheme like that, but from my experience playing with block-programming it's tough to get a good system in place. But if you want to write it yourself I'm sure it would be really popular. I can probably get you started with some of the dynamic drawing stuff, if you want. – b3m2a1 Sep 1 '17 at 3:59
• You might be able to do this with visX, but it hasn't been released yet so you would have to contact the developer to see if it can be used like this. – C. E. Sep 1 '17 at 4:13
• @C.E. wow that is sick. Have you contacted the dev / do you know how it works? Is it desktop-based? (from the graphics) It doesn't look like it's implemented in Mathematica. – b3m2a1 Sep 1 '17 at 4:24
• @b3m2a1 Yeah, it's very nice. I haven't been in contact with the developer personally but it's a standalone software which communicates directly with the kernel without using the Mathematica front end. The author has written more about it on Wolfram Community. – C. E. Sep 1 '17 at 4:46

### Update:

So after adding the missing features I decided to give your actual problem a go. This is what I have:

You'll note a little "Add Layer" ActionMenu on the right. That's really the only actual extension to the basic BlockBuilder[] interface I needed. Wasn't hard to get in.

The Run function turned out to be something of a tough problem. I had code that will turn the data that is used on the back-end into a graph, but it also contains all those line-nodes, too.

In general this is what I did:

Any connected components (a window or a line) are treated as a single node. There's actually meta-info in the core back-end data that allows you to better differentiate which Locator is which, but I didn't want to use it.

Anything without a bound "Expression" (because all of the windows get an expression bound to them) gets dropped from the final graph representation, just for this neural-net-graph making function.

Then the entire function gets used as the Run function for that bottom left hand button.

The command I used to build that interface was:

NNBuilder[Dynamic[nn]]


On pressing Run the graph gets assigned to nn. It looks like this:

You'll notice there's a list of graphs there, because if there were disconnected components in the interface we'd need multiple graphs.

Now, you say, this doesn't actually build a net! That's because a) I don't know enough about NetChain to really know what I'd need to build out a real one and b) because I just wanted to demonstrate that this type of thing is theoretically possible.

If you reallllly want the NetChain let me know what features I'll need and I'll try to get to it eventually.

If you want to work with the BlockBuilder interface let me know and I'll document stuff better, clean up the code, and try to set up a system for extensions that I've been mentally designing but never actually bothering to build.

### Update Description:

The main things I did in the update were interface-based. I added a way to add controls to one of these panes, then layered some standard controls on that. To see what this looks like run this:

Quiet[
Check[ << BlockBuilder,
Get["https://raw.githubusercontent.com/b3m2a1/mathematica-tools/master/BlockBuilder.m"],
Get::noopen
],
Get::noopen
];
BlockBuilder[]


Add a window with the Add Window button, remove it by either clicking the little X icon or command-clicking on one of its locators.

Add a connector line with the Add Line button. Remove it by command-clicking on one of its locators.

The Run button will take a function and supply the core locator data and a basic locator graph. Windows will have "Expression" and "MetaInformation" properties. If you want to learn more message me and I'll explain more how to work with it.

The scan button applies a function or list of events to the graph as a DepthFirstScan. I haven't found a use for it, but maybe someone will.

BlockBuilder can be used like this:

BlockBuilder[Dynamic[var],
controls_ (*a list of functions to add controls to the basic pane*),
run_ (*a function to use as the default "Run" function*),
scan_ (*a function to use as the default "Scan" function*)
]


The interface is a little primitive, but it's an example of how block-programming could be implemented in pure Mathematica.

### Original:

Hopefully you don't mind that this isn't a full answer. Just a starting point on a starting point.

I mentioned I could help build out some of the block-syntax and here's what I've got so far. I don't know if you want to pursue building this feature, but this block system will probably be useful somehow for someone once it's done. It needs maybe and hour or two more work, but I'll get to that when I have time.

My strategy was to do this all with Locator. Each Locator maintains some state in a locatorManager association, has events bound to it, things it's connected to, etc. This is all managed by the locatorManager and by a UUID given to each locator. There are then three main types of objects in the drawing system:

1. Point locators - these are just points in a Graphics pane. Not sure what they're good for, but they were easy.
2. Line locators - these are two locators plus a Line graphic connecting them
3. Block locators - these are a collection of locators which can move as a group or not, depending on options set at the locator level. Generally a Polygon is draw between them. They can be made "Draggable" so that you can move them by the actual block element and not just the locators.

Types 1 and 2 will automatically lock onto a locator of type 3 when they get close enough, and will move with the type 3 locator until they are explicitly moved away. This makes rearranging everything much more convenient.

Then I made a fourth locator on top of the block locator to support Inset specs. Past this I'm not sure what else is needed. It takes two locators and puts and Inset element between them.

I also made a little window element to inset, just to make things prettier.

Finally I had a function that initializes a locatorManager in an Association, determines which locator type to use from a list spec, and formats a Graphics with all these locator elements. Here's an example.

The spec language looks this:

Import@"https://raw.githubusercontent.com/b3m2a1/mathematica-tools/master/BlockBuilder.m"

anchorLocatorModule[
Join[
Map[
{
{#, {160, 0} + #} &@({-400, 400} + {170, -200}*#),
insetWindow[
"Expr" <> ToString[#]
]
} &,
Range[3]
],
Map[
{{{300, #}, {350, #}}} &@(400 - 50*#) &,
Range[2]
]
],
Frame -> True,
FrameTicks -> None
]


Here's the output of that:

And here's what it looks like after I attach some locators and change some expressions in the windows:

When I'm 100% done with this you'll be able to add and delete elements on the fly using a drawing-tools like mechanism.

Then you can get the collection of connected elements by simply building the tree from the locatorManager (I'll support passing your own via Dynamic rather than having to use the copy from the DynamicModule). I'll also get a richer event set in place so you can customize with those. And finally, once I have the simple drawing tools in place, I'll add a button for "Run Blocks" or something where you can provide a function and it will apply it to the block tree.

At that point figuring out how to turn a block tree into an NN is up to you (although I'll give you a little window element where you can provide a set of choices to choose via PopupMenu`)

I'll update this once I've had time to round out the block building system.

• Wow, you have a lot of time :-) (+1) – Szabolcs Sep 2 '17 at 10:29
• @Szabolcs mostly I simply don't sleep enough :-) Gotta feed the coding addiction and all that. – b3m2a1 Sep 2 '17 at 12:50
• Can you add a code snippet for the update? – Kuba Sep 4 '17 at 11:30
• @Kuba I added a brief description of what I added with a quick snippet. If you want more I can definitely add it. – b3m2a1 Sep 4 '17 at 21:24