Matt Mazur

Here's a few things that really helped me out:

http://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
http://brenocon.com/blog/2013/10/tanh-is-a-rescaled-logistic-sigmoid-function/

Sort of a high-level overview (some of this is particular to neural nets as they're implemented in JMP, a product made by SAS, I don't know how well they generalize):

• Neural nets consist of an input layer, an output layer, and some number of hidden layers between the input and output layers
• The hidden layers consist of a number of nodes, which can be thought of as “on-off” switches (called a “step function” if we’re using the proper term), although in practice they’re represented by a smooth sigmoid curve with an upper and lower bound that represent on and off. In JMP, this takes the form of a hyperbolic tangent function, which is bound between -1 and 1.
• Each node has a value that’s called the bias, which if you’re thinking of the nodes as an on-off switch, is the threshold a linear combination of the inputs needs to reach to turn the node on. In the sigmoid curves output by JMP, the bias represents the point at which the hyperbolic tangent function returns 0, given a linear combination of the inputs.
• A “deep learning” model is one that has more than one hidden layer; the lowest hidden layer serves as the input layer to the next hidden layer, and so on.
• The output layer takes the nodes of the highest hidden layer and produces an output. When dealing with probability,…

> are neural nets deterministic at all? Given all their randomness, I'd assume not.

That's actually an interesting question; I'm probably not the person to answer it, since my experience with NNs is admittedly on the "still learning" side of things, and that learning has not been via any credentialed sources...but I'm willing to take a stab at it.

NN models - that is, the thing that results from training a neural network, are typically "fixed", in that once trained, the model doesn't change - at least classically. I imagine that somewhere out there, there may be NN models which can change as data they are processing is run through them, learning on the fly so to speak. I'd have to research it; I'm sure it's something that's done or been done?

But normally, once a NN has been trained, it's model is "fixed" and doesn't change when inputs are presented to it. The training phase is - or appears to be - fairly stochastic. I'm not sure I'd want to call it random, though, because I don't recall any kind of random numbers being used during back-propagation.

But the data that is presented to the neural network is usually randomized in presentation - and sometimes content. That is, say you're training the NN on recognizing horses. You might have several thousand images of horses, but you don't want to show just those. You might want to generate many more - rotate each one just a bit, skew it just a smidge, maybe change the color and/or contrast/brightness, etc - to in effect…

Congrats Matt! Very cool to see.

A question. I'm trying to find a tool to visualise and document the evolution of obscure sports (purely because I find the interactions fascinating). It seems that your tool might be the perfect fit, especially the "writing" examples. I was wondering if it would be possible to visualise something along the lines of a quasi-directed graph. Nodes being specific instances of the inception of a "new" sport (hence the date requirement), and edges being the influence on other spots. Multiple edges for each node are possible, and all sports will have a start date and some an end date. I tried[2] with a basic POC but it seems the issue is that the nodes run without an end date, which breaks the dependency visualisation.

One comment. Once logged in I wasn't easily able to get back to the landing page, which had some useful info around visualisation types that I wanted to review.

And a note. I've apologised to a few people who I subjected to a variation on your SQL screener interview question [2], which contained far too much fiddling with SQLite time functions, entirely my mistake in ruining an otherwise great basic SQL test!

[1] https://www.preceden.com/timelines/868890-obscure-sports?s=e0d469326e067313
[2] https://mattmazur.com/2018/11/12/analyzing-89-responses-to-a-sql-screener-question-for-a-senior-data-analyst-position/

Why can't you model each neuron as an individual object with a unique UID in code?

I am fortunate enough to have a friend that's fairly accomplished in ML and a really nice guy. He spent a video chat + code session with me where he helped me build an NN from scratch in Typescript, we replicated the network in the paper here:

https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/

Here's the Codesandbox link, if you notice the console output it (approximately) matches the figures from the paper above:

https://codesandbox.io/s/node-typescript-vqszi?file=/example/app.ts

I don't really understand much about ML or neural networks, but from a code perspective, if you have these data structures:

class Neuron {
id: number | void = neuronUidGen.next().value;
incoming: Connection[];
outgoing: Connection[];
rate: number;
bias: number;
output: number;
error: number;
}

class Connection {
id: number | void = connectionUidGen.next().value;
from: Neuron;
to: Neuron;
weight: number;
}

Why couldn't you manipulate these however you see fit? There's a Layer implementation in there that's commented out because it doesn't function right that could roughly accomplish something like this theoretically, right?

The below resources are the ones I used when I started to learn about DL/NNs. Some of them are focused specifically on certain applications, but I found them helpful, too.

Basic NNs:

http://www.wildml.com/2015/09/implementing-a-neural-network-from-scratch/ (a whole series, all worth reading)

https://gist.github.com/sthware/c47824c116e6a61a56d9 (my code based on the above)

http://iamtrask.github.io/2015/07/27/python-network-part2/

http://rolisz.ro/2013/04/18/neural-networks-in-python/

ML:

http://cs229.stanford.edu/materials.html

http://onlinestatbook.com/2/index.html

DL in general, RNNs, RNTS, CNNs, some others:

http://cs224d.stanford.edu/syllabus.html

http://cs231n.stanford.edu/syllabus.html

http://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/

http://arxiv.org/pdf/cs/0205070.pdf

http://colah.github.io/posts/2015-08-Understanding-LSTMs/

http://karpathy.github.io/2015/05/21/rnn-effectiveness/

http://nlp.stanford.edu/~socherr/EMNLP2013_RNTN.pdf

http://alexdavies.net/talks/

http://www.socher.org/index.php/Main/SemanticCompositionalityTh
roughRecursiveMatrix-VectorSpaces

http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/

http://colah.github.io/posts/2014-07-NLP-RNNs-Representations/

I recently saw a blog post where someone wrote about the feedback they received and they weren't happy. The same person then revised their questions and tried a more task oriented approach and they were much happier.

http://www.i-proving.ca/space/David+Jones/blog/2010-01-27_1?showComments=true#startComments

Then again, Feedback Army is just a usability testing tool. The right one for the job. I think it's best when a breadth of ideas are needed quickly or you need some ideas about where to start improving your site.

You should use other methods to do testing that requires a lot of depth from one or two people.

Here are a few more posts where folks have written up their experiences:

http://visualwebsiteoptimizer.com/split-testing-blog/how-an-investment-of-10-can-give-you-website-improvement-ideas-worth-thousands-of-dollars/

http://www.mattmazur.com/2010/02/reflections-on-precedens-hackernews-launch/

(p.s. I run Feedback Army :))