Domino Data Lab, a data science platform company, writes about Jupyter, Python and R, modeling, and data science team practices.
HN commenters point to it as a data science blog. They link its posts on mapping with Folium, notebook usage, and data provenance.
Full Disclosure, I am the Chief Data Scientist of Domino Data Lab (https://www.dominodatalab.com), a company that builds tools for collaboration/scaling of data science a quantitative research teams. I spoke about our view on data provenance at UseR! 2016 and if you're curious you can watch the video here https://blog.dominodatalab.com/providing-digital-provenance-modeling-production/. The rest of this comment is not about reproducibility and provenance however, it's mostly about collaboration.
I really like this write-up. It's really quite prescient for a blog post from 2013! There have been amazing advances in the tooling for individual data scientists. RStudio, Jupyter, and Zeppelin notebooks have provided a really fluid REPL for data science. This has really compressed the workflows he discusses, and provides feedback and direction to data scientists much more quickly than the program oriented process. It’s interesting to note how an unexpected direction (notebooks) can impact praxis.
I refer to these two visualizations of the data science process often. The first is by Szilard Pafka, and it details the workflow he has experienced by having models in production for well over a decade at this point. It’s available here:
https://raw.githubusercontent.com/szilard/datascience-1slide/master/ds.png
What I really like about this is that it really shows the centrality of model validation. From model validation, you can often find yourself having to go back to exploration,…
Full disclosure: I work in marketing at Domino.
Domino offers prebuilt deep learning environments on its data science platform. Tensorflow, PyCUDA, Theano, H2O, a variety of R deep learning packages, and many other tools are available.
The environments run on Amazon's Nvidia GRID and Tesla instances.
This post is a little old but will give you an idea as to what you can accomplish:
http://blog.dominodatalab.com/gpu-computing-and-deep-learning/
We offer select prospects access to GPU compute when trialing Domino. Let me know if your team is interested.
They do model/feature storage and versioning just fine. Or at least Luigi does - I use it daily.
And feature extraction and modelling is my main usecase. It works really well, across Spark, Scikit and R, saving data in HDFS.
Some examples:
https://blog.dominodatalab.com/luigi-pipelines-domino/
http://blog.richardweiss.org/2016/10/13/kaggle-with-luigi.html
https://github.com/Atreya22/luigi_rosmann_sales
The submitted article was https://blog.dominodatalab.com/data-scientist-programmer-mutually-exclusive/?r=1 with the title "Data Scientist? Programmer? Are They Mutually Exclusive?"
Since it summarizes a talk, we changed the URL to that of the talk, in keeping with the HN guidelines' request for original sources (https://news.ycombinator.com/newsguidelines.html).
One approach is to convert the code to use parallelism. For an example of how to do it in Python using joblib see this article: http://blog.dominodatalab.com/simple-parallelization/
Even if you can't afford a 32-core instance, you might get to use 4 cores in your laptop.
I think that depends on what you mean;
Jupyter lets you run magic[0] commands (run bash commands in subprocesses), and there's also a bash kernel[1].
[0]: https://blog.dominodatalab.com/lesser-known-ways-of-using-notebooks/
[1]: https://github.com/takluyver/bash_kernel