This is probably my favorite tip to offer. If you haven't heard of Jupyter Notebooks, this is going to be a huge game changer for you.
While I only use Jupyter Notebooks with Python, I know it can be used with lots of languages and frameworks. I only have experience using it for Python and Machine Learning.
Jupyter Notebooks is different than anything I have ever seen before, the closest thing I describe it to is an interactive shell with charting, visualization, interactive widgets, reporting, and so much more.
Python is frequently used for data analysis and machine learning, in this capacity you are frequently working with large datasets and visualization.
You might be doing something where you have a simple dataset where you want to chart it.
To something far more complicated involving interactive charting and complex math.
I am not going to go to in depth as this is more to introduce you to the fact that Jupyter Notebooks exists and what it can do. To cover how to use it would be far more than I can do in a single post.
One use for Jupyter Notebooks that isn't talked about a lot is when you are testing out code and working with an API and want to have instant feedback. One of the nice things is you can run each block individually, so you can run your import block and setup code, then mess around with your experimental code re-running it as needed as you test things out.
If you have to import a new module, just add it to the import block and re-run that block or the entire notebook.
Let's take a simple Steem example:
Now we decide we want to do something with datetime, we can easily update the top block to have import datetime and re-run just that part.
We don't have to re-run the entire script if we don't want to. If you break your scripts into chunks and put them into different cells, it becomes easy to change parts of your code and re-run only what you need.
I don't recommend using Notebooks for everything, but there is a lot of code I run that is more of reporting and data collection on the blockchain and it is far easier to do this in a Jupyter Notebook than command line or an IDE.
When working with lots of data you want to graph and see visually, Jupyter Notebooks cannot be beaten.
If you use Anaconda for your Python distribution (highly recommended) you don't need to do anything to install Jupyter Notebooks, you already have it. Just type jupyter notebook from a command line and it will start running.
There is a newer version of Jupyter Notebook called Jupyter Lab which is in development and beta. It has some additional features like tabs, text editor support, data file viewers, and some new components that makes it much more powerful. It is also fully compatible with Jupyter Notebook files. I suggest using Jupyter Notebooks first, and testing Lab when you are familiar with Notebook.
Sharing
One of the best things about Jupyter Notebooks is the ability to share Notebooks with other people and easily export them into PDF format for those who don't have Python installed.
Github also has full support for Notebooks, if there are notebooks in your repo, you can just view them online without having anything installed. Very cool.
Examples
There are lots of examples you can look at on Jupyter's website under the nbviewer. One of the great things about Jupyter Notebooks is you can mix Markdown with Code and create really great reports and data visualization that can be shared easily. The end user can view the notebook as a PDF, HTML file, or even as a Notebook they can run and edit themselves.
You showing me this a couple months ago has been a game changer for how I interact with python and my server. For troubleshooting and learning it has been a god send. Really do appreciate your help getting started.
Mixing in the markdown with the code is intruiging. I don’t do much statistical stuff but I’m still thinking I should try it out :)
The markdown is good for describing what is going on. If you look at some of the example notebooks, they do a good show showing how it can help.
For example, this simple notebook uses it to give an overview of each code block and what's going on. Most of the time when using Notebooks you are dealing with a lot of data.
Another great example:
https://www.kaggle.com/stkbailey/teaching-notebook-for-total-imaging-newbies
Notebooks are great for tutorials and courses. I was doing a course on deep learning on Coursera and the tutorials/homeworks were Notebooks. I have used Notebooks for Kaggle submissions as well.
I am a bit confused though in terms of the difference between Jupyter Notebook and IPython Notebook, so much so that I can't remember which was the one that I used...
Jupyter is the new name, it used to be called Ipython Notebooks.
Jupyter Notebook is just awesome!
I was introduced to it by a colleague. We used MATLAB and struggled to get a licence because of a shortage. He already used IPython Notebook and at the time it upgraded to Jupyter Notebook he set up a server and introduced his subdivision colleagues. It spread like a wildfire in the whole R&D department. I have never seen a similar adoption and success of an open source tool or any other tool.
This experience was a game changer for me and I started using more open source software like LATEX, Blender and Inkscape. Brave Browser is the latest member of this club.
Jupyter Notebook is best used on Linux. In case you want to try it on Windows i recommend WinPython, it contains Jupyter Notebook beside some other useful tool-sets.
Even Microsoft is in love with Jupyter Notebooks! You can use them for FREE on Azure Notebooks. Microsoft uses a Linux server environment and Python 2.X, 3.X, R and F# Kernels are pre-installed.
Check out Jupyter Lab the next iteration of Notebooks.
This is my first time of hearing about it and I really appreciate your efforts, am gonna get it
Github also have support for jupyter notebook, some hard core python coder will use it to document haha
snake game that I like most in my childhood .. until now I am still a favorite game .. I really like your post ...
Jupyter notebooks are great. I actually run notebooks in Docker. I've ran into problem with python running on my Mac. Docker made life easier.
Python already has a solution for that. Virtual Environments. ‘Virtualenv’
Jupyter Notebook is just awesome!