This notebook is inspired by the [demo notebook](https://mybinder.org/v2/gh/jupyterlab/jupyterlab-demo/18a9793b58ba86660b5ab964e1aeaf7324d667c8?urlpath=lab%2Ftree%2Fdemo%2FLorenz.ipynb) featured at the top of the ["JupyterLab is ready for users"](https://blog.jupyter.org/jupyterlab-is-ready-for-users-5a6f039b8906) announcement on the [Jupyter blog](https://blog.jupyter.org/).\n
\n
Having access to browser APIs and live computation alongside an analysis opens up a few opportunities when it comes to telling certain [computational narratives](https://blog.jupyter.org/project-jupyter-computational-narratives-as-the-engine-of-collaborative-data-science-2b5fb94c3c58). In this case the browser gives us richer real-time interactivity and much greater responsiveness than is typically possible with a remotely hosted kernel, and access to the media APIs offered by the web platform (particularly WebGL) allow us to create 3d graphics that are highly portable.\n
\n
%% js\n
// this is a hack to allow the ode45-cash-karp lib to import as expected.\n
Let\'s explore the Lorenz system of differential equations:\n
\n
$$\n
\\begin{aligned}\n
\\dot{x} & = \\sigma(y-x) \\\\\n
\\dot{y} & = \\rho x - y - xz \\\\\n
\\dot{z} & = -\\beta z + xy\n
\\end{aligned}\n
$$\n
\n
Below we plot the evolution of this system for several different randomly chosen initial conditions (the trajectory for each initial condition is shown in a different color). You can rotate, pan, and zoom to gain a better understanding of the three-dimensional structure of the attractor. Additionally, you can adjust the sliders below to change the parameters of the system.\n
To view the notebook in presentation mode, click on \'Run All Cells\' followed by the \'View\' button\n
\n
%% md\n
# World Happiness Report\n
\n
%% md\n
The World Happiness Report is an annual publication of the United Nations Sustainable Development Solutions Network which contains rankings of national happiness.\n
\n
This notebook uses the dataset available on the website [World Happiness Report 2018](http://worldhappiness.report/ed/2018/) to demonstrate the various aspects which affect the happiness of a country through exploratory analysis. Only 7 main factors have been considered. There were other supporting factors such as Life Ladder, Democratic Quality etc. which are not included in this notebook. The contribution of these supporting factors has been noted down as a Residual score.\n
The above visualization ranks the countries based on Happiness score. Scoring 7.632, **Finland** is at the top while **Burundi** is at the very bottom with a score of 2.905. The Scandinavian countries have taken the top four spots.\n
\n
%% js\n
var dataset;\n
var keys;\n
\n
var width = 1080,\n
height = 1800;\n
\n
var svg = d3.select(".ranking").attr("width", width).attr("height", height),\n
var hapMax = d3.max(dataset, function(d) {return d[\'Happiness score\']})\n
var hapMin = d3.min(dataset, function(d) {return d[\'Happiness score\']})\n
var hapMean = d3.mean(dataset, function(d) {return d[\'Happiness score\']}).toFixed(3)\n
\n
var hapColors = [\'#f0675c\', \'#375d81\',\'#d4273e\'];\n
\n
d3.select(\'.max\').text(hapMax);\n
d3.select(\'.mean\').text(hapMean);\n
d3.select(\'.min\').text(hapMin);\n
\n
d3.select(\'.stats\')\n
.selectAll(\'.count\')\n
.data(hapColors)\n
.style(\'background\', function(d) {return d});\n
\n
%% js\n
keys\n
\n
%% md\n
The scores for various factors are in different scales and thus not suitable for individual comparison. We will normalize the dataset using the below formula in order to achieve uniformity throughout our dataset for a more natural comparison and analysis.\n
1. Perceptions of corruption are bad throughout the world. With **Singapore** at the top, the rest of the world averages with a score of only **0.246**.\n
2. The world average for Social Support is reasonably close to the global maximum. **8** out of the bottom **10** countries in this category are **African**. The exceptions being Afghanistan and Georgia.\n
3. **Myanmar** is the most generous country whereas **Greece** is the least.\n
var hapColors = [\'#c9283e\', \'#c42f59\', \'#0091ff\', \'#e88e34\', \'#5670d6\', \'#9d3bc0\', \'#006f67\'];\n
\n
d3.select(\'.meanStats\')\n
.selectAll(\'.meanCount\')\n
.data(hapColors)\n
.style(\'background\', function(d) {return d});\n
\n
%% md\n
## What makes a country happy?\n
\n
What are the things that make a country happy? Below we have a plot consisting of the top 5 and bottom 5 nations ranked according to their Happiness score.\n
Happy Countries share in common a high GDP, great Social Support, a good life expectancy and a relatively high Freedom to make life choices. Countries with a bad happy score are in general poor and rank low on perceptions of corruption.\n
\n
%% md\n
## Are Happiness and Money correlated?\n
\n
Happiness and Money are two states which are often compared. Let\'s see if they are correlated or not. Before we begin, we would need to remove the component of **GDP Per Capita** from **Happiness Score** for an unbiased comparison. Thus, we create a new parameter defined as -\n
The data points are spread throughout the plot. We can observe that on average happiness increases as GDP per Capita increases. There are outliers such as **Somalia** and **Qatar**, both having similar Happiness score but are on the opposite end of the spectrum in terms of GDP Per Capita.\n
\n
On average countries with higher GDP Per Capita have higher happiness and countries with lower GDP Per Capita have lower happiness. The spread is very diverse for nations having GDP Per Capita in the mid range.\n
\n
%% md\n
## Does region affect Happiness?\n
\n
It is quite evident from the Choropleth map (at the beginning of this report) that some areas are more happy than the others. The below tables shows us the disparities between different regions of the world.\n
\n
%% js\n
var regions = [...new Set(unpack(\'Region indicator\', dataset))];\n
North America and ANZ is the happiest region. They are doing considerably better than the other regions in almost all the categories. Sub-Saharan Africa and South Asia are lagging behind the others. Perceptions of corruption are bad in most of the regions. There is a great resemblance between Western Europe and North America with the current scoring parameters.\n
<h1id="peering-into-the-unknown">Peering into the Unknown</h2>\n
<p>Can you tell what I\'m thinking right now? If you could you would know that I\'m hoping I don\'t have to rewrite this introduction for the third time, but that would make you an uncommon individual. Most people don\'t have the ability to read someone\'s mind let alone the purely physical process of looking into another person\'s head. Those lucky enough to belong to the latter category probably have access to medical imaging technology and someone willing to crawl inside it. The rest of us -- at least for now -- must settle for looking inside my head.</p>\n
<p>Briefly lets tangent into a light background on one particular sort of medical imaging known as MRI. MRI stands for Magnetic Resonance Imaging and you can see a cartoon of one scanner here. The parts of an MRI are roughly as follows: the patient table, which shifts the body part to be scanned into the center of the scanner; the magnetic and radiofrequency transmitter coils; radiofrequency receiver antenna coils.<br/>\n
<p>When the superconducting electromagnet is turned on the hydrogen atoms in the magnetic field align along the axis running through the machine </p><imgsrc="https://drive.google.com/uc?export=view&id=1G9f5lOPDgF5MK8n2ZtvtazEmviOpcow1"alt="machine on"/>. \n
<p>The hydrogens are excited by the radio transmitter pulses and flip direction. The hydrogens return to their original direction and emit a radiofrequency signal that\'s picked up by the receivers within the scanner. </p>\n
The above visualization demonstrates the volume of eviction notices San Francisco, yearly, from 1997 onward, by neighborhood.\n
\n
The dataset is made available by [DataSF](https://data.sfgov.org/Housing-and-Buildings/Eviction-Notices/5cei-gny5), the City of San Francisco\'s data portal. The data updates monthly, maintained by the Rent Arbitration Board.\n
\n
There are many legal reasons for eviction notices - a tenant may fall behind in rent, or perhaps has violated their lease in a way that, to the property owner or manager, merits eviction. In many cases, however, owners pursue various legal means to push out tenants to charge more in rent for a property, or to demolish a building to make way for more dense or more upscale housing. It is no secret that housing prices in San Francisco have skyrocketed past what many residents were previously able to pay, and the consequence of this increased demand and unmatched supply often takes the form of evictions.\n
\n
## Articles that inspired this analysis\n
\n
- [Why S.F. evictions are on the rise](https://www.sfchronicle.com/politics/article/Why-S-F-evictions-are-on-the-rise-6408950.php) (2015)\n
- [San Francisco\'s new pro-tenant rules increased evictions](https://www.bizjournals.com/sanfrancisco/morning_call/2015/07/san-francisco-rent-control-buyout-rules-evictions.html) (2015)\n
- [San Francisco evictions continue to rise each year since 2010](http://www.sfexaminer.com/san-francisco-evictions-continue-rise-year-since-2010/) (2016)\n
\n
<!-- please fix this. I shouldn\'t have to create padding to make this report readable. -->\n
Pyodide brings the Python runtime to the browser via WebAssembly, along with NumPy, Pandas, Matplotlib, parts of SciPy, and NetworkX.\n
\n
press `shift+enter` to step through this notebook.\n
\n
%% raw\n
\n
As you can see, some basic parts of Python work as you might expect. To use Python, just use the `%% py` delimiter in your editor, and hit `shift+enter` to evaluate. Your browser will then go off to grab Python & initialize it.\n
\n
%% py\n
# python\n
import sys\n
sys.version\n
\n
%% raw\n
\n
It also does the parts you might be surprised by! Try evaluating the chunk below (`shift+enter`).\n
\n
%% py\n
import antigravity\n
\n
%% raw\n
\n
Pyodide performs quick translation of data types into javascript, so there is one clear interface for accessing data and representing it.\n
We\'ve built ways of importing anything from javascript land into python. Here, we\'ll import `document`, which should be familiar to javascript web developers everywhere, and use its APIs to manipulate a DOM element.\n
\n
Note that Markdown allows embedding of inline HTML, so we\'ll add target element in a Markdown block.\n
\n
%% md\n
#### Here\'s a DOM element to manipulate:\n
<div id="targetElement">Change me!</div>\n
\n
%% py\n
from js import document\n
elt = document.getElementById("targetElement")\n
elt.innerText = "I am changed!"\n
elt.style.backgroundColor = \'#ffcccc\'\n
\n
%% py\n
# let\'s switch it back\n
elt.style.backgroundColor = \'#ffffff\'\n
\n
%% raw\n
\n
Because we _have_ direct access to the DOM, we can do fun things like make UI widgets.\n
\n
Run the code chunk below (`shift+enter`) and then click on the "Report Preview" tab on the top right. This report preview should render everything you see in these `%% md` chunks, as well as a button. The code you just ran below generated that button. Take a look at what the code does, and then click the button.\n
\n
Don\'t forget to switch back to the "Console" view when you\'re finished here - we have more to show you.\n
\n
%% py\n
from js import iodide\n
button = iodide.output.element(\'button\')\n
button.textContent = "Click me!"\n
count = 0\n
def onclick(evt):\n
global count\n
if elt.style.backgroundColor == \'rgb(255, 255, 255)\':\n
So far so good, but wouldn\'t it be great to use Python from within Javascript as well? When you load Pyodide, you\'ll get `pyodide` in the JS namespace, which lets you import anything on the Py side.\n
\n
%% py\n
# python\n
class Foo:\n
\tdef __init__(self, val):\n
\t\tself.val = val\n
foo = Foo(42)\n
foo\n
%% js\n
// javascript\n
var foo = pyodide.pyimport("foo")\n
foo.val\n
\n
%% raw\n
## The Scientific libraries\n
\n
The real power of Pyodide comes from its scientific computing libraries. So far we\'ve compiled numpy, pandas, matplotlib, parts of scipy, and networkx. At the top of any py chunk, simply write something like `import numpy as np` and run the chunk, and it\'ll begin the process of grabbing numpy.\n
\n
\n
%% py\n
import numpy as np\n
x = np.linspace(0, 2.0 * np.pi, 100)\n
y = np.sin(x)\n
y\n
\n
%% raw\n
\n
Matplotlib defaults to printing into the report preview, so you can run the code chunk below and see the plot. It\'s easy to drag around the plot, zoom in, and zoom out, all pretty effortlessly.\n
\n
%% py\n
from matplotlib import pyplot as plt\n
plt.figure()\n
plt.plot(x, y)\n
plt.show()\n
\n
%% raw\n
\n
It should be clear by now that Pyodide makes it so you don\'t have to choose only one library, nor do you have to choose one language to do your data science. Let\'s download Plotly and create the same graph, but send Plotly over to Python and plot it from within python itself.\n
\n
Switch back to the console, and run the `%% fetch` chunk below (which fetches Plotly from a cdn), then run the `%% py` chunk below that, and switch back to the report view.\n
\n
%% fetch\n
js: https://cdn.plot.ly/plotly-latest.min.js\n
\n
\n
%% py\n
from js import Plotly\n
from js import iodide\n
Plotly.plot(\n
iodide.output.element(\'div\'),\n
[{ \'y\': y, \'x\': x }],\n
)
]]></string></value>
</item>
<item>
<key><string>title</string></key>
<value><string>A Brief Tour through Pyodide</string></value>
Let\'s get started! You can just start trying things out, or you can follow the steps described below to get a quick introduction. Just read along and click or enter keystrokes when instructed.\n
\n
The first thing you\'ll need to know is how to navigate within the notebook. An iodide notebook is one flat text file in a format we call JSMD. Each cell is delimited by `%%`, and that means you can easily add cells just by typing those two characters on a new line. This cell, for instance, has `%% raw`, which means it\'s a raw cell. Raw cells are useful for taking notes and leaving comments in your notebook.\n
\n
Right below this is `%% js` which is a javascript cell.\n
\n
Click somewhere in the javascript cell right below this one, and hit `shift+enter`. This will evaluate whatever chunk your cursor is on. Notice in the console to the right, you\'ll see the code and the output.\n
\n
%% js\n
\n
var tableSize = 10\n
var range = []\n
for (let i=0; i<tableSize; i++){range.push(i)}\n
var A = range.map( (x,i) => range.map( (y,j) => (Math.random() + i + j )))\n
A\n
\n
%% raw\n
\n
This entire buffer is a single text editor, so you can add `%% js` or `%% raw` (or any of our cell types) at any point (starting with a new line) and that automatically creates a new cell.\n
\n
%% raw\n
\n
There is another kind of cell - `%% md`, the _Markdown cell_. The contents of this cell renders in the "Report Preview" pane. You don\'t have to evaluate it the way you did the `js` cell. \n
\n
Click the tab next to "Console", titled "Report Preview" in the top right to see that pane. \n
\n
Markdown cells are where the output of your data analysis will live. It\'s your way of separating your exploratory code with your final explanation.\n
\n
Try editing parts of the markdown cell directly below, and watch it live-update in the Report Preview. When you\'re done, click on the "Console" tab again - we\'re going to need it in a minute.\n
The cell below this is an fetch cell, which allows you to load external libraries, css, and even data. It can be evaluated just like any other cell with `ctrl+enter` or `shift+enter`.\n
\n
When a resource is successfully loaded, you will see `SUCCESS` printed in the console. If a resource fails to load, then you will see `ERROR`.\n
\n
Let\'s go ahead and load the library "three.js", which we\'ll use a little farther below.\n
\n
%% fetch\n
// click inside this chunk and hit shift+enter to load the resource\n
Ok! Time to do something interesting with three.js.\n
\n
%% md\n
\n
You can also use in-line html, which is very handy for creating DOM elements within the flow of a narrative that you can then manipulate. This is great for adding plots and figures.\n
\n
Let\'s create a `div` right below:\n
\n
<divid="totalRows"></div>\n
<divid="spinningThing"></div>\n
\n
...in a moment, we\'ll put something fancy right above this line.\n
\n
%% js\n
// We\'ll now define a real JS function that does something cool.\n
// Go ahead and evaluate this cell.\n
\n
// first off, let\'s clear the div where we draw, just in case.\n
var material = new THREE.MeshLambertMaterial({color: COLOR });\n
var cube = new THREE.Mesh(geometry, material);\n
scene.add(cube);\n
camera.position.z = 12; \n
var pointLight = new THREE.PointLight(LIGHT);\n
pointLight.position.z = 130;\n
scene.add(pointLight);\n
var reqAnimFrame = window.requestAnimationFrame\n
var render = function() {\n
reqAnimFrame(render);\n
var delta = (Math.random()/10) * (0.06 - 0.02) + 0.03;\n
cube.rotation.x += delta;\n
cube.rotation.y += delta;\n
renderer.render(scene, camera);\n
};\n
render();\n
}\n
\n
spinCubeInTarget("#spinningThing")\n
\n
%% raw\n
If you followed all the steps described, you should now see a cube spinning in the markdown cell above!\n
\n
Note that if you make any changes to that markdown cell, you will have to re-run the function `spinCubeInTarget("#targetDiv")`, because each time you change a markdown cell it\'s contents have to be regenerated, which will overwrite changes you\'ve made to it with code.\n
\n
%% raw\n
\n
One last cell type - `%% css`. This lets you change the styles of your presentation in realtime. Select the "Report Preview" tab on the top right, and then uncomment the css in the code below.\n
\n
%% css\n
/* remove the comments here */\n
/* \n
.user-markdown {\n
\tfont-style: italic;\n
} \n
*/\n
\n
%% md\n
\n
## to summarize:\n
\n
Ok! Let\'s end this notebook with a markdown cell. With all of these tools, you\'re ready to get started! For review, we covered the following tools:\n
\n
1. `raw` cells - make notes, add comments\n
2. `js` cells - run javascript code\n
3. `fetch` cells - grab libraries, external stylesheets, and data sets\n
4. `md` cells - write your report, add DOM elements, etc.\n
