\n",
" \n",
" \n",
" \n",
"

\n",
"\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"id": "f9e9b958",
"metadata": {},
"source": [
"# About These Lectures\n",
"\n",
"> “Python has gotten sufficiently weapons grade that we don’t descend into R\n",
"> anymore. Sorry, R people. I used to be one of you but we no longer descend\n",
"> into R.” – Chris Wiggins"
]
},
{
"cell_type": "markdown",
"id": "edba74ee",
"metadata": {},
"source": [
"## Overview\n",
"\n",
"This lecture series will teach you to use Python for scientific computing, with\n",
"a focus on economics and finance.\n",
"\n",
"The series is aimed at Python novices, although experienced users will also find useful content in later lectures.\n",
"\n",
"In this lecture we will\n",
"\n",
"- introduce Python, \n",
"- showcase some of its abilities, \n",
"- discuss the connection between Python and AI, \n",
"- explain why Python is our favorite language for scientific computing, and \n",
"- point you to the next steps. \n",
"\n",
"\n",
"You do **not** need to understand everything you see in this lecture – we will work through the details slowly later in the lecture series."
]
},
{
"cell_type": "markdown",
"id": "c8c26017",
"metadata": {},
"source": [
"### Can’t I Just Use ChatGPT?\n",
"\n",
"No!\n",
"\n",
"It’s tempting to think that in the age of AI we don’t need to learn how to code.\n",
"\n",
"And it’s true that AIs like [ChatGPT](https://chatgpt.com/) and other LLMs are wonderful productivity tools for coders.\n",
"\n",
"In fact an AI can be a great companion for these lectures – try copy-pasting some code from this series and ask the AI to explain it to you.\n",
"\n",
"AIs will certainly help you write pieces of code that you can combine.\n",
"\n",
"But AIs cannot completely and reliably solve a new problem that they haven’t seen before!\n",
"\n",
"You will need to be the supervisor – and for that you need to be able to read, write, and understand computer code."
]
},
{
"cell_type": "markdown",
"id": "82be55c8",
"metadata": {},
"source": [
"### Isn’t MATLAB Better?\n",
"\n",
"No, no, and one hundred times no.\n",
"\n",
"For almost all modern problems, Python’s scientific libraries are now far in advance of MATLAB’s capabilities.\n",
"\n",
"We will explain the benefits of Python’s libraries throughout this lecture\n",
"series, as well as in our later series on [JAX](https://jax.quantecon.org/intro.html).\n",
"\n",
"We will also explain how Python’s elegant design helps you write clean, efficient code.\n",
"\n",
"On top of these features, Python is more widely used, with a huge and helpful community, and free!"
]
},
{
"cell_type": "markdown",
"id": "c8e5a09c",
"metadata": {},
"source": [
"## What’s Python?\n",
"\n",
"[Python](https://www.python.org) is a general-purpose programming language conceived in 1989 by [Guido van Rossum](https://en.wikipedia.org/wiki/Guido_van_Rossum).\n",
"\n",
"Python is free and [open source](https://en.wikipedia.org/wiki/Open_source), with development coordinated through the [Python Software Foundation](https://www.python.org/psf/).\n",
"\n",
"This is important because it\n",
"\n",
"- saves us money, \n",
"- means that Python is controlled by the community of users rather than a for-profit corporation, and \n",
"- encourages reproducibility and [open science](https://en.wikipedia.org/wiki/Open_science). "
]
},
{
"cell_type": "markdown",
"id": "908b91b3",
"metadata": {},
"source": [
"### Common Uses\n",
"\n",
"Python is a general-purpose language used in almost all application domains, including\n",
"\n",
"- AI \n",
"- scientific computing \n",
"- communication \n",
"- web development \n",
"- CGI and graphical user interfaces \n",
"- game development \n",
"- resource planning \n",
"- multimedia \n",
"- etc. \n",
"\n",
"\n",
"It is used and supported extensively by tech firms including\n",
"\n",
"- [Google](https://www.google.com/) \n",
"- [OpenAI](https://openai.com/) \n",
"- [Netflix](https://www.netflix.com/) \n",
"- [Meta](https://opensource.fb.com/) \n",
"- [Dropbox](https://www.dropbox.com/) \n",
"- [Amazon](https://www.amazon.com/) \n",
"- [Reddit](https://www.reddit.com/) \n",
"- etc. "
]
},
{
"cell_type": "markdown",
"id": "327c57ce",
"metadata": {},
"source": [
"### Relative Popularity\n",
"\n",
"Python is, without doubt, one of the [most popular programming languages](https://www.tiobe.com/tiobe-index/).\n",
"\n",
"Python libraries like [pandas](https://pandas.pydata.org/) and [Polars](https://pola.rs/) are replacing familiar tools like Excel and VBA as an essential skill in the fields of finance and banking.\n",
"\n",
"Moreover, Python is extremely popular within the scientific community – especially AI\n",
"\n",
"The following chart, produced using Stack Overflow Trends, provides some evidence.\n",
"\n",
"It shows the popularity of a Python AI library called [PyTorch](https://pytorch.org/) relative to MATLAB.\n",
"\n",
"![https://python-programming.quantecon.org/_static/lecture_specific/about_py/pytorch_vs_matlab.png](https://python-programming.quantecon.org/_static/lecture_specific/about_py/pytorch_vs_matlab.png)\n",
"\n",
" \n",
"The chart shows that MATLAB’s popularity has faded, while PyTorch is growing rapidly.\n",
"\n",
"Moreover, PyTorch is just one of the thousands of Python libraries available for scientic computing."
]
},
{
"cell_type": "markdown",
"id": "d0318359",
"metadata": {},
"source": [
"### Features\n",
"\n",
"Python is a [high-level language](https://en.wikipedia.org/wiki/High-level_programming_language), which means it is relatively easy to read, write and debug.\n",
"\n",
"It has a relatively small core language that is easy to learn.\n",
"\n",
"This core is supported by many libraries, which you can learn to use as required.\n",
"\n",
"Python is very beginner-friendly\n",
"\n",
"- suitable for students learning programming \n",
"- used in many undergraduate and graduate programs \n",
"\n",
"\n",
"Other features of Python:\n",
"\n",
"- multiple programming styles are supported (procedural, object-oriented, functional, etc.) \n",
"- [interpreted](https://en.wikipedia.org/wiki/Interpreter_%28computing%29) rather than [compiled](https://en.wikipedia.org/wiki/Compiler) ahead of time. "
]
},
{
"cell_type": "markdown",
"id": "66e87d31",
"metadata": {},
"source": [
"### Syntax and Design\n",
"\n",
"\n",
"\n",
"One reason for Python’s popularity is its simple and elegant design — we’ll see many examples later on.\n",
"\n",
"To get a feeling for this, let’s look at an example.\n",
"\n",
"The code below is written in [Java](https://en.wikipedia.org/wiki/Java_%28programming_language%29) rather than Python.\n",
"\n",
"You do **not** need to read and understand this code!"
]
},
{
"cell_type": "markdown",
"id": "1370e055",
"metadata": {
"hide-output": false
},
"source": [
"```java\n",
"import java.io.BufferedReader;\n",
"import java.io.FileReader;\n",
"import java.io.IOException;\n",
"\n",
"public class CSVReader {\n",
" public static void main(String[] args) {\n",
" String filePath = \"data.csv\"; \n",
" String line;\n",
" String splitBy = \",\";\n",
" int columnIndex = 1; \n",
" double sum = 0;\n",
" int count = 0;\n",
"\n",
" try (BufferedReader br = new BufferedReader(new FileReader(filePath))) {\n",
" while ((line = br.readLine()) != null) {\n",
" String[] values = line.split(splitBy);\n",
" if (values.length > columnIndex) {\n",
" try {\n",
" double value = Double.parseDouble(\n",
" values[columnIndex]\n",
" );\n",
" sum += value;\n",
" count++;\n",
" } catch (NumberFormatException e) {\n",
" System.out.println(\n",
" \"Skipping non-numeric value: \" + \n",
" values[columnIndex]\n",
" );\n",
" }\n",
" }\n",
" }\n",
" } catch (IOException e) {\n",
" e.printStackTrace();\n",
" }\n",
"\n",
" if (count > 0) {\n",
" double average = sum / count;\n",
" System.out.println(\n",
" \"Average of the second column: \" + average\n",
" );\n",
" } else {\n",
" System.out.println(\n",
" \"No valid numeric data found in the second column.\"\n",
" );\n",
" }\n",
" }\n",
"}\n",
"```\n"
]
},
{
"cell_type": "markdown",
"id": "95e9ca2c",
"metadata": {},
"source": [
"This Java code opens an imaginary file called `data.csv` and computes the mean\n",
"of the values in the second column.\n",
"\n",
"Even without knowing Java, you can see that the program is long and complex.\n",
"\n",
"Here’s Python code that does the same thing.\n",
"\n",
"Even if you don’t yet know Python, you can see that the code is simpler and\n",
"easier to read."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2bfdc70e",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"import csv\n",
"\n",
"total, count = 0, 0\n",
"with open(data.csv, mode='r') as file:\n",
" reader = csv.reader(file)\n",
" for row in reader:\n",
" try:\n",
" total += float(row[1])\n",
" count += 1\n",
" except (ValueError, IndexError):\n",
" pass\n",
"print(f\"Average: {total / count if count else 'No valid data'}\")"
]
},
{
"cell_type": "markdown",
"id": "50428602",
"metadata": {},
"source": [
"The simplicity of Python and its neat design are a big factor in its popularity."
]
},
{
"cell_type": "markdown",
"id": "3f786bb8",
"metadata": {},
"source": [
"### The AI Connection\n",
"\n",
"Unless you have been living under a rock and avoiding all contact with the\n",
"modern world, you will know that AI is rapidly advancing.\n",
"\n",
"AI is already remarkably good at helping you write code, as discussed above.\n",
"\n",
"No doubt AI will take over many tasks currently performed by humans,\n",
"just like other forms of machinery have done over the past few centuries.\n",
"\n",
"Python is playing a huge role in the advance of AI and machine learning.\n",
"\n",
"This means that tech firms are pouring money into development of extremely\n",
"powerful Python libraries.\n",
"\n",
"Even if you don’t plan to work on AI and machine learning, you can benefit from\n",
"learning to use some of these libraries for your own projects in economics,\n",
"finance and other fields of science.\n",
"\n",
"These lectures will explain how."
]
},
{
"cell_type": "markdown",
"id": "3dcf8b96",
"metadata": {},
"source": [
"## Scientific Programming with Python\n",
"\n",
"\n",
"\n",
"We have already discussed the importance of Python for AI, machine learning and data science\n",
"\n",
"Let’s take a look at the role of Python in other areas of scientific computing.\n",
"\n",
"Python is either the dominant player or a major player in\n",
"\n",
"- astronomy \n",
"- chemistry \n",
"- computational biology \n",
"- meteorology \n",
"- natural language processing \n",
"- etc. \n",
"\n",
"\n",
"Use of Python is also rising in economics, finance, and adjacent fields like\n",
"operations research – which were previously dominated by MATLAB / Excel / STATA / C / Fortran.\n",
"\n",
"This section briefly showcases some examples of Python for general scientific programming."
]
},
{
"cell_type": "markdown",
"id": "7084a840",
"metadata": {},
"source": [
"### NumPy\n",
"\n",
"\n",
"\n",
"One of the most important parts of scientific computing is working with data.\n",
"\n",
"Data is often stored in matrices, vectors and arrays.\n",
"\n",
"We can create a simple array of numbers with pure Python as follows:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "efb44e38",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"a = [-3.14, 0, 3.14] # A Python list\n",
"a"
]
},
{
"cell_type": "markdown",
"id": "ad832f4e",
"metadata": {},
"source": [
"This array is very small so it’s fine to work with pure Python.\n",
"\n",
"But when we want to work with larger arrays in real programs we need more efficiency and more tools.\n",
"\n",
"For this we need to use libraries for working with arrays.\n",
"\n",
"For Python, the most important matrix and array processing library is\n",
"[NumPy](http://www.numpy.org/) library.\n",
"\n",
"For example, let’s build a NumPy array with 100 elements"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ddb6833a",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"import numpy as np # Load the library\n",
"\n",
"a = np.linspace(-np.pi, np.pi, 100) # Create even grid from -π to π\n",
"a"
]
},
{
"cell_type": "markdown",
"id": "5dfc2e40",
"metadata": {},
"source": [
"Now let’s transform this array by applying functions to it."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "67a98e90",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"b = np.cos(a) # Apply cosine to each element of a\n",
"c = np.sin(a) # Apply sin to each element of a"
]
},
{
"cell_type": "markdown",
"id": "d1d79e73",
"metadata": {},
"source": [
"Now we can easily take the inner product of `b` and `c`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7fa027b2",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"b @ c"
]
},
{
"cell_type": "markdown",
"id": "10405234",
"metadata": {},
"source": [
"We can also do many other tasks, like\n",
"\n",
"- compute the mean and variance of arrays \n",
"- build matrices and solve linear systems \n",
"- generate random arrays for simulation, etc. \n",
"\n",
"\n",
"We will discuss the details later in the lecture series, where we cover NumPy in depth."
]
},
{
"cell_type": "markdown",
"id": "4cc1e4c2",
"metadata": {},
"source": [
"### NumPy Alternatives\n",
"\n",
"While NumPy is still the king of array processing in Python, there are now\n",
"important competitors.\n",
"\n",
"Libraries such as [JAX](https://github.com/google/jax), [Pytorch](https://pytorch.org/), and [CuPy](https://cupy.dev/) also have\n",
"built in array types and array operations that can be very fast and efficient.\n",
"\n",
"In fact these libraries are better at exploiting parallelization and fast hardware, as\n",
"we’ll explain later in this series.\n",
"\n",
"However, you should still learn NumPy first because\n",
"\n",
"- NumPy is simpler and provides a strong foundation, and \n",
"- libraries like JAX directly extend NumPy functionality and hence are easier to\n",
" learn when you already know NumPy. "
]
},
{
"cell_type": "markdown",
"id": "19428881",
"metadata": {},
"source": [
"### SciPy\n",
"\n",
"The [SciPy](http://www.scipy.org) library is built on top of NumPy and provides additional functionality.\n",
"\n",
"\n",
"\n",
"For example, let’s calculate $ \\int_{-2}^2 \\phi(z) dz $ where $ \\phi $ is the standard normal density."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fba8c60b",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"from scipy.stats import norm\n",
"from scipy.integrate import quad\n",
"\n",
"ϕ = norm()\n",
"value, error = quad(ϕ.pdf, -2, 2) # Integrate using Gaussian quadrature\n",
"value"
]
},
{
"cell_type": "markdown",
"id": "9e7bca63",
"metadata": {},
"source": [
"SciPy includes many of the standard routines used in\n",
"\n",
"- [linear algebra](http://docs.scipy.org/doc/scipy/reference/linalg.html) \n",
"- [integration](http://docs.scipy.org/doc/scipy/reference/integrate.html) \n",
"- [interpolation](http://docs.scipy.org/doc/scipy/reference/interpolate.html) \n",
"- [optimization](http://docs.scipy.org/doc/scipy/reference/optimize.html) \n",
"- [distributions and statistical techniques](http://docs.scipy.org/doc/scipy/reference/stats.html) \n",
"- [signal processing](http://docs.scipy.org/doc/scipy/reference/signal.html) \n",
"\n",
"\n",
"See them all [here](http://docs.scipy.org/doc/scipy/reference/index.html).\n",
"\n",
"Later we’ll discuss SciPy in more detail."
]
},
{
"cell_type": "markdown",
"id": "915e117e",
"metadata": {},
"source": [
"### Graphics\n",
"\n",
"\n",
"\n",
"A major strength of Python is data visualization.\n",
"\n",
"The most popular and comprehensive Python library for creating figures and graphs is [Matplotlib](http://matplotlib.org/), with functionality including\n",
"\n",
"- plots, histograms, contour images, 3D graphs, bar charts etc. \n",
"- output in many formats (PDF, PNG, EPS, etc.) \n",
"- LaTeX integration \n",
"\n",
"\n",
"Example 2D plot with embedded LaTeX annotations\n",
"\n",
"![https://python-programming.quantecon.org/_static/lecture_specific/about_py/qs.png](https://python-programming.quantecon.org/_static/lecture_specific/about_py/qs.png)\n",
"\n",
" \n",
"Example contour plot\n",
"\n",
"![https://python-programming.quantecon.org/_static/lecture_specific/about_py/bn_density1.png](https://python-programming.quantecon.org/_static/lecture_specific/about_py/bn_density1.png)\n",
"\n",
" \n",
"Example 3D plot\n",
"\n",
"![https://python-programming.quantecon.org/_static/lecture_specific/about_py/career_vf.png](https://python-programming.quantecon.org/_static/lecture_specific/about_py/career_vf.png)\n",
"\n",
" \n",
"More examples can be found in the [Matplotlib thumbnail gallery](https://matplotlib.org/stable/gallery/index.html).\n",
"\n",
"Other graphics libraries include\n",
"\n",
"- [Plotly](https://plot.ly/python/) \n",
"- [seaborn](https://seaborn.pydata.org/) — a high-level interface for matplotlib \n",
"- [Altair](https://altair-viz.github.io/) \n",
"- [Bokeh](http://bokeh.pydata.org/en/latest/) \n",
"\n",
"\n",
"You can visit the [Python Graph Gallery](https://www.python-graph-gallery.com/) for more example plots drawn using a variety of libraries."
]
},
{
"cell_type": "markdown",
"id": "3c35f44b",
"metadata": {},
"source": [
"### Networks and Graphs\n",
"\n",
"The study of networks and graphs becoming an important part of scientific work\n",
"in economics, finance and other fields.\n",
"\n",
"For example, we are interesting in studying\n",
"\n",
"- production networks \n",
"- networks of banks and financial institutions \n",
"- friendship and social networks \n",
"- etc. \n",
"\n",
"\n",
"(We have a [book on economic networks](https://networks.quantecon.org/) if you would like to learn more.)\n",
"\n",
"Python has many libraries for studying networks and graphs.\n",
"\n",
"\n",
"\n",
"One well-known example is [NetworkX](http://networkx.github.io/).\n",
"\n",
"Its features include, among many other things:\n",
"\n",
"- standard graph algorithms for analyzing networks \n",
"- plotting routines \n",
"\n",
"\n",
"Here’s some example code that generates and plots a random graph, with node color determined by the shortest path length from a central node."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1af366c4",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"import networkx as nx\n",
"import matplotlib.pyplot as plt\n",
"np.random.seed(1234)\n",
"\n",
"# Generate a random graph\n",
"p = dict((i, (np.random.uniform(0, 1), np.random.uniform(0, 1)))\n",
" for i in range(200))\n",
"g = nx.random_geometric_graph(200, 0.12, pos=p)\n",
"pos = nx.get_node_attributes(g, 'pos')\n",
"\n",
"# Find node nearest the center point (0.5, 0.5)\n",
"dists = [(x - 0.5)**2 + (y - 0.5)**2 for x, y in list(pos.values())]\n",
"ncenter = np.argmin(dists)\n",
"\n",
"# Plot graph, coloring by path length from central node\n",
"p = nx.single_source_shortest_path_length(g, ncenter)\n",
"plt.figure()\n",
"nx.draw_networkx_edges(g, pos, alpha=0.4)\n",
"nx.draw_networkx_nodes(g,\n",
" pos,\n",
" nodelist=list(p.keys()),\n",
" node_size=120, alpha=0.5,\n",
" node_color=list(p.values()),\n",
" cmap=plt.cm.jet_r)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "5c2de5d1",
"metadata": {},
"source": [
"### Other Scientific Libraries\n",
"\n",
"As discussed above, there are literally thousands of scientific libraries for\n",
"Python.\n",
"\n",
"Some are small and do very specific tasks.\n",
"\n",
"Others are huge in terms of lines of code and investment from coders and tech\n",
"firms.\n",
"\n",
"Here’s a short list of some important scientific libraries for Python not\n",
"mentioned above.\n",
"\n",
"- [SymPy](http://www.sympy.org/) for symbolic algebra, including limits, derivatives and integrals \n",
"- [statsmodels](http://statsmodels.sourceforge.net/) for statistical routines \n",
"- [scikit-learn](http://scikit-learn.org/) for machine learning \n",
"- [Keras](https://keras.io/) for machine learning \n",
"- [Pyro](https://pyro.ai/) and [PyStan](https://pystan.readthedocs.org/en/latest/) for Bayesian data analysis \n",
"- [GeoPandas](https://geopandas.org/en/stable/) for spatial data analysis \n",
"- [Dask](https://docs.dask.org/en/stable/) for parallelization \n",
"- [Numba](http://numba.pydata.org/) for making Python run at the same speed as native machine code \n",
"- [CVXPY](https://www.cvxpy.org/) for convex optimization \n",
"- [scikit-image](https://scikit-image.org/) and [OpenCV](https://opencv.org/) for processing and analysing image data \n",
"- [BeautifulSoup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/) for extracting data from HTML and XML files \n",
"\n",
"\n",
"In this lecture series we will learn how to use many of these libraries for\n",
"scientific computing tasks in economics and finance."
]
}
],
"metadata": {
"date": 1729158861.4619565,
"filename": "about_py.md",
"kernelspec": {
"display_name": "Python",
"language": "python3",
"name": "python3"
},
"title": "About These Lectures"
},
"nbformat": 4,
"nbformat_minor": 5
}