Was going to start on the CS50 problem set for Week 9, “Finance”. Which is a Flask based web application for buying and selling stocks. It uses a free IEX Cloud account to get the stock prices. This requires the use of an API key — which of course needs to be protected. Think environment variable.
Didn’t want to mess with the CS50 IDE for something this relatively involved. So decided to set it up on my development system (AMD Ryzen 7 1700, Radeon RX 480, 16GB RAM, Windows 10 Pro).
Sounds simpler than it is.
New Virtual Environment
As mentioned in my earliest posts I use conda (specifically miniconda) to manage my virtual environments. Though I do periodically need to resort to pip to get a module. But I do so within a conda environment. So, I decided to create a new virtual environment specifically for this app. And, I decided to use the latest version of Python available with conda, 3.9.x.
As also mentioned in a previous post, I use Windows Terminal and have configured short cuts to open a terminal with a number of tabs specific to various projects. I created one for working on CS50. Among the tabs opened is an Anaconda Powershell terminal. In that tab, after changing to my problem set directory, I created that new virtural environment.
(base) PS R:\learn\edX\cs50\week9> conda create -n flask-3.9 python=3.9
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: E:\appDev\Miniconda3\envs\flask-3.9
added / updated specs:
- python=3.9
The following packages will be downloaded:
...
The following NEW packages will be INSTALLED:
...
Downloading and Extracting Packages
...
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate flask-3.9
#
# To deactivate an active environment, use
#
# $ conda deactivate
(base) PS R:\learn\edX\cs50\week9> conda activate flask-3.9
(flask-3.9) PS R:\learn\edX\cs50\week9> python.exe --version
Python 3.9.2
CS50 Python Module
After setting up the problem set 9 workspace on I checked the requirements.txt file. It read:
cs50
Flask
Flask-Session
requests
So, I decided to try installing the CS50 module in my new environment.
(flask-3.9) PS R:\learn\edX\cs50\week9> conda search cs50
Loading channels: done
No match found for: cs50. Search: *cs50*
PackagesNotFoundError: The following packages are not available from current channels:
- cs50
Current channels:
- https://repo.anaconda.com/pkgs/main/win-64
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/r/win-64
- https://repo.anaconda.com/pkgs/r/noarch
- https://repo.anaconda.com/pkgs/msys2/win-64
- https://repo.anaconda.com/pkgs/msys2/noarch
To search for alternate channels that may provide the conda package you are
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
Okay, so I checked the CS50 Library for Python documentation. And, it said:
Installation
$ pip3 install cs50
So, I decided to searh PyPI.
(flask-3.9) PS R:\learn\edX\cs50\week9> python.exe -m pip search cs50
ERROR: XMLRPC request failed [code: -32500]
RuntimeError: PyPI's XMLRPC API is currently disabled due to unmanageable load and will be deprecated in the near future. See https://status.python.org/ for more information.
A search on the PyPI website got me cs50 6.0.3. So I went with pip.
(flask-3.9) PS R:\learn\edX\cs50\week9> pip install cs50
Collecting cs50
Downloading cs50-6.0.3.tar.gz (8.5 kB)
Collecting Flask>=1.0
Downloading Flask-1.1.2-py2.py3-none-any.whl (94 kB)
|████████████████████████████████| 94 kB 753 kB/s
Collecting SQLAlchemy
Downloading SQLAlchemy-1.3.23-cp39-cp39-win_amd64.whl (1.2 MB)
|████████████████████████████████| 1.2 MB 6.8 MB/s
Collecting sqlparse
Downloading sqlparse-0.4.1-py3-none-any.whl (42 kB)
|████████████████████████████████| 42 kB 393 kB/s
Collecting termcolor
Downloading termcolor-1.1.0.tar.gz (3.9 kB)
Downloading click-7.1.2-py2.py3-none-any.whl (82 kB)
|████████████████████████████████| 82 kB 446 kB/s
Collecting Werkzeug>=0.15
Downloading Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)
Collecting itsdangerous>=0.24
Using cached itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting Jinja2>=2.10.1
Downloading Jinja2-2.11.3-py2.py3-none-any.whl (125 kB)
|████████████████████████████████| 125 kB 6.4 MB/s
Collecting MarkupSafe>=0.23
Downloading MarkupSafe-1.1.1-cp39-cp39-win_amd64.whl (16 kB)
Building wheels for collected packages: cs50, termcolor
Building wheel for cs50 (setup.py) ... done
Created wheel for cs50: filename=cs50-6.0.3-py3-none-any.whl size=8745 sha256=20e006049af82dfd41f284b2a3e7343314dc776c63bbb87198eb69a583624d0e
Stored in directory: c:\users\bark\appdata\local\pip\cache\wheels\3d\d5\6f\e934a089240a8b3ebec2d6af3c23aeb3b4bc1414b9b560c0d9
Building wheel for termcolor (setup.py) ... done
Created wheel for termcolor: filename=termcolor-1.1.0-py3-none-any.whl size=4829 sha256=f0a90161acfd7548b276f1126250ebe182f4ff85aa329a0906848f767305c427
Stored in directory: c:\users\bark\appdata\local\pip\cache\wheels\b6\0d\90\0d1bbd99855f99cb2f6c2e5ff96f8023fad8ec367695f7d72d
Successfully built cs50 termcolor
Installing collected packages: MarkupSafe, Werkzeug, Jinja2, itsdangerous, click, termcolor, sqlparse, SQLAlchemy, Flask, cs50
Successfully installed Flask-1.1.2 Jinja2-2.11.3 MarkupSafe-1.1.1 SQLAlchemy-1.3.23 Werkzeug-1.0.1 click-7.1.2 cs50-6.0.3 itsdangerous-1.1.0 sqlparse-0.4.1 termcolor-1.1.0
Must admit I wasn’t quite prepared for what I got. Guess the CS50 package had its own prerequisites.
(flask-3.9) PS R:\learn\edX\cs50\week9> conda list
# packages in environment at E:\appDev\Miniconda3\envs\flask-3.9:
#
# Name Version Build Channel
ca-certificates 2021.1.19 haa95532_1
certifi 2020.12.5 py39haa95532_0
click 7.1.2 pypi_0 pypi
cs50 6.0.3 pypi_0 pypi
flask 1.1.2 pypi_0 pypi
itsdangerous 1.1.0 pypi_0 pypi
jinja2 2.11.3 pypi_0 pypi
markupsafe 1.1.1 pypi_0 pypi
openssl 1.1.1j h2bbff1b_0
pip 21.0.1 py39haa95532_0
python 3.9.2 h6244533_0
setuptools 52.0.0 py39haa95532_0
sqlalchemy 1.3.23 pypi_0 pypi
sqlite 3.33.0 h2a8f88b_0
sqlparse 0.4.1 pypi_0 pypi
termcolor 1.1.0 pypi_0 pypi
tzdata 2020f h52ac0ba_0
vc 14.2 h21ff451_1
vs2015_runtime 14.27.29016 h5e58377_2
werkzeug 1.0.1 pypi_0 pypi
wheel 0.36.2 pyhd3eb1b0_0
wincertstore 0.2 py39h2bbff1b_0
zlib 1.2.11 h62dcd97_4
Well that looked to me like I had enough to trying running an application. I had already done Lab9 so I downloaded those files into an appropriate directory and gave it a shot. A little set up required. CS50 IDE has flask run configured to look for an application.py file. Apparently out of the box, flask looks for app.py.
(flask-3.9) PS R:\learn\edX\cs50\week9> cd lab9
(flask-3.9) PS R:\learn\edX\cs50\week9\lab9> flask run
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
Usage: flask run [OPTIONS]
Error: Could not locate a Flask application. You did not provide the "FLASK_APP" environment variable, and a "wsgi.py" or "app.py" module was not found in the current directory.
(flask-3.9) PS R:\learn\edX\cs50\week9\lab9>$env:FLASK_APP = "application.py"
(flask-3.9) PS R:\learn\edX\cs50\week9\lab9> flask run
* Serving Flask app "application.py"
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
INFO: * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
DEBUG: SELECT name, MONTH, DAY FROM birthdays
INFO: 127.0.0.1 - - [21/Mar/2021 16:16:07] "GET / HTTP/1.1" 200 -
INFO: 127.0.0.1 - - [21/Mar/2021 16:16:08] "GET /static/styles.css HTTP/1.1" 200 -
And, sure enough I could access the web page for the Lab9 exercise (Birthdays).
SQLite Tools
I figured the next thing I would do is get the SQLite tools installed. I expected I would need to check/modify the database for the project set at the command line. So, I downloaded the zip file of the tools for Windows. Created a suitable directory on my dev apps partition, and extracted the zip into that location.
Then I created a directory for the problem set, and downloaded the set-up files (http://cdn.cs50.net/2020/fall/psets/9/finance/finance.zip). Next I had a look at the database.
(flask-3.9) PS R:\learn\edX\cs50\week9\lab9> cd ..\finance
(flask-3.9) PS R:\learn\edX\cs50\week9\finance> e:\appDev\sqlite\sqlite3.exe finance.db
SQLite version 3.35.0 2021-03-12 15:10:09
Enter ".help" for usage hints.
sqlite> .schema
CREATE TABLE users (id INTEGER, username TEXT NOT NULL, hash TEXT NOT NULL, cash NUMERIC NOT NULL DEFAULT 10000.00, PRIMARY KEY(id));
CREATE UNIQUE INDEX username ON users (username);
sqlite> .quit
(flask-3.9) PS R:\learn\edX\cs50\week9\finance>
Okay that appears to work.
Check That App Runs in Flask
Ok, since the initial problem set files were downloaded and installed. I thought I best check it runs as expected.
(base) PS R:\learn\edX\cs50\week9\finance> conda activate flask-3.9
(flask-3.9) PS R:\learn\edX\cs50\week9\finance> $env:FLASK_APP = "application.py"
(flask-3.9) PS R:\learn\edX\cs50\week9\finance> $env:API_KEY = "pk_???????????"
(flask-3.9) PS R:\learn\edX\cs50\week9\finance> flask run
* Serving Flask app "application.py"
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
Usage: flask run [OPTIONS]
Error: While importing "application", an ImportError was raised:
Traceback (most recent call last):
File "e:\appdev\miniconda3\envs\test-3.9\lib\site-packages\flask\cli.py", line 240, in locate_app
__import__(module_name)
File "R:\learn\edX\cs50\week9\finance\application.py", line 5, in <module>
from flask_session import Session
ModuleNotFoundError: No module named 'flask_session'
Well, guess best install flask_session. And, another one I couldn’t find with conda.
(flask-3.9) PS R:\learn\edX\cs50\week9\finance> pip install Flask-Session
Collecting Flask-Session
...
Flask-Session) (1.1.2)
Collecting cachelib
...
Installing collected packages: cachelib, Flask-Session
Successfully installed Flask-Session-0.3.2 cachelib-0.1.1
(flask-3.9) PS R:\learn\edX\cs50\week9\finance>
And, another import error ModuleNotFoundError: No module named 'requests'. So, let’s install requests: (flask-3.9) PS R:\learn\edX\cs50\week9\finance> conda install requests. And try again.
(flask-3.9) PS R:\learn\edX\cs50\week9\finance> flask run
* Serving Flask app "application.py"
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
INFO: * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
INFO: 127.0.0.1 - - [22/Mar/2021 10:06:57] "GET / HTTP/1.1" 302 -
INFO: 127.0.0.1 - - [22/Mar/2021 10:06:57] "GET /login HTTP/1.1" 200 -
INFO: 127.0.0.1 - - [22/Mar/2021 10:07:20] "GET /login HTTP/1.1" 200 -
INFO: 127.0.0.1 - - [22/Mar/2021 10:07:21] "GET /static/styles.css HTTP/1.1" 200 -
INFO: 127.0.0.1 - - [22/Mar/2021 10:07:21] "GET /static/favicon.ico HTTP/1.1" 200 -
INFO: 127.0.0.1 - - [22/Mar/2021 10:07:38] "GET /register HTTP/1.1" 400 -
INFO: 127.0.0.1 - - [22/Mar/2021 10:07:38] "GET /static/styles.css HTTP/1.1" 200 -
And that did appear to work. But, I didn’t really want to keep re-entering the above environment variables every time I started working on the problem set (well at least if I didn’t have to).
API Key and Envrionment Variablas
I then tried to configure the conda enviroment to automatically load an .env file. That failed miserably. So, I decided to try using the dotenv module to load an .env file in my CS50 directory that would create the above environment variables for me.
(flask-3.9) PS R:\learn\edX\cs50\week9\finance> pip install python-dotenv
Collecting python-dotenv
Using cached python_dotenv-0.15.0-py2.py3-none-any.whl (18 kB)
Installing collected packages: python-dotenv
Successfully installed python-dotenv-0.15.0
Next I created the .env file in my cs50 directory. I figured the further it was from my finance project directory the better. But, I did still add it to my .gitignore as the file was within the git repository I created for my CS50 efforts. I then wrote a simple test module to check things out.
# r:\learn\edX\cs50\.env
API_KEY=pk_???????
FLASK_APP=application.py
# r:\learn\edX\cs50\week9\finance\test.py
import os
from dotenv import load_dotenv
load_dotenv()
if os.environ.get("FLASK_APP"):
print(f"FLASK_APP = '{os.environ['FLASK_APP']}`")
if not os.environ.get("API_KEY"):
raise RuntimeError("API_KEY not set")
And?
(flask-3.9) PS R:\learn\edX\cs50\week9\finance> python.exe test.py
FLASK_APP = 'application.py`
Okay that appears to have worked. So, I edited the finance\application.py to include the dotenv module and to load the .env file. I closed and reopened my terminal window. Wanted to make sure the enviroment variables I had set manually were cleared.
import os
from dotenv import load_dotenv
...
# Load environment variables, and print for testing/debugging
load_dotenv()
if os.environ.get("FLASK_APP"):
print(f"FLASK_APP = '{os.environ['FLASK_APP']}`")
(base) PS R:\learn\edX\cs50\week9\finance> conda activate flask-3.9
(flask-3.9) PS R:\learn\edX\cs50\week9\finance> flask run
* Serving Flask app "application.py"
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
Usage: flask run [OPTIONS]
Error: Could not import "application".
I spent a lot of time messing about over a period of a few days. Found an article on the web that suggested I add the following to the bottom of application.py and run it directly.
if __name__ == "__main__":
app.run()
(flask-3.9) PS R:\learn\edX\cs50\week9\finance> python.exe application.py
FLASK_APP = 'application.py'
* Serving Flask app "application" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
INFO: * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
INFO: 127.0.0.1 - - [22/Mar/2021 10:39:43] "GET / HTTP/1.1" 302 -
INFO: 127.0.0.1 - - [22/Mar/2021 10:39:43] "GET /login HTTP/1.1" 200 -
INFO: 127.0.0.1 - - [22/Mar/2021 10:39:43] "GET /static/styles.css HTTP/1.1" 200 -
INFO: 127.0.0.1 - - [22/Mar/2021 10:39:46] "GET /register HTTP/1.1" 400 -
INFO: 127.0.0.1 - - [22/Mar/2021 10:39:47] "GET /static/styles.css HTTP/1.1" 200 -
And, it appeared to work. So, tried flask run again. No go. Was driving me nuts.
However, I didn’t want to run the application module directly. I breezed over a post that indicated there were some differences between the two approaches. At the time I didn’t feel like digging into the matter, and still haven’t. For testing on the CS50 IDE I was going to be using flask run. So figured I best do the same on my development system.
Rebuild Conda Environment
After a day or three, I decided to delete the virtual environment and start over. As it was a new day, I did start with a clean terminal session. Which was likely a good thing, as I really didn’t know what was going on. I also removed the dotenv code from application.py.
This time, I installed a number of requirements myself before finally installing CS50. Wasn’t sure having CS50 install its additional requirements was making a difference or not. But, I had read that conda environments don’t always like conda intalls along with pip installs. So I left the latter to the last.
(flask-3.9) PS R:\learn\edX\cs50> conda deactivate flask-3.9
(base) PS R:\learn\edX\cs50> conda remove --name flask-3.9 --all
(base) PS R:\learn\edX\cs50> conda info --envs
# conda environments:
#
base * E:\appDev\Miniconda3
ani-3.8 E:\appDev\Miniconda3\envs\ani-3.8
base-3.8 E:\appDev\Miniconda3\envs\base-3.8
py30days E:\appDev\Miniconda3\envs\py30days
(base) PS R:\learn\edX\cs50> conda create -n flask-3.9 python=3.9
(base) PS R:\learn\edX\cs50> conda activate flask-3.9
(flask-3.9) PS R:\learn\edX\cs50> conda config --append channels conda-forge
(flask-3.9) PS R:\learn\edX\cs50> conda install requests
(flask-3.9) PS R:\learn\edX\cs50> pip install cs50
(flask-3.9) PS R:\learn\edX\cs50> pip install Flask-Session
(flask-3.9) PS R:\learn\edX\cs50> cd week9\finance
(flask-3.9) PS R:\learn\edX\cs50\week9\finance> $env:FLASK_APP = "application.py"
(flask-3.9) PS R:\learn\edX\cs50\week9\finance> $env:API_KEY = "pk_?????"
Executing flask run seemed to work just fine. So, I once again installed python-dotenv. And, tried running flask without adding the dotenv code to the application module. And boom!
(flask-3.9) PS R:\learn\edX\cs50\week9\finance> flask run
* Serving Flask app "application.py"
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
Usage: flask run [OPTIONS]
Error: Could not import "application".
Okay, so now I knew I needed to search the web for flask and dotenv. I found Use of dotenv causes flask cli to fail. I moved the .env file from ..\cs50 to ..\cs50\week9\finance. And?
(flask-3.9) PS R:\learn\edX\cs50\week9\finance> flask run
* Serving Flask app "application.py"
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
INFO: * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
INFO: 127.0.0.1 - - [22/Mar/2021 10:58:55] "GET / HTTP/1.1" 302 -
INFO: 127.0.0.1 - - [22/Mar/2021 10:58:56] "GET /login HTTP/1.1" 200 -
INFO: 127.0.0.1 - - [22/Mar/2021 10:58:56] "GET /static/styles.css HTTP/1.1" 200 -
INFO: 127.0.0.1 - - [22/Mar/2021 10:59:02] "POST /login HTTP/1.1" 403 -
INFO: 127.0.0.1 - - [22/Mar/2021 10:59:02] "GET /static/styles.css HTTP/1.1" 200 -
Bingo. So, now let’s activate the dotenv related code in application.py. Clear the environment variables. And try again.
(flask-3.9) PS R:\learn\edX\cs50\week9\finance> flask run
* Serving Flask app "application.py"
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
FLASK_APP = 'application.py'
INFO: * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
INFO: 127.0.0.1 - - [22/Mar/2021 11:01:15] "GET / HTTP/1.1" 302 -
INFO: 127.0.0.1 - - [22/Mar/2021 11:01:15] "GET /login HTTP/1.1" 200 -
INFO: 127.0.0.1 - - [22/Mar/2021 11:01:15] "GET /static/styles.css HTTP/1.1" 200 -
INFO: 127.0.0.1 - - [22/Mar/2021 11:01:17] "GET /register HTTP/1.1" 400 -
INFO: 127.0.0.1 - - [22/Mar/2021 11:01:18] "GET /static/styles.css HTTP/1.1" 200 -
Done!
That whole process took the best part of two or three days. I still wish I knew why using conda to create the environment variables failed to work. Will try again some day perhaps. But, after CS50 don’t know if I will be using Flask again.
By the way, the environment variables are removed once the application/Flask exits (Ctrl+C).
That’s it for this one. See you next time.