Well, hopefully I can finally get around to tackling multiple users concurrently requesting the animation.

Not sure how to go about this. Nor how many concurrent users should be allowed to access the animations. Expect my free account is going to have some limits regarding resources for concurrent users. Especially as the writing to file is relatively time consuming (perhaps resource intensive).

I am thinking I will use a dictionary (in memory) keyed on the file id with the time of the last post request for the animation page. Each time the app is restarted, the dictionary will start from scratch. At some point (probably whenever the home or help pages are loaded) I will review the dictionary and drop any entries that have not been updated for at least 10 minutes. I am also going to try and set a 10 to 30 minute time limit on the sessions.

Initially I will only allow 5 entries in the dictionary. If any user would be the 6th or greater, they will get a message saying, sorry no can do.

Adding Multiple Users

Started by adding some new globals in g_vars.py. These two are likely just the start; there will, I am sure, be more.

# animation related vars
max_ani = 5     # max animation file ids
file_ids = {}   # key = string, letter or two?; value = timestamp of last animation request

Then I modified set_fileId() in main.py to randomly generate new file ids. Currently limited to the 26 letters of the lowercase alphabet.

def set_fileId():
  if len(g.file_ids) >= g.max_ani:
    return None
  if 'fileId' not in session:
    fid = chr(ord('a') + sal.rng.integers(0, 26))
    while fid in g.file_ids:
      fid = chr(ord('a') + sal.rng.integers(0, 26))
    session['fileId'] = fid
    g.file_ids[fid] = time.perf_counter()
  else:
    fid = session['fileId']
    # new request so give user more time
    g.file_ids[fid] = time.perf_counter()
  return fid

I then refactored the route, ani_basic(). I added afl_id = set_fileId() at the top of the POST request block. I then had to account for the possiblity that afl_id is None. I put most of the current code in an if block, if afl_id is not None:. To prevent a few errors, I had to move the curve generation, and definition of some of the data dictionaries being sent to the web template outside the if block. I also now add the file id to the data being sent to the template.

Then I modified the template so that if it got a file id, it proceeds as expected and displays the video and the curve related data. Otherwise (i.e. a file id of None) it would display a message advising the user that the maximum number of animation tickets had been issued. And to try again later. But, there was a bit of a hiccup. Jinja doesn’t use None, it uses none. So the if block in the template looks like, {% if p_data["file_id"] is not none %}.

I figured I didn’t need to include the code for the above refactorings in the post.

Bug

I only tested with three browser windows, so missed testing some edge cases. But while looking at the code for set_fileId() I realized I had a serious bug—logical bug— in my code. Did/do you see it? As it stands, if there are five users registered for accessing animations, even if a valid user requests a new animation, they will be denied.

I should first check if there is a session with a file id. If so, check whether or not it is in the dictionary of current file ids. If yes, proceed accordingly. If not, then check to see if maximum users reached and proceed accordingly. So, I think the following is now correct.

def set_fileId():
  fid == None
  if 'fileId' in session:
    fid = session['fileId']
    if fid in g.file_ids:
      g.file_ids[fid] = time.perf_counter()
      return fid
  if len(g.file_ids) >= g.max_ani:
    return None
  else:
    fid = chr(ord('a') + sal.rng.integers(0, 26))
    while fid in g.file_ids:
      fid = chr(ord('a') + sal.rng.integers(0, 26))
    session['fileId'] = fid
    g.file_ids[fid] = time.perf_counter()
    return fid

Simple Test

Opened three browser windows (Firefox, Firefox private, Chrome). Requested animations in all three. Here’s the contents of g.file_ids after the third request: file ids: {'a': 1602146.7856067, 's': 1602145.2629602, 'y': 1602136.9321456}.

But, the dev server is definitely not scaleable. Here’s the timings for generating the videos.

File id: y => Save animation to file: 33.9418 seconds
  More or less total time prior to render: 34.5130 seconds

File id: a => Save animation to file: 62.0242 seconds
  More or less total time prior to render: 62.3701 seconds

File id: s => Save animation to file: 62.7477 seconds
  More or less total time prior to render: 63.1937 seconds

In fact, things are even worse than I expected. All three videos were more or less the same. One of them was incomplete and very jumpy. The others also showed some jaggedness.

Big Trouble

Not going to be an easy fix. All three browsers simultaneously made a request to the same route. I would have expected each call to end up as a separate entry on the stack, along with it’s own variables. But, I use a good many global variables. The current curve is defined in those globals. So the last call would have overwritten those globals. And effectively overwrote the animation being saved by the three requests.

For example, the three calls, in sequence, showed the following curve parameters.

File id: y
  Wheels: 12 wheels (shape(s): square (s))
  Symmetry: k_fold = 3, congruency = 1

File id: s
  Wheels: 6 wheels (shape(s): square (s))
  Symmetry: k_fold = 3, congruency = 1

File id: a
  Wheels: 6 wheels (shape(s): tetracuspid (t))
  Symmetry: k_fold = 3, congruency = 1

Took about 60 seconds for all three to get something displayed. All very jagged. All in different files. But all a jagged video of the curve generated using the tetracuspids.

Unfortunately, given my code, there is no easy fix. It will take a very significant refactoring of the main spirograph module(s) to come up with a workable solution. I am not sure I am prepared to do that at this time.

Will look into using a semaphore, before I get too creative or distraught. Though that is most likely going to affect performance. But if the WSGI server on PythonAnywhere spawns multiple processes things might work in that enviroment. Guess I will need to test that out as well.

Test in Production

Okay, I uploaded the code as it stands to my pythonanywhere project and restarted the app.

I opened three browser windows and quickly, in succession, clicked the buttons to generate and display an animation. Lo and behold, each window ended up with a different animation. No jaggies. And all three in roughly the same amount of time. Which tends to be faster than on my dev system. The times were not stacked on one another as was the case for the development server. So, I am not going to try to fix the code to work on the dev server as it appears to work as desired in production.

But, I will have to limit my multiple user testing to the production environment.

Add Session Duration

Added a new session related variable in g_vars.py: max_life = 5 # maximum session duration in minutes, since latest request. Added a new import, from datetime import timedelta. Then after the line setting the app secret key, added this one: app.permanent_session_lifetime = timedelta(minutes=g.max_life). Then modified set_fileID() to set session.permanent.

def set_fileId():
  fid = None
  if 'fileId' in session:
    fid = session['fileId']
    session.permanent = True
    if fid in g.file_ids:
      g.file_ids[fid] = time.perf_counter()
      return fid
  if len(g.file_ids) >= g.max_ani:
    return None
  else:
    fid = chr(ord('a') + sal.rng.integers(0, 26))
    while fid in g.file_ids:
      fid = chr(ord('a') + sal.rng.integers(0, 26))
    session['fileId'] = fid
    session.permanent = True
    g.file_ids[fid] = time.perf_counter()
    return fid

And, that seems to work just fine. At least the browser shows the expected expiration datetime for the cookie. Only problem is that it is of no help in controlling the entries in g.file_ids.

Removing File Ids Not Used for Some Duration

That duration will also be provided by g.max_life. I am going to add a new function to main.py, though it likely could go in sp_app_lib.py. And I will call it at the top of the route’s code before the call to set_fileId().

def purge_sessions():
  f_ids = list(g.file_ids.keys())
  c_tm = time.perf_counter()
  dur_secs = g.max_life * 60
  for fid in f_ids:
    if c_tm - g.file_ids[fid] >= dur_secs:
      del g.file_ids[fid]

The does appear to work in some limited testing. But, you may be wondering about why I didn’t just iterate over g.file_ids.items() in the for loop. Go ahead and give it a try. But I can assure you Python will generate a run-time error. It will complain, rather loudly, that RuntimeError: dictionary changed size during iteration. I had forgotten about that and did actually try to delete entries in a loop iterating over the dictionary. Hopefully I will remember next time.

But, I also want to delete the related file from the static directory, though I am not sure that is really necessary. I am doing so mainly because I could potentially have 26 mp4 files stored in that directory. Seems a touch unneccesary given I am only allowing 5 users to access animations at any one time.

After some thought, I decided it made more sense and would likely save a little time when calling the purge function, if I just used less possible file ids. The files are relatively small and don’t currently seem to impact my file quota on pythonanywhere. So, I am going to limit the ids to the letters in spirograph. 9 possible ids for 5 possible users, seems reasonable.

I added a new global poss_ids = list("spirograph"). (Yes, I know two ‘r’s. If that bothers you, take the “set” of the list and convert back to a list.) I used the random choice() method to select one that isn’t already in the current id dictionary.

def set_fileId():
  fid = None
  if 'fileId' in session:
    fid = session['fileId']
    session.permanent = True
    if fid in g.file_ids:
      g.file_ids[fid] = time.perf_counter()
      return fid
  if len(g.file_ids) >= g.max_ani:
    return None
  else:
    fid = sal.rng.choice(g.poss_ids)
    while fid in g.file_ids:
      fid = sal.rng.choice(g.poss_ids)
    session['fileId'] = fid
    session.permanent = True
    g.file_ids[fid] = time.perf_counter()
    return fid

With some basic testing things appear to work as expected. So, will upload to production and do somemore testing.

Note: as I was reviewing the post prior to publishing (it was written weeks prior), I realized I could likely use set operations to remove that repeated loop (if fid in g.file_ids:) in the set_fileId function. May eventually do so, just as an exercise in basic logic.

Done

I think that’s it for this one. I still haven’t added this animation to the menu, so a bit more to do. But, I think I am going to try and get another style of animation working before I look at making the animations generally available to all visitors.

Until then, have may your animations whirl dervishly.

Resources