You should likely not bother reading this post. It is of interest only to me. They are notes regarding a major code refactoring. Wanted to keep the notes as a reference. My git commits didn’t do the changes justice as they were in general overly large and not well documented.

The other day, I started moving functions, variables and the like from the existing modules into some new modules I created.

The existing modules:

• main (contains functions other than routes)
• sp_app_lib.py (hodge-podge of functions, including refactored versions of some of those in spiro_plotlib)
• g_vars
• repeats
• spiro_plotlib (the original plotting code from my command line version of the app)
• spiro_get_rand (from the original command line app)

The tenative new modules, including some of the existing ones.

• main (only code related to generating the web app, i.e. mostly routes and some other set up code)
• g_vars
• repeats
• sp_app_data (anything related to generating/manipulating spirograph curve data)
• sp_app_plot (anything related to actually plotting curves or manipulating plots)
• sp_app_util (anything that didn’t fit into the preceding two modules)

I then started copying, slowly, functions from the first set of modules to the second. Early one morning (the day I started the draft of this post) it occurred to me creating the new modules then doing a single huge refactoring to get the app working with the new modules was likely a bad idea. I decided that after moving (copying) anything to one of the new modules, I should refactor the app code to use the code/function in the new module.

This occurred to me because I realized that the refactoring covered in the previous few posts altered functions I had already moved to the new modules. Creating a bit of a nightmare. So, going to start over.

Don’t know that there will be anything more to add to this post. Time will tell. Could end up being the shortest of my current posts.

Start Simple

Okay, I took the least invasive change I could find for a test. I moved two functions from the main module to the sp_app_util module. Namely, purge_sessions() and set_fileId(). Only used in the spirograph animation routes.

In the main module, I commented out the two functions and added an import to the main module: import sp_app_util as sau. Then modified the calls for the functions to read: sau.purge_sessions() and sau.set_fileId(). There were only two changes for both functions so didn’t use the find/replace feature of VSCode. Unusual for me, love to use a regex any chance I get.

Once that was done I called the basic animation route and everything worked fine. Multiple times when using the Draw another one! button. Not so much if I repeated page request (icon or Ctrl-R). Expect there is an issue with one of my recent refactors as I did not test them with animation routes. But for now I will consider this refactor a success. Make note to sort error.

History

Session 1)

• copy current version of get_curve_data() from main to sp_app_data
  • add import to main: import sp_app_data as sad
  • refactor calls, regex: s/= get_curve_data\(/= sad.get_curve_data( (main)
  • this function used object rng defined in sp_app_lib, that is now defined in sp_app_util. Wasn’t sure whether to create new one in this module, or call the one in the util module, settled on latter => s/= rng/= sau.rng/
• copy current version of init_curve() to sp_app_data
  • remove sal. from calls to function in sp_app_data
  • in main module => s/= sal.init_curve/= sad.init_curve/
• quick test (generate one of each type of image, excluding animations), things seem to be working just fine
• copy current versions of proc_curve_form() and fini_curve_form() to sp_app_data
  • fini_curve_form() only called from proc_curve_form()
  • proc_curve_form() only called from get_curve_data, so remove sal. from call
• copy current versions of get_globals() and set_globals() to sp_app_data, no other changes required as only called from fini_curve_form()
• quick test, things seem to be working just fine
• move get_gnarly_dset() and get_sym_curve() from sp_app_lib to sp_app_data
  • number of calls to get_sym_curve() from get_gnarly_dset(), so moved both at same time
  • had to remove module, sal., from call to get_gnarly_dset() in get_curve_data()
• move make_mult_shapes() from sp_app_lib to sp_app_data
  • only called from within proc_curve_form(), so no other refactoring needed
• move current version of fini_image() and setup_image() from main to sp_app_plot (though I suppose they could be considered utility type functions)
  • in sp_app_plot add imports: import g_vars as g, import numpy as np
  • add import to main: import sp_app_plot as sap
  • in main refactor calls to the two functions to use the new module
    • regex: s/setup_image\(/sap.setup_image(/, ditto for the other one
• quick test, things seem to be working just fine
• move get_trfm_params() from main to sp_app_util (though guess could go in sp_app_plot)
  • in main refactor all calls to function to use new library nickname
    • s/= get_trfm_params\(/= sau.get_trfm_params(/
• move make_transforms() from main to sp_app_data
  • in main refactor all calls to function to use new library nickname
    • s/= make_transforms\(/= sad.make_transforms(/
• move mk_trfm_params() from main to sp_app_util
  • in main refactor one call in repeat route to use the new module
• move reset_globals() from main to sp_app_util
  • in main refactor one call in repeat route to use the new module
• move set_rpt_data() from main to sp_app_data
  • in main refactor one call in repeat route to use the new module

Well I think that’s all the non-route functions moved out of main. Let’s, after a break, get started on clearing out the sp_app_lib module.

Session 2)

• move bgcm_mpl, bg_qualitatives, bg_lights from sp_app_lib to sp_app_plot as only really involved with plotting
  • refactor all references in sp_app_lib and sp_app_data to use new module (s/bgcm_mpl\[/sap.bgcm_mpl[\)
• move set_clr_map() from sp_app_lib to sp_app_plot
  • add module reference for call to rng object
  • remove sal module reference for call in setup_image()
• move set_margins() from sp_app_lib to sp_app_plot
  • add module import and reference for call in sal.cycle_lw()
  • remove sal. module reference from all calls to function in sp_app_plot
• move fig_2_base64() from sp_app_lib to sp_app_util
  • remove module reference sal. from call in sp_app_plot.fini_image()
  • correct module reference for calls in main
• move cnt_btw variable from sp_app_lib to sp_app_plot
  • remove references to old module in sp_app_plot code
  • fix module references for use in ‘main’ (s/sal.cnt_btw/sap.cnt_btw)
• quick test, things seem to be working just fine
• move blotch_bg_2() from sp_app_lib to sp_app_plot
  • fix module reference for call to rng object
  • remove references to old module in sp_app_plot.set_bg()
• move get_transform_data() from sp_app_lib to sp_app_data
  • add module reference back to sp_app_lib to some calls in the function
  • fix module references in main (s/sal.get_transform_data/sad.get_transform_data/)
  • remove reference to old module in sp_app.data.make_transforms()

This is more circular/repetitive than I would have imagined. Constantly changing and rechanging module references as I move each function or variable. Calling it a day.

Session 3)

Okay back for another hour or two. Have decided to add/create a transforms related module. Right now the code related to transforms is going into all three of the new modules. That seems to me to be a fair bit confusing. So a refactor of a refactor. Adding module sp_app_transforms.py. And going to move any transforms related code/variables moved to one of the other three new modules to it.

Beginning to think I might do the same for cycling line width related code. Not sure about code for gnarly images. As some of that is used by other image types as well.

I pretty much run a quick test after every change. A slightly more thorough one every few to several changes.

• move get_trfm_params() and mk_trfm_params() from sp_app_util to sp_app_transforms
  • in main import new module, import sp_app_transforms as sat
  • in mainreset module reference for calls to these functions to use new import s/sau\.get_trfm_params/sat.get_trfm_params/, s/sau\.mk_trfm_params/sat.mk_trfm_params/
  • appear to have a bug with mosaic transform to transform image type, doesn’t make sense; if I reset app (stop/start), it seems to work if it is the first image I generate (another day)
• move get_transform_base() from sp_app_util to sp_app_transforms
  • fix module references in function, couple call to function still in sp_app_lib and call to rng needs new module reference
  • same bug mentioned above, will need to figure out if call to a specific other image type is causing the problem
• move get_transform_data() from sp_app_data to sp_app_transforms
  • fix all references in main, s/sad\.get_transform_data/sat.get_transform_data/
  • add import, fix refererences in sp_app_data. make_transforms() until it gets moved so can test changes (no errors ??)
• move make_transforms() from sp_app_data to sp_app_transforms
  • in the function remove earlier added module references, i.e. sat. (see above; this is what I meant by ‘circular’)
  • in main fix calls to use new module, s/sad\.make_transforms/sat.make_transforms/
• move affine_transformation() from sp_app_lib to sp_app_transforms
  • remove old module reference from call in sp_app_transforms.get_transform_data()
• quick test, one of each image type, things seem to be working just fine (except that bug mentioned above)

Session 4)

And again back for another hour or two.

• move btw_qds() from sp_app_lib to sp_app_transforms
  • add module to calls to rng (3 places)
  • add module back to sp_app_lib for call to incr_data() in moved function
  • in main modify module used in calls to btw_qds()
  • test failed, issue with variable cnt_btw
• move variable cnt_btw to sp_app_transforms
  • in main modify references to cnt_btw to use sp_app_transforms module variable
    s/sap\.cnt_btw/sat.cnt_btw/
  • same for sp_app_lib, also had to remove sat.cnt_btw from globals declaration in btw_n_apart()
• repeating the mosaic transform to transform image type can cause IndexError to be thrown, will leave debugging for later
  • (index 9 is out of bounds for axis 0 with size 7).
  • Well it looks like the list rys2_i is not being changed when clicking the Draw another one! button??
  • That would appear to be because when doing so, sat.cnt_btw is not getting reset to 0. So, in sp_app_plot I made sure setup_image() was setting the correct variable. Had to add suitable import. And that appears to have fixed things. Missed that bug much earlier.

Decided to add module related to cycling line width and mosaic functionality. Once again keeping the module function more focused on the type of images that in one fashion or another generate images with changing line widths. I think there are enough functions and variables involved to make it worthwhile doing so.

• create new module sp_app_cycling
• move variable cnt_btw from sp_app_transforms to sp_app_cycling where it now belongs (more circular refactoring)
  • modify all the references in all modules to use the new location (s/sat\.cnt_btw/sac.cnt_btw/)
  • that caused some serious issues with sp_app_transforms.btw_qds()
• move sp_app_transforms.btw_qds() to sp_app_cycling
  • fix module references as appropriate in all modules
  • needed to move additional variables from sp_app_lib to sp_app_cycling
    • that caused issues with sp_app_lib.incr_data(), so moved that to sp_app_cycling
    • fix references to those variables in all modules to use the correct location module
• move incr_data() to sp_app_cycling
  • fix all calls in all modules to use the correct module
• move btw_n_apart() from sp_app_lib to sp_app_cycling
  • fix call to rng to use correct module, call to incr_data to still use sp_app_lib
  • add cnt_btw to globals statement
  • remove references to
  • in main add import for new module, then fix calls to btw_n_apart to use correct module
    (s/sal.btw_n_apart/sac.btw_n_apart)

Session 5)

And, back for some more effort and possibly grief. Though not expecting much if any of the latter. But do expect lots of effort will be needed. Good thing I had a good night’s rest.

• move sp_app_lib.blotch_bg() to sp_app_plot
  • not currently used in app, so no other refactoring required, just wanted to keep it handy
• move sp_app>lib.clw_btw_make() to sp_app_cycling
  • in main change calls to use new module (only 2 locations)
• move sp_app_lib.clw_btw_plot() to sp_app_plot
  • fix module for call to rng
  • in main modify calls to use the correct module (only 2 locations)
• move sp_app_lib.cycle_lw() to sp_app_plot & as cycle_lw() uses them also move sp_app_lib.get_cycle_params(), sp_app_lib.get_marker_cycle() and sp_app_lib.get_markers() to sp_app_cycling
  • add proper module to calls to rng (1 case)
  • in main modify calls to cycle_lw() use correct module (s/sal\.cycle_lw\(/sap.cycle_lw()

Session 6)

Took some time out for a nice lunch of borscht and pumpernickel with butter. Tasty, filling and perhaps healthy. Now to continue clearing out that original, messy app module.

• move sp_app_lib.cycle_lw_btw() to sp_app_plot; wasn’t sure where to put this one as it can be called with a parameter specify no plot in order to just get the modified curve’s data. However, it does not appear to have been used in that fashion, so…
  • fix call to rng to use appropriate module
  • modify 2 calls in main to use the correct module
• simple testing says things working as expected
• moved a number of functions and a variable from sp_app_lib to sp_app_util: get_angles(), get_axes_coord(), get_curve_dtl(), get_image_dtl() and rd_45. Figured it would not be a major headache to do so. Had to temporarily leave a copy of rd_45 in sp_app_lib
  • fix module references for all moved items, the greatest number of calls (16 each) were for the functions returning curve and image data and all of those in main. So used regexes. (s/sal\.get_curve_dtl\(/sau.get_curve_dtl( & s/sal\.get_image_dtl\(/sau.get_image_dtl()
  • quick test says things appear to be working as expected
• move lists_equal() and lists_overlap() from sp_app_lib to sp_app_util
  • fix calls to use correct module, 1 each in sp_app_transforms
• move sp_app_lib.rotate_pt() to sp_app_util
  • correct module reference for one call to the function in sp_app_transforms.get_transform_data()
• move sp_app_lib.set_cycle_freq() to sp_app_util
  • correct module reference for four calls to the function in main module (replace in VSCode)
• I did not move sp_app_lib.use_again() anywhere as it is not used in the app (didn’t work properly)
• move sp_app_lib.get_plot_bnds() to sp_app_util
  • fix all calls to use function in new module
• move sp_app_lib.get_rnd_qds() to sp_app_transforms
  • fix 4 calls function in sp_app_transforms.get_trfm_params()

There are a number of module variables and six functions left in sp_app_lib. They are all related to producing animated versions of spirograph images. So, new module, sp_app_animate. Moved them all into this new module. And fixed calls to use the functions in the new module.

Have some leaf raking to do. So, a bit of a break before continuing. Think I should pull needed functions into the web app modules from the command line app modules so that the two apps are completely separate. I currently import and use modules from the command line app in the Flask app.

Done

It is the end of November 2023, ~3°C, as I work on the draft of this post. Picking up wet leaves, even with gloves on, left my hands very cold. My lungs didn’t much like breathing the cool air either. But the lawns are now pretty much clear of leaves. Can’t say the same for the flower beds or the areas behind the carport and garage. But, that will likely just have to wait until next year.

Back to the matter at hand. I am going to search the existing app modules for function calls or variable references that use one of the three command line app modules. Then see if I can move the variable or function over to one of the existing web app modules. Or perhaps a new one. Let’s see what I find.

But this post is already plenty long enough. Full of mostly useless information for anyone other than me. That said, I think I will cover this next step I am considering in a new post. Again, likely one only of interest to me.

Until next time, never be afraid to refactor your code. And, if you have to rake leaves, do try and get it done before the weather makes that more uncomfortable than one would like.