Add blog post about refactoring, with refactoring image and render source

[ndcode_site.git] / blog / 20220128 / index.html.jst

return async env => {
  let blog_post = await _require('/_lib/blog_post.jst')
  let fa_cloud_upload_alt = await env.site.get_min_svg('/_svg/fa_cloud-upload-alt.svg')
  let fa_server = await env.site.get_min_svg('/blog/20220128/fa_server.svg')
  let icon_cross = await env.site.get_min_svg('/_svg/icon_cross.svg')
  let icon_tick = await env.site.get_min_svg('/_svg/icon_tick.svg')

  await blog_post(
    env,
    // head
    async _out => {},
    // body
    async _out => {
      p {
        'So after gaining considerable momentum to complete the refactoring project of the last 3–4 weeks, I am happy with things and can take a breather. Whilst the site still has the same basic functionality (read static content, sign up, verify email, password reset, sign in/out, change details and password), things are very different under the hood, and there are also visible UX improvements.'
      }

      p {
        'The '
        a (href="../20220114/index.html") {
          tt {'LogJSON'}
        }
        ' database has been in a useable state for about 2 weeks and since then I have been changing all the flows to store the data (such as user-account data upon sign up) in the central '
        tt {'LogJSON'}
        ' database for the site, rather than individual JSON files. But since this required rewriting almost all code, I decided the rewrite would encompass the following points:'
        ul {
          li {
            'Change data storage to use '
            tt {'LogJSON'}
            ' (the original goal of the rewrite).'
          }
          li {
            'Partition the logic into API endpoints and pages.'
          }
          li {
            'Rearrange the code into a sensible directory structure.'
          }
          li {
            'UX improvements, such as custom forms and fewer pop-ups.'
          }
        }
        'I will discuss each point in further detail next.'
      }

      h4 {
        'Change data storage to use '
        tt {'LogJSON'}
      }

      h5 {
        'The experience with '
        tt {'LogJSON'}
        ' to date'
      }

      p {
        'After some weeks of using '
        tt {'LogJSON'}
        ' for the backend storage, the experience has been very good. Compared to the traditional 2-dimensional structure of a relational database (SQL), the tree-like structure of '
        tt {'LogJSON'}
        ' proves to be quite convenient.'
      }

      p {
        'The JSON structure of the database at the moment looks like '
        pre {
        `{
  "globals": {...},
  "sessions": {...},
  "accounts": {...},
  "navigation: {...}
  "nodemailers: {...}
}
`
        }
        'where '
        tt {'globals'}
        ' stores miscellaneous information such as the site’s title and copyright message, '
        tt {'sessions'}
        ' stores information about each browser session connected to the site (including sign-in status), '
        tt {'accounts'}
        ' stores information about signed-up users, '
        tt {'navigation'}
        ' stores the navigation tree of the site (basically a mapping from directory names to page titles, in a hierarchical structure), and '
        tt {'nodemailers'}
        ' stores information about mail accounts and servers which we use for verifying users’ email addresses and similar purposes.'
      }

      p {
        'The '
        tt {'sessions'}
        ' and '
        tt {'accounts'}
        ' JSON Objects are keyed by the user’s session or account identifier respectively, and lead to a JSON Object per session or account. These latter do not have a predefined structure, as any part of the business logic is able to add a new key to the session or account as required. This is useful because normally this kind of addition would require a SQL structure change or a new table, both of which are inconvenient and time-consuming to implement.'
      }

      p {
        'Of course, it is also easy to add new top-level keys to the database for new functionality that doesn’t reside in the session or account. The general approach I have taken is that any code accessing the database and expecting a particular key should add that key if it does not exist. Ultimately we might want to have a schema for the database, and predefined objects with predefined field sets. This would also work well, but would require a bit more effort upfront.'
      }

      p {
        'From a historical perspective, my understanding was that IBM was making object-oriented databases in the 70s and 80s before Oracle popularized the relational paradigm with SQL in the late 80s and 90s. Having taught the relational algebra at undergraduate level, I appreciate there are many good things about it, and I would not want to dismiss it out of hand. But I still think that the idea of object databases deserves reconsideration, hence my experiment here.'
      }

      h5 {
        'Use of '
        tt {'LogJSON'}
        ' transactions'
      }

      p {
        'In my initial attempt at the conversion to '
        tt {'LogJSON'}
        ', I was trying to keep the number of transactions to a minimum, particularly considering that all parent objects to a changed object must be rewritten to the log every transaction.'
      }

      p {
        'So each API endpoint would, for example, invoke the code to generate a new session cookie and enter or locate a session object in the database, after having opened the transaction it would use to perform its own operation. This avoided having an extra transaction just to deal with the session cookie, but also made the logic for each API endpoint more complex. A similar issue arose when serving pages and dealing with analytics or sign in/out status for the navbar.'
      }

      p {
        'A particularly annoying thing about this ‘optimization’ was that transactions which were inherently read-only were becoming read-write in general, since they might write a new session even if the rest of the transaction was only reading. This added considerable complexity to the read-only transactions such as ‘get’ API endpoints and the navbar template.'
      }

      p {
        'Therefore, I eventually realized that this was not really an optimization, and so I changed it so that when you hit the server, it will usually run 3 or 4 smaller database transactions throughout the process, e.g. for the session cookie, the analytics, the navbar, and whatever operation is needed by the page itself. Some of these are read-only, which will improve concurrency. A cost is that the log grows slightly faster, as there are multiple write transactions too.'
      }

      h5 {
        'Potential issues with '
        tt {'LogJSON'}
        ' and solutions'
      }

      p {
        'It must be admitted that the '
        tt {'accounts'}
        ' object is not scalable as things stand, since for example when I create a management interface for users, I will want to display something like 20 users per page sorted by email, and at present you would have to extract the entire set of keys from the '
        tt {'accounts'}
        ' object simultaneously and sort them to make any sense of them.'
      }

      p {
        'Therefore, I plan to create a B-Tree index of email addresses in sorted order, which will be maintained as accounts are added and removed. This B-Tree will be encoded into JSON and stored in the database as well. I will do this before attempting to create the management interface.'
      }

      p {
        'Another scalability issue is that '
        tt {'LogJSON'}
        ' generally writes out an entire JSON Object (plus all its parent objects) when it changes. Although it does not write out the child objects that have not changed, an object with many keys will still place a strain on the system (since all keys and pointers to their corresponding child objects must be written out even if only one key or pointer has changed). I see two possible solutions to this issue:'
      }

      p {
        b {
          'Improve '
          tt {'LogJSON'}
          ' to only write out changed keys.'
        }
        ' Later on we will need to track the deltas (changed keys) of each transaction precisely in order to improve concurrency (needed in turn if the load on the webserver is high). If we write the transactions to the log in delta-format, then they will be quicker to write. But conversely, they will be slower to read.'
      }

      p {
        b {
          'Modify database clients to avoid writing large objects.'
        }
        ' We could use a hashing approach to implement this. Suppose the user’s account key was '
        tt {'jane@doe.com'}
        ', we could hash this to find a hex value of say '
        tt {'0x569a'}
        ', and then we break up the '
        tt {'accounts'}
        ' object by storing under the hash first, e.g.'
        pre {
          `{
  ...
  "accounts": {
    ...
    "56": {
      ...
      "9a": {
        ...
        "jane@doe.com": {...},
        ...
      },
      ...
    },
    ...
  },
  ...
}
`
        }
        'and this way the top 2 levels of the '
        tt {'accounts'}
        ' table would have at most 256 entries each (easy to write out each time they change) and the next level would have about 1/65536 the number of entries as the original table. This would scale to at least hundreds of thousands of accounts. An adaptive scheme might also be considered.'
      }

      p {
        'A cheap way to achieve the second goal might be simply to store the data in B-Trees (implemented via JSON Arrays of some fixed maximum size corresponding to the B-Tree block size) rather than JSON Objects directly. It would give the advantage of being able to extract data in order as well as by key. Even if we didn’t use this latter feature, the nature of the B-Tree itself provides an adaptive way of keeping the index in manageably sized chunks.'
      }

      h5 {
        'Final comments on the '
        tt {'LogJSON'}
        ' experiment'
      }

      p {
        'I also fixed several minor bugs in '
        tt {'LogJSON'}
        ' over the last few weeks and it appears to be stable at this point. I must admit I have not constructed a large-scale programmatic test (add and remove thousands of items at different hierarchical levels, perhaps concurrently, and test that everything is stored correctly throughout the process) but I will do so eventually. I’ll also make a similar test to simulate heavy load on the website and exercise the database in the process.'
      }

      p {
        'Overall, I’m satisfied with the progress of the '
        tt {'LogJSON'}
        ' experiment, though I haven’t implemented the planned B-Tree layer yet. (I have been doing some research into B-Trees in preparation and have found some MIT-licensed sample code, although it’s not perfect for my requirement).'
      }

      h4 {
        'Partition the logic into API endpoints and pages'
      }

      p {
        'Under the new scheme, API endpoints use POST requests and return machine-readable responses, whereas pages use GET requests and return human-readable markup. This helps to separate business logic from presentation logic. Whilst I’m not generally a huge fan of model/view/controller type schemes, I must admit that my previous code was a bit messy when both logics were intermixed, and it was hard to see what was happening when you returned to the code later on.'
      }

      p {
        'In the previous way, with the business logic encoded into the particular pages it was used on, the page would generally check for a POST request and do something different in such case, for instance it might save something in the database and return a thank-you page.'
      }

      p {
        'I did have a few machine-readable API endpoints for special cases such as the sign-in, and some interesting API endpoints uch as the sign-out which would return status text to be displayed in the page. But as there was no clear separation between API endpoints and pages, these endpoints simply appeared as pages that the user would not normally read directly.'
      }

      p {
        'The new API-based way is much better from the UX perspective too, as we do not have to return a new page to the user each time they interact with the site, and given that APIs return machine-readable responses it is easy to indicate the progress of an operation with ticks, crosses and the like. And we can make the API more granular and present a running status to the user during a sequence of several API calls, which is easier to implement and better for the UX.'
      }

      p {
        'Interestingly, I can still use some text-based status messages (where the API endpoint returns text to be embedded in the page, like it used to) because I have standardized all API errors to return an '
        tt {'application/problem+json'}
        ' object mostly following IETF standards, and this contains a '
        tt {'detail'}
        ' field where I can describe an error message for the user. However, in the success case, any message for the user is generated by client-side code rather than being returned from the API as in the old way.'
      }

      p {
        'The new API endpoints are very easy to use, since I have the utility class '
        tt {'Problem'}
        ' and utility function '
        tt {'api_call()'}
        ' which take care of marshalling the call across the client/server boundary. At the server side, each endpoint such as '
        tt {'/api/account/sign_in.json'}
        ' is held in a corresponding source file such as '
        tt {'/api/account/sign_in.json.jst'}
        ' and the marshalling is done by a subroutine '
        tt {'post_request()'}
        ' in '
        tt {'/_lib/post_request.jst'}
        ', which also avoids a lot of duplication compared to before. To create a new endpoint, you simply copy a similar endpoint source file to the new endpoint name.'
      }

      h4 {
        'Rearrange the code into a sensible directory structure'
      }

      p {
        'Basically under the new system, things are placed in different directories from the root according to their function, similar to how in a '
        tt {'unix'}
        ' system you have '
        tt {'/bin'}
        ', '
        tt {'/lib'}
        ', '
        tt {'/usr'}
        ' and so on, and a package will divide up its files between these directories. Whilst there is some argument about this (would it not be better to have each package keep all its files in the same place?), there are also advantages to separating things out by function.'
      }

      p {
        'So under the new scheme, API endpoints are under '
        tt {'/api'}
        ', utility code and templates are under '
        tt {'/_lib'}
        ', and pages are under the original directory structure reflecting the navigation tree of the site.'
      }

      p {
        'In the previous way, it mainly followed the navigation tree of the site, which was not very flexible, since the navigation tree does not necessarily reflect the structure we need for the logic, and also was quite messy as API, utility and template code occurred at various levels of the hierarchy among the code that generated the pages.'
      }

      p {
        'The previous theory was that if for example I wanted to add a blog functionality to some other website, I’d just copy the '
        tt {'/blog'}
        ' directory and all business logic and templates would automatically be installed as well. But I have decided that mixing everything together was too impractical, so I might have to write an installation procedure at some point if I want a component system for sites. For the time being I’ll just install things in the needed places manually, which is not really much of a hassle.'
      }

      h4 {
        'UX improvements, such as custom forms and fewer pop-ups'
      }

      h5 {
        'Old style form filling'
      }

      p {
        'In the old way we were simply using plain old HTML forms with a ‘Submit’ button, and the browser would encode them into something like JSON (I am not entirely clear how this encoding works) and submit them with a POST request.'
      }

      p {
        'I created a little in-browser simulation to demonstrate how this the old style of form worked. Please have a play with it, and then we will demonstrate some UX improvements. Note that for example purposes, you can trigger a server-side error by entering the name ‘Jane Doe’.'
      }

      div.row.mb-3 {
        div.col-md-8 {
          div.card {
            div.card-body(style="height: 400px;") {
              p {
                a#old-refresh(href='#') {'Refresh'}
              }

              div#old-main {
                p {
                  'Please enter your details to receive a greeting message!'
                }

                form#old-form {
                  div.row {
                    div.col-md-6 {
                      div.form-group {
                        label.form-label(for="old-given-names") {'Given names *'}
                        input.form-control#old-given-names(type="text" placeholder="Your given names" required maxlength=256) {}
                      }
                    }
                    div.col-md-6 {
                      div.form-group {
                        label.form-label(for="old-family-name") {'Family name *'}
                        input.form-control#old-family-name(type="text" placeholder="Your family name" required maxlength=256) {}
                      }
                    }
                  }
                }
                button.btn.btn-success#old-submit(type="button") {'Submit'}
                p.text-muted.'mt-3'.mb-0 {'* This field is required.'}
              }

              p.mb-0#old-message(hidden) {}
            }
          }
        }
        div.'col-md-4'.align-self-center {
          div.icon {
            svg(xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512") {
              circle#old-pip0(cx="64" cy="256" r="16") {}
              circle#old-pip1(cx="128" cy="256" r="16") {}
              circle#old-pip2(cx="192" cy="256" r="16") {}
              circle#old-pip3(cx="256" cy="256" r="16") {}
              circle#old-pip4(cx="320" cy="256" r="16") {}
              circle#old-pip5(cx="384" cy="256" r="16") {}
              circle#old-pip6(cx="448" cy="256" r="16") {}
            }
          }
          div.icon.'ml-3'#old-server {
            _out.push(fa_server)
          }
          pre.wrap#old-details(style="height: 250px;") {
            `details = undefined
`
          }
        }
      }

      p {
        'In the above demonstration form, we had followed one good UX practice:'
      }
      div.alert.alert-success {
        div.icon24-outer.mr-2 {
          div.icon24-inner {_out.push(icon_tick)}
        }
        'Tell the user why we need this information and what we will do with it.'
      }

      p {
        'However, we had also broken another recommended UX practice:'
      }
      div.alert.alert-danger {
        div.icon24-outer.mr-2 {
          div.icon24-inner {_out.push(icon_cross)}
        }
        'Don’t repeat the field name in the placeholder text.'
      }
      div.alert.alert-success {
        div.icon24-outer.mr-2 {
          div.icon24-inner {_out.push(icon_tick)}
        }
        'Use a real person’s name or details for example purposes, as this will speed data entry.'
      }

      p {
        'We are using the browser’s built-in validation style above, which isn’t consistent across browsers. On Chromium, if you do not fill in a required field, a tooltip appears saying ‘Please fill in this field’. I don’t think this text can be changed, and more annoyingly, only the first error is indicted. Whilst this doesn’t specifically violate any UX guidelines, we can do better:'
      }
      div.alert.alert-success {
        div.icon24-outer.mr-2 {
          div.icon24-inner {_out.push(icon_tick)}
        }
        'Use a custom validation style to highlight errors with colour and provide useful hints for the expected input.'
      }

      p {
        'Another UX issue that is demonstrated in the above example is:'
      }
      div.alert.alert-danger {
        div.icon24-outer.mr-2 {
          div.icon24-inner {_out.push(icon_cross)}
        }
        'Don’t throw away the user’s partial input if they have to refresh the page!'
      }
      div.alert.alert-success {
        div.icon24-outer.mr-2 {
          div.icon24-inner {_out.push(icon_tick)}
        }
        'Save a draft to the server during data entry, and pre-populate the form on refresh.'
      }

      p {
        'Form filling was one of the killer applications for the early web, and the old way of doing things worked well for its time. Whilst the traditional way is often like it is for a reason, and I do not agree with change for the sake of change, there are definitely improvements to be made in this case. We will look at the improved form UX next.'
      }

      h5 {
        'New style form filling'
      }

      p {
        'In the new way we use an API endpoint, or several, to receive the user’s input. This allows us to stay on the same page and provide interactive feedback to the user, e.g. if something goes wrong. We can also implement advanced features, such as the saving of drafts, etc.'
      }

      p {
        'Here is an in-browser simulation to demonstrate the improved form-filling UX. Please have a play with it, and recall that for example purposes, you can trigger a server-side error by entering the name ‘Jane Doe’.'
      }

      div.row.mb-3 {
        div.col-md-8 {
          div.card {
            div.card-body(style="height: 450px;") {
              p {
                a#new-refresh(href='#') {'Refresh'}
              }

              div#new-main {
                p {
                  'Please enter your details to receive a greeting message!'
                }

                form#new-form {
                  div.row {
                    div.col-md-6 {
                      div.form-group {
                        label.form-label(for="new-given-names") {'Given names *'}
                        input.form-control#new-given-names(type="text" placeholder="Jane" required maxlength=256) {}
                        div.invalid-feedback {'Please enter a name we can address you by.'}
                      }
                    }
                    div.col-md-6 {
                      div.form-group {
                        label.form-label(for="new-family-name") {'Family name *'}
                        input.form-control#new-family-name(type="text" placeholder="Doe" required maxlength=256) {}
                        div.invalid-feedback {'Please enter something. You can enter ‘X’ if you do not have a family name.'}
                      }
                    }
                  }
                }
                button.btn.btn-success#new-submit(type="button" disabled) {
                  div.icon24-outer.mr-2#new-icon {
                    div.icon24-inner {_out.push(fa_cloud_upload_alt)}
                  }
                  div.icon24-outer.mr-2#new-tick(hidden) {
                    div.icon24-inner {_out.push(icon_tick)}
                  }
                  div.icon24-outer.mr-2#new-cross(hidden) {
                    div.icon24-inner {_out.push(icon_cross)}
                  }
                  div.icon24-outer.mr-2#new-spinner(hidden) {
                    div.icon24-inner {
                      div.spinner-border.spinner-border-sm(role="status") {}
                    }
                  }
                  'Submit'
                }
                p.text-muted.'mt-3'.mb-0 {'* This field is required.'}
              }

              div.alert.mb-0#new-message(hidden) {}
            }
          }
        }
        div.'col-md-4'.align-self-center {
          div.icon {
            svg(xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512") {
              circle#new-pip0(cx="64" cy="256" r="16") {}
              circle#new-pip1(cx="128" cy="256" r="16") {}
              circle#new-pip2(cx="192" cy="256" r="16") {}
              circle#new-pip3(cx="256" cy="256" r="16") {}
              circle#new-pip4(cx="320" cy="256" r="16") {}
              circle#new-pip5(cx="384" cy="256" r="16") {}
              circle#new-pip6(cx="448" cy="256" r="16") {}
            }
          }
          div.icon.'ml-3'#new-server {
            _out.push(fa_server)
          }
          pre.wrap#new-details(style="height: 250px;") {
            `draft_details = undefined
details = undefined
`
          }
        }
      }

      p {
        'In this simulation we are able to see all communication with the server, including the saving of the draft which is normally silent from the user’s viewpoint. The draft is saved every 3 seconds (plus the round-trip time) whilst editing is taking place. The simulation correctly shows also that you can trick the system by refreshing the page very quickly after an edit.'
      }

      p {
        'Note that the draft can be saved '
        i {'after'}
        ' the form is submitted, which is probably redundant but harmless. It might be good to clear the form once submitted, and the corresponding draft, but I have not formed a clear policy on this. For the time being, I’m relying on stale drafts expiring at the server after a timeout of 1 day.'
      }

      p {
        'Some other good UX practices that we are adopting in the improved form are:'
      }
      div.alert.alert-success {
        div.icon24-outer.mr-2 {
          div.icon24-inner {_out.push(icon_tick)}
        }
        'Provide an icon in every button, this speeds entry and also assists non-English speaking users.'
      }
      div.alert.alert-success {
        div.icon24-outer.mr-2 {
          div.icon24-inner {_out.push(icon_tick)}
        }
        'Provide strong visual indication of completion, success or failure by means of icons and colours.'
      }
      div.alert.alert-success {
        div.icon24-outer.mr-2 {
          div.icon24-inner {_out.push(icon_tick)}
        }
        'Show a pacifier when undertaking an operation that takes time, this confirms that the last click was received and also makes the operation seem quicker.'
      }
      div.alert.alert-success {
        div.icon24-outer.mr-2 {
          div.icon24-inner {_out.push(icon_tick)}
        }
        'Disable UI elements such as buttons when it doesn’t make sense to interact with them yet.'
      }

      h5 {
        'Final comments on form filling'
      }

      p {
      }

      p {
        'Whilst the old way was sufficient, it was only that, sufficient. The new way provides a more vibrant experience for the user, while capturing the data in a more efficient way by guiding the user through the process. We are also striving to reduce user frustration that could result in them leaving the form (and lost sales/conversions in a commercial context).'
      }

      p {
        'Sadly I have not had time to make the form accessible yet, and that will probably be the subject of a future article. Apart from that, the experiment with providing a modern form-filling UX has been very successful. Although it has taken a week or so of experimentation, I consider this time well spent, as I learned something about modern UX design.'
      }

      h4 {
        'Conclusions'
      }

      p {
        'The new internals of the site are much more pleasant to work on, as things are much more organized, and the '
        tt {'LogJSON'}
        ' database allows plenty of flexibility. Thus we can quickly implement UX features such as the saving of drafts. Moreover, the new UX (implemented during the database upgrade) is much more visually pleasing and easier to use.'
      }
    },
    // scripts
    async _out => {
      //script(src="/js/utils.js") {}

      script {
        document.addEventListener(
          'DOMContentLoaded',
          () => {
            let id_old_details = document.getElementById('old-details')
            let id_old_family_name = document.getElementById('old-family-name')
            let id_old_form = document.getElementById('old-form')
            let id_old_given_names = document.getElementById('old-given-names')
            let id_old_main = document.getElementById('old-main')
            let id_old_message = document.getElementById('old-message')
            let id_old_pips = [
              document.getElementById('old-pip0'),
              document.getElementById('old-pip1'),
              document.getElementById('old-pip2'),
              document.getElementById('old-pip3'),
              document.getElementById('old-pip4'),
              document.getElementById('old-pip5'),
              document.getElementById('old-pip6')
            ]
            let id_old_refresh = document.getElementById('old-refresh')
            let id_old_server = document.getElementById('old-server')
            let id_old_submit = document.getElementById('old-submit')

            let id_new_cross = document.getElementById('new-cross')
            let id_new_details = document.getElementById('new-details')
            let id_new_family_name = document.getElementById('new-family-name')
            let id_new_form = document.getElementById('new-form')
            let id_new_given_names = document.getElementById('new-given-names')
            let id_new_icon = document.getElementById('new-icon')
            let id_new_main = document.getElementById('new-main')
            let id_new_message = document.getElementById('new-message')
            let id_new_pips = [
              document.getElementById('new-pip0'),
              document.getElementById('new-pip1'),
              document.getElementById('new-pip2'),
              document.getElementById('new-pip3'),
              document.getElementById('new-pip4'),
              document.getElementById('new-pip5'),
              document.getElementById('new-pip6')
            ]
            let id_new_refresh = document.getElementById('new-refresh')
            let id_new_server = document.getElementById('new-server')
            let id_new_spinner = document.getElementById('new-spinner')
            let id_new_submit = document.getElementById('new-submit')
            let id_new_tick = document.getElementById('new-tick')

            // old simulation
            let old_pips_to_server = async () => {
              for (let i = 0; i < 7; ++i) {
                id_old_pips[i].setAttribute('r', 32)
                await new Promise(resolve => setTimeout(resolve, 100))
                id_old_pips[i].setAttribute('r', 16)
              }
            }
            let old_pips_at_server = async success => {
              id_old_server.classList.add(
                success ? 'text-success' : 'text-danger'
              )
              await new Promise(resolve => setTimeout(resolve, 100))
              id_old_server.classList.remove(
                success ? 'text-success' : 'text-danger'
              )
            }
            let old_pips_from_server = async () => {
              for (let i = 6; i >= 0; --i) {
                id_old_pips[i].setAttribute('r', 32)
                await new Promise(resolve => setTimeout(resolve, 100))
                id_old_pips[i].setAttribute('r', 16)
              }
            }

            id_old_refresh.addEventListener(
              'click'
              async event => {
                event.preventDefault()
                id_old_main.hidden = true
                id_old_message.hidden = true
                await old_pips_to_server()
                id_old_details.textContent =
                  `details = undefined
`
                await old_pips_at_server(true)
                await old_pips_from_server()
                id_old_given_names.value = ''
                id_old_family_name.value = ''
                id_old_main.hidden = false
              }
            )
            id_old_submit.addEventListener(
              'click',
              async () => {
                if (!id_old_form.reportValidity())
                  return
                let details = {
                  given_names: id_old_given_names.value.slice(0, 256),
                  family_name: id_old_family_name.value.slice(0, 256)
                }
                await old_pips_to_server()
                id_old_details.textContent =
                  `details = ${JSON.stringify(details, null, 2)}
`
                let success =
                  details.given_names.toLowerCase() !== 'jane' ||
                    details.family_name.toLowerCase() !== 'doe'
                await old_pips_at_server(success)
                await old_pips_from_server()
                if (success) {
                  //id_old_message.classList.remove('text-danger')
                  id_old_message.textContent = `Hello, ${details.given_names} ${details.family_name}!`
                }
                else {
                  //id_old_message.classList.add('text-danger')
                  id_old_message.textContent = 'Our server has determined that you are not a real person!'
                }
                id_old_main.hidden = true
                id_old_message.hidden = false
              }
            )

            // new simulation
            let new_pips_to_server = async () => {
              for (let i = 0; i < 7; ++i) {
                id_new_pips[i].setAttribute('r', 32)
                await new Promise(resolve => setTimeout(resolve, 100))
                id_new_pips[i].setAttribute('r', 16)
              }
            }
            let new_pips_at_server = async success => {
              id_new_server.classList.add(
                success ? 'text-success' : 'text-danger'
              )
              await new Promise(resolve => setTimeout(resolve, 100))
              id_new_server.classList.remove(
                success ? 'text-success' : 'text-danger'
              )
            }
            let new_pips_from_server = async () => {
              for (let i = 6; i >= 0; --i) {
                id_new_pips[i].setAttribute('r', 32)
                await new Promise(resolve => setTimeout(resolve, 100))
                id_new_pips[i].setAttribute('r', 16)
              }
            }

            let new_details, new_draft_details
            let new_refresh_details = () => {
              id_new_details.textContent =
                `draft_details = ${JSON.stringify(new_draft_details, null, 2)}
details = ${JSON.stringify(new_details, null, 2)}
`
            }

            let new_input_semaphore = new BinarySemaphore(false)
            ;(
              async () => {
                while (true) {
                  await new_input_semaphore.acquire()
                  await new Promise(resolve => setTimeout(resolve, 3000))
                  new_input_semaphore.try_acquire()
                  draft_details =
                    id_new_given_names.value.length === 0 &&
                      id_new_family_name.value.length === 0 ?
                      undefined :
                      {
                        given_names: id_new_given_names.value.slice(0, 256),
                        family_name: id_new_family_name.value.slice(0, 256)
                      }
                  await new_pips_to_server()
                  new_draft_details = draft_details
                  new_refresh_details()
                  await new_pips_at_server(true)
                  await new_pips_from_server()
                }
              }
            )() // ignore returned promise (start thread)

            let new_edited = () => {
              new_input_semaphore.release()

              id_new_submit.disabled =
                id_new_given_names.value.length === 0 &&
                  id_new_family_name.value.length === 0
              id_new_icon.hidden = false
              id_new_tick.hidden = true
              id_new_cross.hidden = true
              id_new_spinner.hidden = true
              id_new_message.hidden = true
            }

            id_new_given_names.addEventListener('input', new_edited)
            id_new_family_name.addEventListener('input', new_edited)

            id_new_refresh.addEventListener(
              'click'
              async event => {
                event.preventDefault()
                id_new_main.hidden = true
                id_new_message.hidden = true
                await new_pips_to_server()
                await new_pips_at_server(true)
                let draft_details = new_draft_details
                await new_pips_from_server()
                id_new_given_names.value =
                  draft_details ? draft_details.given_names : ''
                id_new_family_name.value =
                  draft_details ? draft_details.family_name : ''
                id_new_submit.disabled =
                  id_new_given_names.value.length === 0 &&
                    id_new_family_name.value.length === 0
                id_new_main.hidden = false
              }
            )
            id_new_submit.addEventListener(
              'click',
              async () => {
                id_new_icon.hidden = false
                id_new_tick.hidden = true
                id_new_cross.hidden = true
                id_new_spinner.hidden = true
                // the below causes an ugly flicker, so just keep the message
                //id_new_message.hidden = true

                if (!id_new_form.checkValidity()) {
                  id_new_form.classList.add('was-validated');

                  id_new_icon.hidden = true
                  id_new_cross.hidden = false
                  return
                }
                id_new_form.classList.remove('was-validated');

                let details = {
                  given_names: id_new_given_names.value.slice(0, 256),
                  family_name: id_new_family_name.value.slice(0, 256)
                }

                id_new_icon.hidden = true
                id_new_spinner.hidden = false
                await new_pips_to_server()
                new_details = details
                new_refresh_details()
                let success =
                  details.given_names.toLowerCase() !== 'jane' ||
                    details.family_name.toLowerCase() !== 'doe'
                await new_pips_at_server(success)
                await new_pips_from_server()

                id_new_spinner.hidden = true
                if (success) {
                  id_new_tick.hidden = false

                  id_new_message.classList.add('alert-success')
                  id_new_message.classList.remove('alert-danger')
                  id_new_message.textContent = `Hello, ${details.given_names} ${details.family_name}!`
                }
                else {
                  id_new_cross.hidden = false

                  id_new_message.classList.remove('alert-success')
                  id_new_message.classList.add('alert-danger')
                  id_new_message.textContent = 'Our server has determined that you are not a real person!'
                }
                id_new_message.hidden = false
              }
            )
          }
        )
      }

      style {
        .wrap {
          white-space: pre-wrap;
          word-wrap: break-word;
        }

        .icon {
          display: inline-block;
          vertical-align: center;
          width: 128px;
          height: 128px;
          fill: currentColor;
          stroke: currentColor;
        }
      }
    }
  )
}