From f0d95cec374f1d07e3f956be7efbe6b0b09c5e71 Mon Sep 17 00:00:00 2001 From: Nick Downing Date: Wed, 31 Oct 2018 01:25:26 +1100 Subject: [PATCH] Improvements and corrections to the README.md text --- README.md | 80 ++++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 56 insertions(+), 24 deletions(-) diff --git a/README.md b/README.md index 691d607..9dfb118 100644 --- a/README.md +++ b/README.md @@ -17,16 +17,16 @@ closing tags since HTML is bracketed by `{ }` rather than by `...`. ### HTML tags in templates The plain JavaScript in the file is absolutely normal and is expected to use -CommonJS conventions, for instance require(...) is supported and so forth. +CommonJS conventions, for instance, `require(...)` is supported and so forth. -The embedded HTML uses a syntax similar to JavaScript's function syntax, +The embedded HTML uses a syntax similar to JavaScript function definitions, `function(arg1, arg2, ...) { ... }` except that instead of the word "function" any valid HTML (or other) tag name is allowed, and instead of just argument names, HTML attributes of the form `attrname` or `attrname=value` -are supported. In the `attr=value` syntax, the `value` is any valid JavaScript +are supported. In the `attrname=value` syntax, `value` is any valid JavaScript expression. No commas are required between attributes, so the parser has to automatically detect where one expression ends and another begins (similarly to how automatic semicolon insertion works in regular JavaScript). @@ -70,12 +70,43 @@ This generates the text: Hello, world ``` +If the text is multi-line, backquoted (template) strings are be used, to allow +embedded newlines. When the entire text is held in a variable, e.g. `myvar`, a +template string such as `\`${myvar}\`` should be used, to convert it into a +standalone statement consisting of only a string contant, and hence into HTML. +In the future, we may extend the parser to do this automatically for statements +that consist of only a variable name, which would have no effect in JavaScript. + +Note that ordinary single- or double-quoted string constants should be used in +preference to template strings where possible, since the constant strings will +be HTML-escaped at template compile time rather than at template run-time. + Note that in ordinary HTML, certain tags are more sensitive to whitespace than others, according to complicated rules about inline block elements and so on. This is never an issue with JavaScript templates, we can use as much indenting and other whitespace as we want, and only quoted whitespace will be output. -### Shorthand for classes and IDs +### Special tags in templates + +The HTML `script` tag is treated specially, because it contains JavaScript, +which is understood directly by the template parser. The JavaScript inside the +script tag is normalized (comments stripped, unnecessary braces and parentheses +removed, semicolons added and so on), and then converted to a string constant, +which will be copied out to the HTML file whenever the template is executed. + +Code within `script` tags is output without any indentation, which reduces the +wasted space in the HTML file quite considerably. In the future, we can add +automatic minification and/or other transformations, such as ES5 transpilation. +As well as browser-oriented transformations, other potential transformations +such as TypeScript could be applied to all code rather than just `script` tags. + +The HTML `style` tag is currently not handled specially, thus the content will +most likely be given as a backquoted (template) string, to support embedded +newlines. To defeat the HTML escaping, it can be wrapped in `_out(...)`, which +will be explained further below. In the future, we will switch to a dedicated +CSS parser, or potentially the `less` or `sass` compiler, for `style` tags. + +### HTML class and id shorthand The tag name can be followed by `#name` for as shorthand for the HTML attribute `id="name"` @@ -87,9 +118,9 @@ These must come before any ordinary attributes (before an opening parenthesis). ### Parser limitations -Certain tag or attribute names present difficulty since they contain "-" signs +Certain tag or attribute names present difficulty since they contain `-` signs or other characters invalid in JavaScript identifiers, or they may be reserved -words in JavaScript. The parser copes with this quite well (many syntax errors +words in JavaScript. The parser copes with this quite well (most syntax errors can already be re-parsed as tag or attribute names), but in difficult cases it could be necessary to quote the tag and/or attribute names. For example, `div.class-1.class-2 {}` @@ -101,7 +132,7 @@ Another slight limitation of the current parser is that it is more permissive than normal in parsing regular JavaScript code, for instance commas are not required between function arguments, because HTML templates are basically parsed as function calls until we see the opening `{` that identifies it as a -template. This can also likely be improved in a future `js_template` version. +template. This can also likely be improved in a future version of the system. ## Expression vs statement templates @@ -131,14 +162,14 @@ Running the above program will generate the output: It is also possible to use statement-level HTML templates or strings to build up an output buffer, which can be used for whatever purpose. The output buffer -must be called _out and it must be a JavaScript array of string fragments. As +must be called `_out` and must be a JavaScript array of string fragments. As each output fragment (such as an opening or closing tag or an embedded string) is generated, it will be sent to the output buffer by an `_out.push(...)` call. If there is dynamically generated text in the template, then there must be a function `_html_escape()` provided in the current scope, that escapes the text. -For example, consider a realistic template which we use on our web server: +For example, consider a realistic template which we use on a our demo server: ``` html(lang=lang) { head { @@ -191,8 +222,8 @@ for `_require()` they will be taken relative to the webserver's document root; should be parsed and any HTML constructs converted into regular JavaScript; and (4) The template importing is an asynchronous operation, so you normally use -`await _require(...)` whereas `require()` is synchronous. Because templates can -be changed and recompiled automatically while the webserver is serving pages, +`await _require(...)`, whereas `require()` is synchronous. Because templates +which have been edited are recompiled automatically by the running webserver, we don't want to stop the world, especially as templates are slow to compile. A slight difference between CommonJS exports and JavaScript template exports, @@ -207,7 +238,7 @@ Here is an example of a template which takes some text and wraps it in the standard HTML tags to create a page. Many templating systems work like this, so that you can put standard scripts, viewport commands etc, in one place. -Save this as `page.jst`, as it's the template for all pages in the system: +Save this as `page.jst`, since it's the template for all pages in the system: ``` return body_text => html { head { @@ -232,9 +263,9 @@ you can, for example, avoid HTML escaping of particular text where appropriate. ### Templating with callbacks Another way, which is more sophisticated, is to use a callback function to -generate the body. If doing this, we pass the output buffer _out into the +generate the body. If doing this, we pass the output buffer `_out` into the callback function, so that the entire HTML of the page can be generated in an -unbroken stream, without having to concatenate the partial page repeatedly: +unbroken stream, without having to concatenate the partial page repeatedly. The page wrapper template `page.jst` which takes a body callback and calls it: ``` @@ -247,6 +278,7 @@ return body_callback => html { } } ``` + Then the specific page template `index.html.jst` which provides the callback: ``` let page = await _require('/page.jst') @@ -259,7 +291,7 @@ return html( ``` Note that in the above example, the `{ }` around the callback function body is -essential (unlike in regular JavaScript), to make `p {...}` occur in statement +essential, unlike in regular JavaScript, so that `p {...}` occurs in statement rather than expression context. As an expression it would return a string which would be ignored, since the page template is expecting output to be in `_out`. @@ -286,7 +318,7 @@ return (lang, name) => html(lang=lang) { } } ``` -Suppose the above is saved as `/hello/index.html.jst` under your document root, +Suppose the above is saved as `/hello/index.html.jst` under your document root `/var/www/html`. You call `js_template(root, dirname, pathname)`, like this: ``` let template_func = js_template( @@ -326,23 +358,23 @@ you get the same object twice (it is a 2-argument function returning `String`). As well as this, the compiled templates are also cached on disk, which requires the document root to be writeable. For instance if you compile `index.html.jst` in a `hello/` subdirectory of the document root, you get a hidden file with the -same name and a `.js` subscript, here it would be `hello/.index.html.jst.js`. +same path and a `.js` subscript, here it would be `hello/.index.html.jst.js`. The main reason for the disk caching is really to fool the node.js `require()` system into using correct relative paths, if the template uses `require()` as well as `_require()`. However, it is also handy that the templates only need to -be re-evaluated and not recompiled when the webserver is stopped and restarted. +be re-evaluated and not recompiled if the webserver is stopped and restarted. Before using either a disk-cached or a memory-cached template, the modification times are checked and the template is recompiled if it is stale. Thus you can edit your website while it is live, and each page will be recompiled as needed -each time it is served. Templates deleted on disk are deleted from the cache. +each time it is served. Templates deleted on disk are also hidden in the cache. -Note: deletion of stale pages from the memory cache is not implemented yet (it -will use a long timeout such as one week), so the webserver should be restarted -occasionally. This will be fixed if it becomes an issue in practice. Also, if -the document root is not writeable, there is a simple expedient of making sure -that all the needed `.jst.js` files are up to date, so no write is attempted. +Note: deletion of rarely or never accessed pages from the memory cache is not +yet implemented (it will use a long timeout, e.g. one week), so the webserver +should be restarted occasionally. This will be fixed if it becomes an issue in +practice. Also, if the document root is not writeable, a simple expedient is to +make sure that all `.*.jst.js` files are up to date, so no write is attempted. ## Conclusions -- 2.34.1