onway router media server
 

onway router media server


Introduction

The onway router HTTP media and content server provides a mechanism to serve static files over HTTP to clients on the network. In addition, it can be used to implement simple applications to browse and discover that content.

To realize applications, the content storage may provide templates to implement to look of the application with appealing content navigation. By serving additional client side logic in Javascript or CSS for styling through either templates or static files, rich client applications can be implemented in the clients browser.

Configuration

To access media and content on onway routers, the device must be explicitly configured to provide such a service. This configuration includes the interface and/or client subnets the content is accessible to, and the storage backend device with the content that is accessible. The configuration also defines the base URL with hostname and port where the media server provides its service. The base URL defined by the configuration is constructed as:

<base-url> = http://<hostname>:<port>

All parameters are configuration specific; usually the content is accessible under the anycast address 185.12.129.55, which is publicly resolvable in the DNS under media.onway.ch. For user convenience, it is advised to use the default HTTP port, hence the base-URL would be:

<base-url> = http://media.onway.ch

The content storage can be provided by different means using storage backend implementations. Currently supported is storage from removable media, such as USB sticks, or from filesystem bundles synchronized automatically from the backend.

Hierarchial content organization

The media and content server serves content that is provided under a hierarchical structure, very similar to a file system directory structure. Each directory in this structure may have sub-directories and files with content to serve.

Serving static content

Static content is delivered under the base URL of the server appended with the magic URL path content. This content is required to differentiate serving static files from serving pages rendered through templates.

Any value following the content selector in the URL identifies the hierarchical path to the file to serve. Each sub-directory is separated by a slash (/), and the final component identifies the filename. A GET request to the URL:

<base-url>/content/foo/bar/baz.tar.gz

delivers the static content that is provided by the file baz.tar.gz under the directory hierarchy foo followed by bar.

Discovery of the content under this structure must be provided by templates.

Rendering and serving templates

A storage backend can provide templates that allow the navigation and consumption of the content. If the base URL is not followed by the magic selector content, the URL is handled by rendering a template.

If the selector following the base URL matches to the name of a template, the template under that name is rendered. Templates must be organized all in the same directory, nesting templates in sub-directories is not supported. This allows the remaining part of the URL following the template name to be interpreted as a directory path for navigation purposes in the application.

If no template is found, the default template name called index is used, and everything following the base URL is handled as directory path.

To allow pretty URLs, but still serve files under appropriate MIME types, template names can be aliased. A template selector string list matches the template name list.htm, list.js or a any other file named list with an arbitrary file extension. The template is served with the appropriate MIME type indicated by the found extension.

For example, if the storage backend provides the templates index.htm and script.js, the following URLs render the corresponding templates:

<base-url>/foo/bar
<base-url>/index
<base-url>/index/foo
<base-url>/index.htm/foo/bar

<base-url>/script.js
<base-url>/script/foo

The parameters following the template selector defines the dataset passed to the template for rendering.

Template format

Templates provided by a storage backend must be in the Mustache template format. Different content types may be rendered using templates, non-verbatim Mustache variables are HTML-Escaped.

Currently not supported are implicit iterators and custom delimiters. Also, whitespace is currently not strictly handled according to the Mustache specification.

Template dataset

To render templates, the media server provides a dataset the Mustache template can access for rendering. The dataset depends on the directory path appended to the URL. This allows the application to implement site navigation based on the content directory structure. The dataset always points to a specific directory path, and provides additional information of the surrounding parent and child directories and files.

The dataset passed to the template is defined by the following JSON-schema in YAML encoding:

$schema: http://json-schema.org/draft-07/schema#
definitions:
  directoryName:
    description: Directory name, empty string if root directory
    type: string
  directoryPath:
    description: Full directory path on base URL, separated by /
    type: string
  directoryMeta:
    description: Directory metadata, if any
    type: object
  directoryInfo:
    type: object
    properties:
      name:
        $ref: '#/definitions/directoryName'
      path:
        $ref: '#/definitions/directoryPath'
      meta:
        $ref: '#/definitions/directoryMeta'
      first:
        description: Set to true if first directory in containing list
        type: boolean
      last:
        description: Set to true if last directory in containing list
        type: boolean
    required:
    - name
    - path
  contentVariant:
    type: object
    properties:
      path:
        description: Path to content under base URL and _content_ path
        type: string
      mime:
        description: MIME type of content
        type: string
      first:
        description: Set to true if first file variant in containing list
        type: boolean
      last:
        description: Set to true if last file variant in containing list
        type: boolean
    required:
    - path
    - mime
  contentItems:
    type: array
    items:
      type: object
      properties:
        name:
          description: Name of the content object
          type: string
        variants:
          description: File format variants provided for content
          type: array
          items:
            $ref: '#/definitions/contentVariant'
        first:
          description: Set to true if first file item in containing list
          type: boolean
        last:
          description: Set to true if last file item in containing list
          type: boolean
type: object
properties:
  current:
    description: Information about current target directory
    $ref: '#/definitions/directoryInfo'
  parents:
    description: Parent directories in ascending order
    type: array
    items:
      $ref: '#/definitions/directoryInfo'
  children:
    description: Child directories under current path
    type: array
    items:
      $ref: '#/definitions/directoryInfo'
  content:
    description: Files under current directory
    properties:
      video:
        description: List of video files in current directory
        $ref: '#/definitions/contentItems'
      audio:
        description: List of audio files in current directory
        $ref: '#/definitions/contentItems'
      image:
        description: List of image files in current directory
        $ref: '#/definitions/contentItems'
      other:
        description: List of unclassified files in current directory
        $ref: '#/definitions/contentItems'
required:
- current
- parents
- children
- content

The first and last helper properties can be used to assist in template rendering, for example the last property is useful when generating JSON through mustache templates, as it allows to separate items with commas properly.

Rendering a template without a specific content path implies the root path. An example of the template data may look like:

current:
  name: ""
  path: ""
  meta:
    page-title: Hello World!
parents: []
children:
- path: videos
  name: videos
- path: pictures
  name: pictures
content: []

This examples shows two subdirectories for videos and pictures, and the template can render these subcategories accordingly. When passing the videos path as GET argument path, the following example shows data that could be passed to the same template:

current:
  name: videos
  path: videos
parents:
- path: ""
  name: ""
  first: True
  last: True
  meta:
    page-title: Hello World!
children:
- path: videos/movies
  name: movies
  first: True
- path: videos/safety-instructions
  name: safety-instructions
  meta:
    title: Safety Instructions
    Description: Information about your trip safety.
  last: True
content: []

When descending further into videos/safety-instructions as path, the following example shows the content available:

current:
  name: safety-instructions
  path: videos/safety-instructions
  meta:
    title: Safety Instructions
    Description: Information about your trip safety.
parents:
- path: ""
  name: ""
  meta:
    page-title: Hello World!
  first: True
- path: videos
  name: videos
  last: True
children: []
content:
  video:
  - name: intro
    variants:
    - path: video/safety-instructions/intro.mp4
      mime: video/mp4
      first: True
    - path: video/safety-instructions/intro.webm
      mime: video/webm
      last: True
    first: True
    last: True
  image:
  - name: Logo
    variants:
    - path: video/saftey-instructions/logo.jpg
      mime: image/jpeg
      first: True
      last: True
    first: True
    last: True

Based on this information, the template can render the image and a HTML5 video player using different file formats.

Storage backends

The content and the structure used implicitly for navigating the content through rendered templates can be provided by different storage backends. Currently supported are backends for removable media, such as USB sticks, or filesystem bundles synchronized from the backend.

Removable media

Removable media devices must be formatted using NTFS. Due to the volume and file size restrictions of the FAT filesystem, its use is discouraged. The partition number used of a removable media device is configuration specific, usually it is the first partition that is considered.

Under the partition root filesystem, two directories are expected: content for the actual content files, and templates for the Mustache template files to consider for rendering.

The files provided under content are exposed over the HTTP server as static data under the base URL followed by the content path. In addition to raw content files, each directory may contain metadata in JSON format. The metadata file is called meta.json, and the file contents are provided under the template input data for the corresponding directory under the meta key. There are no restrictions to the contents of the metadata, as long as it is valid JSON. JSON files must be valid and must be encoded in UTF-8 without BOM.

The files under templates are handled as Mustache templates. The extension of the file defines the MIME type of the rendered template served to clients. Hence it is advisable to use extensions such as htm, css or js, even if the files are actually Mustache templates. When referencing templates, the file extension may be omitted to allow the use of pretty URLs. Template names can be referenced in URL using their URLified representation.

To produce pretty URLs for directory, file and template names, these names are mapped to a pretty representation. Names are not url-encoded, as this would make the URLs highly unreadable. Instead, any non-“Unreserved Character” gets replaced during this URLification to a dash, whereas subsequent dashes are combined to a single dash. Upper-case characters get converted to their lower-case variant. When naming files, these rules should be considered to ensure that files are uniquely selectable via URLs.

Filesystem bundles

Alternatively to serving files from removable media, content may be provided via filesystem bundles. Bundles contain a directory structure, and are automatically synchronized to routers.

The input to create such bundles can be provided through an OCI distribution compatible registry, such as Docker Hub, GitHub Container Registry or others. Content for the media server as specified herein must be provided as Artifact using the application/x-onway-webroot Content-Type.

The contents of such an Artifact must contain a content and/or a templates folder, as discussed in the previous section. The content and templates folders may be nested in a common parent directory.

To upload an OCI Artifact, the use of the oras tool is recommended. It creates a tarball of a directory containing the content/templates folders and uploads it to the registry. Using the Content-Type application/x-onway-webroot is required to annotate the Artifact.

oras push cr.example.com/my/webroot:v1 my-webroot:application/x-onway-webroot

          \--------------------------/ \--------/ \-------------------------/
             Repository and tag          folder           Content-Type

Development tool

To avoid the need for a physical router for template and web content development and speed up the edit/deploy/test loop, a helper tool is provided to simulate the onway media server.

The onway-media-server development tool is provided as Debian package intended for installation on the latest Ubuntu LTS release. It depends on libjansson4 and libmicrohttpd12, and can be installed using:

apt install libjansson4 libmicrohttpd12
dpkg -i onway-media-server*.deb

After installation, the tool can be spawned using onway-media-server. The --dir option takes a path containing the templates and content subdirectories with the file structure layout discussed in the previous section. The --port option can define a HTTP server port different than 8080, and additional options are documented under the --help command option.

Once the tool is running, a web browser can access http://localhost:8080 (or the appropriate custom port) to view the rendered templates as they would appear when rendered on the router.

While template files under the template directory are reloaded upon mtime modifications on these files, please note that directories and files under content may not. After modifying files under these paths, the onway-media-server must be restarted to reflect the changes. Also consider caching effects of the browser if changes are not visible immediately; Firefox disables chaching by default if the Developer Tools are open. Chrome requires checking the Disable Cache checkbox under the Networking tab in the Developer Tools to disable caching.

Limitations of VirtualBox shared folders

When running the onway-media-server in a virtual machine under VirtualBox, please be aware the limitations of the vboxsf driver (shared mounts). Changes from the host systems may not be visible through the webserver, as the driver has caching issues when working with mmap()ed files. As a work-around, try newer VirtualBox guest utilities or copy the files from the shared mount to a folder on a native Linux filesystem and run the onway-media-server test tool from there.