Changelog

Version history of Helmut4 Scout.

Changelog — Helmut4 Scout

All notable changes to this project are documented here.

v4.6 — Audit History (Initial Release) & Stream Workshop Patch Annotations (current)

New Audit History page for tracking changes between import runs; Stream Workshop now stamps patched nodes with a subtitle annotation; Dynamic Reference severity badge fixed.

Audit History (/audit) — initial release

  • New Audit History page records every import run — entity counts, issue totals per severity, and a delta comparison against the previous run
  • Delta chips at the top of each run detail show count changes for every tracked category (red = increase, green = decrease)
  • Issue changes shown as the hero section: new issues and resolved issues listed with severity badges, issue type, reference key, and stream context
  • Per-category change tracking for streams, variables, metadata, metadata sets/groups, profiles, tasks, watchfolders, and cron — expandable sections for added, removed, and modified entries
  • Modified entries display only the fields that changed in a compact Was → Now table — no full side-by-side dumps
  • All diff sections start collapsed; expand on demand
  • Initial release — cross-run comparison (time machine) and additional tooling planned for a future version

Stream Workshop — patch annotations

  • When a node is upgraded to a newer class, Node has been patched by Helmut4 Scout to match version X.Y is appended to the node's subtitle field in the output exchange key — version matches the selected reference version
  • When old %wildcard% syntax is converted to {wildcard}, Wildcard has been patched by Helmut4 Scout to match new pattern is appended to the subtitle
  • Both annotations are written when a node is both upgraded and has wildcards converted

Bug fix

  • Fixed: Dynamic Reference severity badge rendered unstyled

v4.5 — Stream Surgeon Layout Engine, Preview Polish & License Info API

Node Conductor now offers Stream Surgeon as an additional built-in alignment algorithm beside Spaghetti Solver; preview rendering supports both layout styles; pre-release badges moved from Beta to Gamma.

Node Conductor

  • Added stream_surgeon as a second built-in alignment algorithm next to the existing spaghetti_solver
  • spaghetti_solver keeps the general graph-flow layout, while stream_surgeon applies a more style-aware Helmut4 layout profile
  • Adopted Node Conductor preview rendering so both alignment options are displayed with matching card width, height, and collision spacing
  • Improved Stream Surgeon preview spacing to avoid overlapping rendered nodes, including on larger stream exports

Release stage badges

  • Changed the Stream Workshop and Node Conductor pre-release badges from Beta to Gamma
  • Replaced the old purple badge treatment with a teal Gamma badge to signal the final pre-release stage before stable release

License API

  • Added /settings/license-info as a JSON endpoint for reading the current license state from the frontend and API clients

v4.4 — Node Conductor & API Endpoints

New Node Conductor documentation and API support; stream tooling now returns JSON from dedicated API endpoints; manual and OpenAPI updated to reflect the current workflow.

Node Conductor

  • Added Node Conductor as a first-class stream layout preview and alignment tool for working with Helmut4 stream exchange keys
  • Supports previewing the current layout and generating an aligned exchange key via the built-in spaghetti_solver engine or the built-in style-aware stream_surgeon engine
  • Added dedicated manual documentation describing the purpose of Node Conductor, the alignment workflow, and the current default algorithm

API endpoints

  • Added machine-facing JSON endpoints for Stream Workshop and Node Conductor: POST /api/stream-workshop and POST /api/node-conductor
  • Both endpoints now require a submitted unlock_key and return JSON responses instead of rendering the HTML pages
  • Stream Workshop API responses can return an upgraded exchange key; Node Conductor API responses can return an aligned exchange key and the selected alignment algorithm

Routing and docs cleanup

  • Removed the legacy /layout-preview redirect route — /node-conductor is now the only public HTML route for the tool
  • OpenAPI documentation updated to describe the new JSON endpoints, license requirement, and Node Conductor alignment algorithm input
  • The sidebar/manual references were updated to point at the current Node Conductor route and workflow only

License verification

  • Protected stream-tool POST actions now verify the submitted unlock key before running
  • The front end reuses the stored license key so the protected tools stay in sync with the Settings page

v4.3 — Backup Selection, Schedule for All Sources & Status Improvements

Directory-scanner backup picker; Import Schedule available for both live and backup; uncompressed mongodump directory support; WARNING lines surfaced in workflow status.

Settings — Backup selection

  • The Backup Archive path text input has been replaced with a directory-scanner + dropdown: enter the parent directory, click ↻ to scan, and select from a sorted list of eligible files and directories
  • Entries are ordered by last-modified date, newest first; the modification date is shown alongside each entry
  • Uncompressed mongodump directories are detected automatically and labelled [DB]; generic directories are labelled [DIR]; archive files show their format

Backup formats — uncompressed directory

  • Added support for uncompressed mongodump output directories alongside .zip, .tar.gz, and .archive.gz archives
  • Detection: a directory containing sub-directories with .bson files is treated as a mongodump backup and used directly — no extraction step

Import Schedule — all sources

  • Removed the live-only restriction; the Import Schedule panel is now visible and functional regardless of the selected import source
  • Backup mode auto-select — scheduled backup runs automatically pick the most recently modified file or directory from the backup scan directory; no manual selection required at run time
  • Safety: import source change disables schedule — switching between live and backup immediately unchecks the schedule toggle in the UI and forces sync_enabled = false on save, preventing cross-source execution without explicit re-activation

Workflow Status — WARNING lines

  • WARNING: lines from the build output are appended to the Message field after a successful run and displayed persistently
  • Flags optional collections absent from the backup without aborting the import (affected data is simply omitted)

Bug fixes

  • Fixed: /settings/backup-list returned HTML on unexpected OS errors, causing a JSON parse failure in the UI — endpoint now guarantees valid JSON in all error conditions
  • Fixed: /settings/backup-list and /api/cron-human were blocked by the no-database redirect guard (returning an HTML redirect instead of JSON) — both endpoints are now correctly excluded from the guard
  • Fixed: build script watchfolder collection used the wrong database prefix (mio_io → mcp_io)

Manual

  • Reworked in-app manual: restructured as collapsible <details> sections — summarised view always visible, detail on demand. All content updated to reflect current features

v4.2 — Import Schedule & Settings UX

Automatic scheduled import from live Helmut4 systems

Import Schedule

  • New Import Schedule panel in Settings — right column below Compatibility Checkup, visible only when Live Helmut4 System is selected
  • Enable/disable toggle (default off) — each run purges and rebuilds the database identically to Run workflow. Hover tooltip on the inline warning communicates this clearly
  • Cron expression — standard 5-field format, interpreted in the server's local timezone. Live human-readable translation updates as you type. A ? badge links to crontab.guru
  • Last execution — panel header shows the date and time of the most recent scheduled or test run, persisted across restarts
  • Source-switching: switching to MongoDB backup disables and unchecks the schedule; switching back to live re-enables controls but leaves the toggle unchecked (manual re-activation required)
  • APScheduler runs jobs in a background daemon thread; the Workflow Status console and progress bar update via a new 15-second background check that detects cron-triggered runs while the Settings page is open

Debug — Test sync now

  • Hidden Test sync now button — triggers an immediate live import for configuration validation
  • After clicking, the status console starts polling immediately so output appears in real time

Settings page — connection guidance

  • Helmut4 server — inline info block after the URL field: HTTP and HTTPS supported, container networking and firewall/proxy requirements noted
  • Bearer token — inline info block after the token field: only Bearer tokens accepted; paste without the Bearer prefix

API

  • POST /settings/sync-now — triggers an immediate live import (license required; 409 if already running)
  • GET /api/cron-human?expr=… — returns a plain-language translation of a cron expression, used by the settings UI for live feedback

v4.1 — Resolver, Node Catalog & Sort Options

New Issue Resolver diagnostic page; Nodes catalog refactored as grouped view; sort controls added to all catalog pages; pymongo added for MongoDB backup imports.

Issue Resolver (/issues/resolver)

  • New sub-page under Issues — accessible via a collapsible sidebar item with a ▶ expand indicator on the Issues link
  • Case mismatch detection — finds variables and metadata keys that appear in a Missing Definition error AND a Defined-But-Not-Used info item case-insensitively but not identically (e.g. My-Storage-ID vs My-storage-ID). Suggests renaming the definition to match stream usage. Covers store variables, Helmut variables, encrypted variables, and metadata
  • Compatibility issues — lists all Severity: Compatibility findings with node name (resolved from UUID), stream deep-link, and explanation
  • Deprecated issues — lists all Severity: Deprecated findings with stream deep-link and recommendation
  • Temp variable orphans — Read/Write Pairs, Read Without Write, Write Without Read. Variable names link to Issues; stream names deep-link to the node within the stream
  • Stream links handle both full format Name [EVENT/HOST] (id) and short format Name (id), and correctly link multiple streams in a single field
  • Export to Markdown — downloads a structured resolver-report.md covering all sections

Nodes catalog — grouped view

  • Grouped by node_type + node_class — one row per unique definition; V5 and V6 remain separate
  • Main row: unique node names, node definition, events, hosts, stream count, deprecated badge
  • Expanded detail: node class + a mini-table of every stream using the node — stream link, node title, event, host, and ↗ View in stream deep-link to the exact node row

Sort options — all catalog pages

  • Variables, Metadata, Metadata Set/Group, Profiles, Watchfolders, Tasks, Cron, Nodes, and Wildcards all now have sort-by, direction, and rows-per-page controls with full pagination
  • Sort preferences persisted in localStorage per page

Bug fix — MongoDB backup import

  • Previously missing dependecy caused No module named 'bson' errors when importing MongoDB backups. Rebuild the Docker image to apply.

v4.0 — Sticky Navigation, Profiles & Issues UX

Dynamic sticky header system across all table views; metadata set names linked in Profiles; issues table panel header stays in view while scrolling.

Sticky header system

  • A JavaScript-driven CSS custom property system (--page-head-height, --issue-tabs-height, --filter-height, --issue-panel-head-height) is calculated at runtime and updated on resize, giving every sticky element precise offset awareness without hardcoded pixel values
  • Filter form — sticks below the page title (and below the severity tabs on the Issues page) so it stays accessible while scrolling through long tables
  • Table headers — stick below the filter form so column labels remain visible regardless of scroll position
  • Issues page — severity filter tabs stick below the page title; the "Reference Issues" panel header sticks below the tabs and filter; table headers account for all three layers — no more overlapping elements when scrolling

Profiles — metadata set name

  • The Metadata set column in the main table now shows the metadata set name (linked to its detail page) instead of a raw hex ID
  • The expanded detail row shows the Metadata set name as the first metadata entry (linked), followed by the ID and tags — making it immediately clear which set a profile references

Issues — table UX

  • The issues-table CSS class enables the sticky panel header: the "Reference Issues" heading stays visible while scrolling through a long issue list
  • Table header row correctly offsets for the panel title height, preventing overlap with the "Reference Issues" heading

v3.9 — Watchfolders, Tasks & Cron; Stream Workshop Wildcard Patch

Watchfolder, Task, and Cron reference pages with cross-linked data; Stream Workshop now patches streams that contain only old wildcard syntax; cron expressions rendered in human-readable form.

Stream Workshop — old wildcard patch

  • The Patch nodes callout now appears whenever the stream has old wildcard syntax (%wildcard%), even if no deprecated or unsupported nodes are present
  • Callout wording adapts: wildcard-only streams show a dedicated explanation; mixed streams show both reasons combined

Bug fix — duplicate "Incomplete Reference Syntax" issue

  • Streams using %node.result.uuid% were generating two issues — the correct Old Wildcard finding and a false-positive Deprecated / Incomplete Reference Syntax. The malformed-reference detector now protects any %word% span, preventing the duplicate

Watchfolder data import

  • Watchfolder definitions fetched from GET /v1/io/watchfolder (live) and mio_io/watchfolderDBO.bson (backup), stored in a new watchfolder_reference table: ID, name, path, Profile ID, group, enabled, recursive, interval, growing-file check tries, use-cron flag
  • Active Directory group mappings parsed from ACTIVE_DIRECTORY type preferences (GET /v1/preferences), stored in a new active_directory_reference table: UUID, AD Group (LDAP DN), Helmut Groups, Products, Access Preset, Role, Actions

New reference pages

  • Watchfolders (/watchfolders) — name, path, enabled badge, recursive, interval; expanded detail: ID, profile name (linked), profile ID, group, growing-file check tries, use-cron, and any cron schedules that reference this watchfolder (linked)
  • Tasks (/tasks) — task name, profile, profile type, filter mode, creator; expanded detail: task ID, profile name (linked), profile ID, created/modified, filters
  • Cron (/cron) — name, type, expression, resolved target, enabled badge; expanded detail includes:
    • Schedule — human-readable translation of the cron expression; seconds shown as :SS only when non-zero
    • Expression breakdown — visual field-by-field grid labelled Second / Minute / Hour / Day of month / Month / Day of week, making 6-field Quartz expressions immediately readable
    • Target resolution by type: WATCHFOLDER → watchfolder name (linked); ACTIVE_DIRECTORY → CN extracted from LDAP DN, full AD config in detail; other types show target/task/profile name

Cross-references

  • Cron → Watchfolder: target column and detail row link to the Watchfolder page
  • Watchfolder → Cron: expanded detail lists all cron schedules that trigger this watchfolder (linked)
  • Tasks and Watchfolders → Profiles: Profile name shown and linked to the Profiles page
  • All three pages use the same expandable-row design as Streams, Variables, Metadata, and Profiles

v3.8 — Stream Workshop: Patch Engine & Reference Integrity

Improved upgrade/downgrade engine with automatic node.result reference rewriting, blocked-key protection for unresolvable nodes, and interactive donut charts.

Node patch engine

  • The patch action now works correctly for both upgrades (e.g. V5 → V6) and downgrades (e.g. V6 → V5) — wording updated throughout: "Upgrade nodes" → Patch nodes, "upgraded" → "patched"
  • When a node is upgraded and receives a new UUID, all {node.result.<old_uuid>} references in every other node's parameters are automatically rewritten to use the new UUID — previously the output stream was broken for any downstream node consuming that result
  • Rewriting also handles the indexed/named variant: {node.result.<uuid>_0}, {node.result.<uuid>_test} — the suffix is preserved while only the UUID portion is replaced
  • Blocked output: if any node has no replacement in the target version, no exchange key is generated — a "Blocked" notice replaces the output field. Previously a broken/incomplete stream key was produced that would fail on import

Migration report

  • Nodes whose node.result references were rewritten now show a Ref updated badge in the Migration Report status column
  • The Upgrade Results legend shows a "Reference updated" row with the count and a descriptive note

Compatibility callout

  • The patch callout now includes the full version string: "1 node can be automatically patched to Helmut4 version 4.13.0"
  • The Alpha badge on the patch button and output panel replaced with Beta

Interactive donut charts

  • Both the Compatibility Score and Upgrade Results donut charts are now hoverable: hovering a segment dims the others, updates the center text to show the segment's count and percentage, and restores on mouse-leave — matching the dashboard chart behaviour
  • Fixed: secondary buttons (Paste from clipboard, Copy to clipboard) showed an unreadable dark-blue background on hover — now correctly renders a light grey hover state

v3.7 — Catalog Views & Data Refinement

Every reference catalog page rebuilt with a consistent expandable-row design, humanised type names, grouped wildcard entries, and cross-catalog search that auto-expands matching rows.

Unified catalog page design

  • All catalog pages (Streams, Nodes, Variables, Metadata, Metadata Set/Group, Profiles, Wildcards, Issues) now use a consistent expandable-row design: a ▶ toggle per row reveals secondary detail in a structured two-column grid
  • Search results auto-expand the detail row so matched fields are immediately visible
  • Stream and metadata set links preserved everywhere — no raw IDs in place of linked names
  • The global search bar moved into each page's header section, right-aligned alongside the page title; not shown on Stream Workshop, Settings, or Manual

Variables

  • Main: Variable type (humanised), Variable name, Status, Defined, Value/expression, Readers, Setters — "Usage count" renamed to Readers to pair symmetrically with Setters
  • Expanded: Direction, Dynamic, Definition source, Description, Depends on, Used/Set by streams (linked)

Metadata

  • Main: Metadata type (humanised), Metadata key, Field type, Allowed values/expression, Assigned to sets/groups (linked), Readers, Setters
  • Expanded: Status, Defined, Dynamic, Assignment count, Mandatory, Readonly, Preresolve, Filterable, Definition source, Assignment tags, Parent metadata sets (linked), Used/Set by streams (linked)

Metadata Set / Group

  • Main: Name (linked), Tags, Type (IO → Metadata Set · FX → Group), Metadata count
  • Expanded: ID (linked), Assignment category, Additional stream metadata

Streams

  • Main: Stream (linked), Event, Host, Product, Nodes, Readers (was Reference count), Setters, Errors (red when non-zero), Info, Complexity score, Disabled
  • Expanded: Tags, Warning count, Deprecated nodes, all reference-type and node-type counts, HTTP/Command/JavaScript complexity sub-scores

Nodes

  • Main: Node name, Node definition (type), Stream (linked), Event, Host, Disabled, Deprecated (badge)
  • Expanded: Stream ID (linked), Stream deprecated/tags, Created/Modified dates, Reference count, Summary, References, Old wildcards (highlighted)

Wildcards

  • Grouped by wildcard token — previously one row per stream usage; now one row per unique wildcard with a stream count. Old %percent% syntax flagged with an Old syntax badge
  • Expanded: Category, Syntax, Supported events/hosts, Description, mini-table of all streams using the wildcard (linked, sorted alphabetically)

Profiles

  • Main: Profile name, Profile type, Metadata set, Pre-stream (linked), Main stream (linked)
  • Expanded: IDs (linked), Priority, Hidden, Node filters, stream event/host/product/tags for both pre and main stream

Issues

  • Main: checkbox, ▶ toggle, Severity badge, Issue type, Reference type (humanised), Reference key (linked)
  • Expanded: Stream (linked), Explanation, Recommendation — all acknowledge functionality preserved

Type name humanisation

  • job_metadata → Job Metadata, store_variable → Store Variable, helmut_variable → Helmut Variable, encrypted_variable → Encrypted Variable, stream_variable → Temp Stream Variable — applied consistently across all catalog pages

Bug fixes

  • Checkboxes and radio buttons excluded from the global -webkit-appearance:none CSS rule — checkboxes were invisible after earlier appearance normalisation

v3.6 — Stream Detail & Reference Intelligence

Deep linking between issues, references, and nodes on the stream detail page; Old Wildcard as a first-class severity; sort preference persistence.

Stream detail page

  • Issues panel and References panel converted to collapsible <details> sections with Show/Hide toggle buttons — both open by default, matching the Nodes panel behaviour
  • Issue detail rows (explanation and recommendation) rendered as native <details>/<summary> triangle elements, matching the parameter actions style used in the Stream Workshop migration report — collapsed by default, click ▶ Details to expand
  • Issues table — "Reference" column renamed "Node"; node-type issues now show #N Node Title as a clickable anchor link that scrolls directly to the matching row in the Nodes table
  • Nodes table — new "Issues" column showing severity badges for any issues linked to that node, giving an at-a-glance signal without cross-referencing manually
  • References table — new "Node" column with #N Node Title anchor links; design-source references now resolve the node UUID from the operator key in the walk path, eliminating blank node cells for parameter-level references
  • Duplicate "Stream" source reference rows eliminated — $.design and $.nodeList paths are now excluded from the stream-level walk since both are covered by the design-level walk with full node UUID tracking

Compatibility metric box

  • A 7th metric box on the stream detail page shows the compatibility score against the highest available reference version: percentage (green / amber / red), a breakdown line (ok · dep · uns · wc), and the version checked
  • Clicking the box opens Stream Workshop with the stream pre-loaded and the analysis already run — no manual pasting required
  • GET /verify?from_stream=<id> new query parameter: loads the exchange key from the database and auto-runs the compatibility analysis; used by the metric box link and documented in the OpenAPI spec

Old Wildcard severity

  • "Old Wildcard" is now a dedicated severity level, sorted between Warning and Orphan in all views — previously old wildcards were grouped under "Deprecated"
  • Detection moved from the wildcard catalog (per-stream, no node link) to per-node parameter scanning in design_json.operators, matching the logic in Stream Workshop — issues are now stored with Reference type: "node" and a real UUID so they link directly to the affected node in the stream detail page
  • _verify_exchange (Stream Workshop) now scans both parameter and globalParameter sections for old wildcard syntax, including ARRAY_STRING list values — detection now matches the upgrade conversion logic exactly

Sort preference persistence

  • Sort order (order_by, sort_dir) and rows-per-page (per_page) are now saved to localStorage per page when the user changes them
  • On next visit, the stored preference is restored automatically — only applies when a sortable table is present and no explicit sort params are in the URL

v3.5 — UI Polish & API Coverage

Global style improvements, sidebar refinements, and complete OpenAPI coverage for Stream Workshop.

UI / Style

  • Added padding-bottom: 48px to the main content area — content no longer clips against the browser bottom edge on any page, including Stream Workshop with long results
  • Primary button now has a hover state (darkens to --brand-dark) — previously no visual feedback on hover
  • Focus rings added for all input, select, and textarea elements (2px brand-colour outline) and button elements (focus-visible) — improves keyboard navigation and accessibility
  • page-head bottom padding increased from 8px to 16px for a more comfortable gap between the page title and the first panel
  • Navigation sidebar now sets overflow-y: auto so the nav list scrolls rather than clips when items exceed the sidebar height
  • App credit block redesigned: top border separator, MoovIT · Vienna · 2026 on one line, version displayed as a styled pill link centred below it
  • Sidebar collapse/expand toggle button moved to the left edge (was right-aligned)

Stream Workshop

  • Both the Analyse and Upgrade actions require an active license — the form shows a warning with a direct link to Settings when no license is active
  • The Verify stream and Upgrade nodes buttons are disabled in the UI when no license is active

OpenAPI specification

  • Added a dedicated Stream Workshop tag with a full description of the standalone analysis and upgrade capability
  • GET /verify documented as the Stream Workshop page entry point
  • POST /verify fully documented: both inspect and upgrade actions, all parameters (exchange_json, compatibility_version, action), wildcard conversion behaviour, parameter migration, and the returned exchange key
  • License requirement explicitly noted in the POST /verify description and the global info.description
  • /graph description updated to reflect the v3.4 search improvements (per-node focus, match navigation, keyboard shortcuts)

v3.4 — Graph Search

Improved graph search with per-node focus and match navigation.

  • Graph search now zooms into the matched node at a consistent 1.5× scale instead of fitting all results into view — making it usable on large graphs with many nodes
  • All matching nodes are highlighted with a yellow border so their positions are visible across the full graph
  • When a search returns multiple matches, ◀ / ▶ navigation buttons appear to step through each result one at a time; the counter shows the current position (e.g. 3 / 7)
  • Keyboard shortcuts: Arrow Down / Arrow Up step through matches, Escape clears the search
  • A ✕ clear button removes highlights, deselects nodes, and resets the search field
  • "No match" is shown inline next to the input instead of an alert dialog

v3.3 — Stream Workshop

Unified stream analysis and node upgrade in a single tool.

  • The standalone Stream Updater page has been removed; its node upgrade functionality is fully merged into the former Stream Inspector, which is now renamed Stream Workshop
  • Compatibility analysis and node upgrading now share the same exchange key input, reference version selector, and results area — no need to switch between two separate pages
  • A separator line above Stream Workshop in the sidebar navigation visually groups it apart from the database-driven stream list

Compatibility analysis

  • All previous Stream Inspector functionality carried over unchanged: node-by-node compatibility report with deprecated nodes, version mismatches, unknown node classes, enum option issues, wildcard syntax issues, trigger validation, variable and metadata reference extraction, and the SVG donut compatibility score panel

Node upgrade (Alpha)

  • Automatically replaces deprecated or unsupported nodes with their best replacement for the selected Helmut4 reference version
  • Parameter values are migrated by variable name; operator keys, canvas positions, and stream transition links are preserved
  • Old %variable.name% wildcard syntax is converted to the current {variable.name} form across all node parameters, including ARRAY_STRING parameters where the value is a JSON list of strings
  • A Migration Report lists every affected operator with its old and new node class, old and new UUID, and a full log of parameter actions taken during the upgrade
  • Dual status badges: nodes that were both upgraded and had wildcard syntax patched now display both an Upgraded badge and a Wildcard patched badge simultaneously — previously only the Upgraded badge was shown
  • The upgrade summary legend includes a note when one or more upgraded nodes also had wildcards converted (e.g. + 2 upgraded nodes also had wildcards patched)
  • Bug fix: wildcard conversion actions for upgraded nodes were previously overwritten by parameter migration actions in the report — both action lists are now correctly combined and preserved
  • The resulting exchange key is presented in the Updated Exchange Key panel and can be copied directly to the clipboard for pasting back into the Helmut4 stream designer
  • Alpha badges are shown on the Updated Exchange Key heading and the Copy to clipboard button to signal that the upgrade feature is in early access

v3.2 — Stream Workshop

Standalone stream verification — no database import required.

  • New Stream Workshop tool accessible directly from the sidebar (positioned near Streams)
  • Paste a Helmut4 stream exchange key (plain JSON or base64-encoded) and select a reference version to run a full compatibility analysis without importing the stream into the database
  • Automatic format detection: accepts raw design JSON or the base64-encoded exchange key produced by the "Copy Stream Exchange to Clipboard" button
  • Extracts stream identity from the startNode: name, event, host, and product (FX / IO / CO / HK) — product resolved from bundled reference files where available
  • Compatibility checks run against the selected Helmut4 reference version:
    • Deprecated nodes flagged per node
    • Node version mismatches with directional V-number messaging (node too new or too old for target)
    • Unknown node classes (plugin nodes or future-version nodes)
    • Design enum values not present in the target version (removed or renamed options)
    • Design enum options missing from the reference (outdated option list)
  • Unknown stream trigger detection: if the stream event does not appear in any product catalog (FX / IO / CO / HK) for the selected version, an Error-severity issue is raised and the compatibility score is set to 0% / Blocked
  • Variable and metadata reference extraction: all {placeholder} patterns in the design are classified (store variable, Helmut variable, metadata, node result, wildcard, etc.) and listed in a deduplicated reference table
  • Compatibility score panel shown alongside the input — SVG donut chart with OK / Deprecated / Unsupported breakdown; Blocked state renders a full red ring with 0% when a trigger error is detected
  • Product event files (FX.json, IO.json, CO.json, HK.json) added to the node reference catalog format; fetch_helmut4_stream_package_reference.sh now downloads them alongside the node/wildcard catalogs
  • Paste-from-clipboard icon button on the exchange key input for quick entry

v3.1 — Stream Configuration Compatibility

Precision version-mismatch detection and enhanced enum compatibility checks.

  • Compatibility check now always pre-selects the highest available reference version on page load — version choice is no longer persisted between sessions
  • Node version mismatch detection extracts the V suffix from both the stream node class and the reference catalog to produce specific, directional messages:
    • Node too new — uses ActionNodeV5 but target version only provides up to V3: flagged with the maximum available version and a recommendation to replace or verify
    • Node too old — uses ActionNodeV1 but target version starts at V2: flagged with the minimum available version and an upgrade recommendation
    • No related entry — class not found and no same-base candidates: flagged as Unknown Node Class (plugin or future version)
  • New compatibility check: design enum values that do not exist in the selected reference version are flagged as Compatibility: Design Uses Enum Values Not In Reference — catches option values removed or renamed between Helmut4 versions
  • Existing Design Enum Options Missing check (reference has values absent from design) continues unchanged alongside the new check

v3.0 — Helmut4 Scout

Major code refactoring, PWA support, and UX polish.

New

  • Progressive Web App (PWA) support — installable directly from the browser on desktop and mobile
  • App icons generated from brand assets (icon-192.png, icon-512.png, favicon.ico, apple-touch-icon.png)
  • Service worker with network-first caching strategy for static assets
  • Web app manifest (manifest.json) with theme colour, display mode, and orientation settings
  • Import form conditional field visibility — live/backup-specific fields shown or hidden based on selected source
  • Workflow progress bar in Import settings while a run is active
  • Dynamic Layout toggle replaces raw "physics" label in the graph
  • Graph light/dark theme toggle with full node label colour correction across all groups
  • ZIP archive support as a third backup import format
  • Native MongoDB 8.x archive format (.archive.gz) parsed directly in Python without external tools
  • Severity Distribution donut chart on Dashboard — acknowledged issues shown as faded arc segments with hover interactivity
  • Snapshot comparison — delta column (vs. previous import) in the severity distribution table
  • Stream node summary displays success/failure target node titles (#index Name) instead of raw UUIDs
  • Scroll offset fix for sticky header when navigating to a node anchor
  • License Activation renamed from Settings Unlock — "Unlicensed" / "Licensed" badge in panel head
  • Drop database action no longer requires a license key

Improved

  • Backup import auto-detects format (tar.gz → zip → MongoDB archive) without configuration
  • Backup import node properties fall back to direct BSON node fields — fixes missing type, title, description, and summary for MongoDB 3.x/8.x backups
  • Design metadata (title, description) correctly parsed from backup imports — dict format from BSON was previously silently skipped
  • purge_output_dir and cleanup_output_artifacts both preserve license key, hardware ID, and snapshot files
  • save_issue_snapshot called with proper Flask app context in background thread
  • /streams/{id}/exchange endpoint returns the stream design object only, not the full exchange JSON
  • Settings import source dropdown renamed: "Live Helmut4 System" / "MongoDB v3.4 / 8.x backup"
  • Code cleanup: removed output_has_data(), stream_id_from_row(), unlockPanel JS variable, .log-output.empty CSS rule, pymongo from requirements

v2.9 — Branding & Production Hardening

  • Official rename and rebranding from Reference Viewer to Helmut4 Scout
  • New logo assets: H4Scout.png, Helmut4Scout_small.png, Helmut4Scout_collapsed.png, H4Scout_square.png
  • Updated sidebar, page titles, and all documentation to reflect new name
  • Switched from Flask development server to Gunicorn WSGI server for production deployments
  • Removed key generator from the Docker image — generate_scout_unlock_key.sh kept as a separate deployment utility
  • about.md and CHANGELOG.html introduced as project documentation

v2.8 — Analytics, Reverse Engineering & Clipboard Integration

  • Deep node property analysis: cmd, filePath, variableName, storeVariableKey, metadataName, successNodeUUID, failedNodeUUID captured and displayed in node summary
  • Node transition rendering: success/failure targets linked directly in stream node detail
  • Design metadata extraction from stream exchange JSON: per-node title, description, and label from Helmut4 operator properties
  • Stream complexity scoring: node count, reference count, deprecated node count, dynamic reference count, HTTP/command/JavaScript complexity sub-scores
  • Highest Complexity Streams table on Dashboard
  • Issue acknowledgement system — acknowledge individually, by severity, or in bulk; acknowledged issues hidden by default with toggle
  • Issue snapshot saving after each successful workflow run
  • Stream exchange clipboard copy extracts only the design property instead of the full stream object; design payload encoded before write with graceful fallback
  • Metadata issue detail view merges "Used by streams" and "Set by streams" into a unified reference table for a complete dependency overview

v2.7 — License Model & Settings Refactor

  • License activation system: hardware ID derived from a persistent random token stored in the data volume (.helmut4_scout_hardware_id)
  • External key generator: generate_scout_unlock_key.sh using HMAC-SHA256 — kept outside the Docker image to prevent abuse
  • Protected actions (Run workflow, Compatibility Checkup) gated behind valid license key; server-side validation on every protected POST
  • Browser-side key caching in localStorage/cookie as convenience — server re-validates each action regardless
  • Settings page refactored: License Activation, Import, Compatibility Checkup, Workflow Status, and Data sections
  • Hardware ID copy-to-clipboard; license key paste-from-clipboard
  • HELMUT4_SCOUT_UNLOCK_SECRET environment variable for key generation

v2.6 — Severity Distribution & Snapshot Comparison

  • Severity distribution donut chart on the Dashboard using pure SVG arc rendering — no external charting library
  • Acknowledged issue counts tracked separately from totals
  • Snapshot files (issue_snapshot.json, issue_snapshot_previous.json) written after each workflow run
  • "vs. prev" delta column in the severity distribution table when two snapshots exist
  • Snapshot timestamp caption shown below the chart

v2.5 — Graph, Table UI & Visual Improvements

  • Interactive vis-network reference graph: streams, variables, metadata, profiles, tasks, cronjobs, and products
  • Graph modes: documentation (default), audit, configuration — controls which catalog entries and variable types are included
  • Graph node tooltips as DOM elements (compatible with vis-network v9+ plain-text rendering)
  • Light / Dark theme toggle for the graph
  • Static SVG issue summary graph via Graphviz (when available)
  • Node group colours per type (stream, variable, metadata, profile, module, encrypted variable, etc.) with fixed legend
  • Table drag-and-drop column reordering; resizable columns; sticky table headers
  • Column order and widths persisted in browser storage; reset clears all overrides
  • Fixed table cell overflow where long values could visually overlap neighbouring columns

v2.4 — Backup Import & Extended Variable Support

  • MongoDB backup import support alongside live API import
  • tar.gz format: standard mongodump directory archive
  • Nested archive detection and extraction for wrapped backups
  • import_backup_data() handling all major Helmut4 MongoDB collections
  • Encrypted variable support — {helmut.encrypted.variable.<name>} references; dedicated Markdown reference file
  • Virtual variables for PATH_MAPPING Windows/UNIX path entries
  • Filtering of sensitive/internal Helmut preference keys (PURGE_JOBS, VARIABLES, TEMPLATES, etc.) from normal variable display
  • MODULE_* Helmut variables excluded from documentation/variable tables; grouped separately in configuration graph mode
  • H4_IMPORT_SOURCE, H4_BACKUP_ARCHIVE environment variables for non-interactive runs

v2.3 — Compatibility Checkup

  • Compatibility checkup against bundled Helmut4 node and wildcard reference versions
  • node_reference/<version>/ directory with per-version node and wildcard catalogs
  • Checks: missing nodes, deprecated nodes, renamed/removed properties, outdated enum option lists in stream designs
  • Wildcard compatibility matching with ? and # placeholders; parameterised wildcard definitions resolved correctly
  • Compatibility issues written to reference_issues table with Compatibility: prefix
  • Selectable reference version in Settings; highest available version selected by default
  • Compatibility check skips customer-modified default values where the change is intentional

v2.2 — Global Search & OpenAPI

  • Full-text global search across streams, issues, variables, metadata, profiles, nodes, and wildcards in a single query
  • /search route with unified result view
  • OpenAPI 3.0 specification (openapi.yaml) covering all API endpoints
  • Swagger UI available at /api-docs — accessible without license
  • Support for {project.custom.?} and {job.custom.?} as custom integration metadata references; scope display updated accordingly
  • Fixed project.custom / job.custom references causing false missing-definition errors
  • Fixed false missing-definition error for helmut.profile
  • Fixed false encrypted-variable error caused by documentation example XXX

v2.1 — Issue Severity System & Stream Detail

  • reference_issues SQLite table with Severity, Issue type, Reference type, Reference key, Stream
  • Seven severity levels: Error, Deprecated, Compatibility, Warning, Orphan, Dynamic Reference, Info
  • Issues page with severity filter tabs, search, pagination, and bulk acknowledgement
  • Issue links to stream and node detail pages where resolvable
  • Issue Summary and Recent Issues panels on Dashboard
  • Stream detail node table with columns: #, Type, Title, Description, Summary, References
  • Node title and description enriched from design metadata per operator
  • Deprecated node detection and deprecated node count added to stream overview data
  • Dynamic reference candidate detection across all reference types
  • Fixed case-sensitive handling for metadata and variable name lookups
  • Fixed metadata rows incorrectly described as unreferenced for Description fields
  • Fixed deprecated stream-node detection and malformed/old wildcard syntax reported as deprecated

v2.0 — Docker Application (Initial Release)

  • Flask web application wrapping the reference builder
  • Docker image: python:3-slim base; docker-compose.yml with named volume for data persistence
  • viewer_settings.json for persisted import configuration
  • Initial sidebar navigation: Dashboard, Issues, Variables, Metadata, Profiles, Streams, Settings, Manual, Table
  • Collapsible sidebar with compact icons/initials in collapsed state
  • Settings page: live API configuration with side-by-side Import and Compatibility Checkup panels; Workflow Status; Data/download actions
  • Always-visible workflow console with live log polling; toast notifications for actions
  • In-app Manual page
  • Generated Markdown documentation files: profile_reference.md, metadata_reference.md, store_variable_reference.md, helmut_variable_reference.md, encrypted_variable_reference.md, temporary_variable_reference.md
  • Variable Markdown tables: Helmut variables include value column; store variables include value and dynamic columns
  • User, group, access preset, and access assignment references extracted
  • Event-to-product mapping from events_by_product catalog
  • Stream exchange JSON stored per stream for design inspection and clipboard copy
  • Fixed profile Markdown product tags by deriving product from main stream event
  • Removed separate combined variable Markdown approach in favour of per-type pages

v1.9 — SQLite3 Database

  • All CSV and JSON intermediate files imported into a single helmut_reference.sqlite database
  • SQLite used as the primary data store for all viewer queries — replaces direct file reads
  • Output pipeline refactored: build CSVs → import to SQLite → clean up intermediates
  • _manifest table tracking table names, source files, and row counts
  • Streaming pagination and column-based sorting on all table views
  • Generic table browser at /tables/<name> with full-text search

v1.1 – v1.8 — Mechanics, References & Graph Generation

Iterative development of the core reference builder shell script:

  • v1.1 — Initial live API calls to /v1/streams; stream ID extraction via jq
  • v1.2 — Stream export decoding (base64 + JSON); individual stream JSON files written; profile-to-stream and profile-to-metadata-set reference extraction
  • v1.3 — Variable catalog extraction: store variables, Helmut variables, preferences; task and cronjob reference extraction
  • v1.4 — Metadata and metadata set import; profile and stream cross-referencing; user, group, and access preset reference extraction
  • v1.5 — Wildcard reference scanning; old percent-syntax detection; Helmut wildcard handling for known system wildcards
  • v1.6 — Node catalog and product event catalog import; event-to-product mapping; stream node reference catalog generation
  • v1.7 — Markdown reference file generation; stream complexity metrics; deprecated node detection introduced
  • v1.8 — Interactive HTML reference graph using vis-network with nodes/edges for profiles, streams, pre-streams, metadata sets, tasks, cronjobs, products, and encrypted variables; Graphviz SVG issue summary

v1.0 — Reference Viewer (Shell Script)

The origin. A single Bash script — originally named createReferencesForVariablesMetadataAndStreams.sh — that connected to a live Helmut4 API, downloaded all stream exchange data, and generated a set of Markdown reference files. No database, no web interface, no graph. Just structured text output for documentation repositories.

The script established the foundational data model that every subsequent version built on: streams, variables, metadata, profiles, and references as the core entities of Helmut4 configuration analysis.