# AyMON — every page in one file Hand this file to an AI assistant and ask it about AyMON. Source: https://github.com/akya-dev/AyMON ============================================================================== PAGE: AyMON URL: http://docs.aymon.ca/ ============================================================================== # AyMON AyMON is an industrial monitoring and automation engine. This repository is where you get it: the engine for Windows and Linux in `bin/`, and runnable projects for learning, teaching and hobby use. Every example starts with one command and needs **no licence, no hardware and no database** unless its own README says otherwise. The same pages as a searchable site: **https://docs.aymon.ca/** ``` demo\ bin\win-x64\ the engine (AyXDB.exe) and its runtime, Windows x64 bin/linux-x64/ the same engine for Linux x86-64 (AyXDB + lib*.so) bin\lic\ the education / personal licence -- one file, every example and every platform PRJEDU_101\ first steps: timers, scripts, a calculated value, an alarm PRJEDU_modbus\ Modbus TCP, both ends, no hardware PRJEDU_sqlite\ history in a database, nothing to install PRJEDU_odbc\ the same history on SQLite, SQL Server or PostgreSQL, chosen by a connection string PRJEDU_rep\ replication: two engines, one plant PRJEDU_cluster\ a cluster: two engines, one job, and a failover - kill the master, the spare carries on from the same unit PRJEDU_web\ a live web page: HTTP, REST, MQTT over ws:// and wss:// PRJEDU_pages\ web pages the engine writes: .ayp templates (<% %>, as in ASP / JSP / EJS) read the plant on each request - no JavaScript, no API PRJEDU_mqtt\ MQTT, both ends: AyMON as the broker and as a client PRJEDU_telnet\ a maintenance shell into a running engine, and the three gates in front of it PRJEDU_hardening\ a plant delivered with seven access weaknesses: find and fix them, with a self-check and an access test on the page PRJEDU_snmp\ SNMP, both ends: a manager, and an agent written in script PRJEDU_opcua\ an OPC UA client against a moulding machine's own server: poll, subscription, writes, methods, certificates both ways (the machine is Python/asyncua: needs Python 3.10+) PRJEDU_mcp\ an AI assistant reads the plant, and may change one value PRJEDU_gauge\ a camera reads a dial gauge: Python and numpy beside the engine, the picture a blob fetched over HTTP (needs Python 3.11) PRJEDU_pycall\ Python called from the XML and from JScript, asynchronously: a pump bearing's vibration analysed in Python (needs Python 3.11) PRJEDU_vision\ a camera inspects bottles on a filling line, in the engine: the camera is a connection, the inspection IMG.* in JScript (~500 frames/s), the line watched by PRO.vision without a script PRJEDU_pid\ a PID lab: a heater, its controller, step responses and a suggested tuning PRJEDU_plc\ a PLC program for a bottle filling line: remote I/O over Modbus, a step sequence, interlocks, an HMI PRJEDU_oee\ OEE of a production line, shift by shift: a machine behind seven Modbus registers, an MES, losses, a shift report in SQLite PRJEDU_robotarm\ a robot arm in a cell: a six-axis cobot (dashboard + Modbus, as on a UR) and the cell controller around it PRJEDU_robot_mcp\ the same cell run by an AI assistant over MCP - inside rules it cannot change (seven connections, inside FREE) PRJPER_tello\ fly a Tello drone from a web page - or the simulated one that comes with it PRJPER_adsb\ the aircraft overhead on a map, from an ADS-B receiver's SBS-1 stream - or the simulated receiver that comes with it PRJPER_phone\ your phone's motion sensors as a device with alarms - no app, a page over https on the same Wi-Fi (or a simulated phone) docs\ six short guides, written for these examples licenses\ third-party licence texts ``` ## Run one ```bat cd PRJEDU_101 run.bat ``` `run.bat` finds the engine in `..\bin\win-x64` and hands it the licence in `..\bin\lic` with `-lic:` — one licence per process, so projects under different licences can run side by side on one machine. Stop it with CTRL-C. Everything the engine did is in that project's `log\` directory and on the console. To point at an engine installed somewhere else: ```bat run.bat "C:\Program Files\AyMON\bin" ``` ## On Linux Every example has a `run.sh` beside its `run.bat` (and `run_plant.sh` / `run_centre.sh`, `shell.sh`, `try.sh` where there is a `.bat` of that name): ```bash cd PRJEDU_101 ./run.sh ``` It does what `run.bat` does: finds the engine in `../bin/linux-x64`, hands it the licence in `../bin/lic` with `-lic:`, and starts it in the project directory. Stop it with Ctrl-C (or ESC); Ctrl-X opens the maintenance prompt. The same education licence works on both platforms. | | | |---|---| | **Machine** | x86-64 with glibc 2.34 or newer and OpenSSL 3: Ubuntu 22.04+, Debian 12+, Fedora 35+, RHEL / Rocky 9, and WSL on Windows. The engine takes `libssl.so.3` from the system (`sudo apt install libssl3` if `run.sh` says it is missing); everything else ships in `bin/linux-x64`. ARM (Raspberry Pi) is not in this repository yet. | | **Helpers** | `PRJEDU_telnet/shell.sh` and `PRJEDU_mcp/try.sh` run small Python 3 scripts (standard library only) — the counterparts of `shell.ps1` / `try.ps1`. | | **Python in the engine** | only `PRJEDU_gauge` and `PRJEDU_pycall`: `libAyLIB_Python.so` in `bin/linux-x64` runs their scripts in **Python 3.11** (Debian 12's default; deadsnakes on Ubuntu) — the gauge with numpy installed for that version, pycall with the standard library only. Every other example runs without it — the module is loaded only when a document has a Python script. | | **ODBC** | `PRJEDU_odbc` runs on its embedded SQLite out of the box. For PostgreSQL or SQL Server install unixODBC and the driver (`sudo apt install unixodbc odbc-postgresql`); on Linux the drivers are named in `/etc/odbcinst.ini` — `{PostgreSQL Unicode}`, not Windows' `{PostgreSQL Unicode(x64)}`. | | **Paths** | the projects write `%PRJ%` / `%DB%` paths with `/`, which both platforms read; a file database written as `Database=log\x.db` works too — on Linux the engine reads the `\` as a separator. | | **Copied from Windows?** | a zip loses the executable bits: `chmod +x */*.sh bin/linux-x64/AyXDB`. A `git clone` keeps them. | ## The examples | Project | What it teaches | Needs | |---|---|---| | **PRJEDU_101** | the document tree, a thread and its clock, a timer-driven value, a calculated value, an alarm with limits, reading the log | nothing | | **PRJEDU_modbus** | `` / `` / `` / ``, the block as the unit of communication, register and BIT addressing, scaling and alarms as configuration — and the **slave** side, so the plant and the monitoring side are two documents of one engine; a live page at http://127.0.0.1:8103/ | nothing | | **PRJEDU_sqlite** | the SQL surface: `exec_direct` vs `prepare`/`parameter`/`execute` and why a value is never pasted into a statement, `SQL.qry` turning rows into addressable elements, what an SQL NULL is in script, and a history that survives a restart; a live page at http://127.0.0.1:8106/ | nothing | | **PRJEDU_odbc** | connection strings kept in the root document and handed to the `` in `xdb_init`, the ODBC and SQLite drivers attaching themselves, one script whose SQL runs unchanged on three databases — and the traps between them (LIMIT/TOP, `round()`, a Turkish collation); a live page at http://127.0.0.1:8107/ | nothing for SQLite; SQL Server or PostgreSQL to try those | | **PRJEDU_rep** | replication between two engines on one machine: the mirror that has no file of its own, direction decided by the document and not the connection, one connection row that works out its own end, and why an unlicensed feature is a *local* feature; a live page at http://127.0.0.1:8108/ | nothing | | **PRJEDU_cluster** | two engines that share one job: an election (`PRO.clt`) over a link on each node, the role written into the job document's `sta` (TRANSIT, master, spare), replication (`PRO.rep`) whose direction follows the role, one set of files for both nodes told apart by `tid`, `CLT.info` for the cluster's state in a script, a write on the spare forwarded to the master, a planned hand-over — and a page that shows both nodes side by side while you kill one; measured: master again 2 s after the kill, 0 units lost | nothing | | **PRJEDU_web** | a live page in a browser: static files over HTTP, a JSON snapshot over REST, live values and a write-back over MQTT inside a WebSocket — one port for all three, the same again over TLS (https / wss), and why the web side is configuration, not code | a browser | | **PRJEDU_pages** | web pages the engine writes: a file ending in `.ayp` is a template (`<% %>`, `<%= %>` escaped, `<%- %>` raw, `include`, as in ASP / JSP / EJS) that the engine runs on every request with the same `evt.GetValue` every script uses — no JavaScript in the browser, no API to design; a tank farm's overview, a form that changes set points (checked on the server, POST then redirect), the same data as JSON for a program, and a page that shows it runs with the **visitor's** rights (`xdb_acl`); a live page at http://127.0.0.1:8115/ | nothing; engine 4.3.4.588 or later | | **PRJEDU_mqtt** | MQTT with nothing to install: AyMON as the broker that publishes its own document and takes writes on `.../set`, and as a client whose ``/``/`` are a publisher, a subscription and a topic — retain from the document, what stays local, writing back without an echo, and pointing it at mosquitto instead; a live page at http://127.0.0.1:8104/ | nothing; mosquitto to try the optional steps | | **PRJEDU_telnet** | getting inside an engine that has no console: the maintenance shell over plain TCP and over TLS, the three gates in front of it (`con_acl` at accept, `con_auth` at login, `xdb_acl` per element), what four different roles see of the same plant, project verbs through `pro_oncmd`, and following the log live — with a terminal for it in the box | nothing | | **PRJEDU_hardening** | a security course's lab, the defending side: a small plant (tank, pump, high-level trip, a secret recipe) delivered with seven weaknesses in who may do what — no address filter, no login and every connection an Operator, no default deny, the safety trip open to operators, the recipe open to all, passwords in clear text, unlimited guesses — to be fixed in its XML; the plant checks its own configuration, and the page logs in as four identities and tries what each may and may not do; a solution file marks every fix; measured: without a password the tank could be made to overflow, after hardening 8 of 8 | nothing | | **PRJEDU_snmp** | SNMP with nothing to install: the manager side as you would write it for a real device — one request carrying every point of a block, the OID split between `` and ``, types and scaling — and the device side written in script on `PRO.raw`, which is how any protocol the engine does not speak can be tried out first; a live page at http://127.0.0.1:8105/ | nothing | | **PRJEDU_net** | the engine writes mail and files: a tank's level alarm sends one mail when it comes and one when it is gone — a mail is a **write to a point** of a `PRO.smtp` connection and the server's answer comes back **into the same point**, the connection is made for the message and let go when quiet, sending runs on a thread of its own; a shift report is written, zipped (`FILE.zip`) and uploaded by `PRO.ftp` as `.part` then renamed, and the directory listed; the mail and file servers at the other end are written in script on `PRO.raw` — they speak first (`pro_onaccept`) and hear a connection end (`pro_onclose`, the end of an FTP upload), so nothing is installed; how to point it at a real mailbox or FTP server and the licence's 20 operations a day; a live page at http://127.0.0.1:8114/ | nothing; engine 4.3.4.568 or later and its module `AyLIB_net` (in `bin`) | | **PRJEDU_opcua** | an OPC UA client against a machine that comes with its own server — a simulated injection moulding machine on asyncua, somebody else's stack: NodeIds by namespace URI (`nsu=`) rather than index, a poll and a subscription of the same value side by side (30 against 135 changes in 30 s), why the subscription gets a connection of its own (a Publish waits at the server and held the 1 s poll to 0.65 reads a second), one session one thread so the page's requests are carried out by a timer on the session's thread, writes and method calls (`OPC.call`) signed and encrypted as a user, the anonymous write the server refuses, certificates trusted in both directions (`OPC.cert`, the machine's rejected folder, `pro_trust`), and `OPC.browse` / `OPC.gen` for where the NodeIds come from | Python 3.10+ for the machine (`setup.bat` / `setup.sh` installs asyncua into the example's own folder) | | **PRJEDU_mcp** | an AI assistant (Claude Code, Claude Desktop, any MCP client) on the plant: publishing a document with `mcp_pub`, a read-only and a read+write endpoint, why the endpoint's role and each element's `xdb_acl` decide everything, the four answers to a write (`OKY` / `refused` / `PARTIAL` / `dry_run`), and tools a project adds in script — with every call runnable without an AI; a live page at http://127.0.0.1:8109/ | nothing; an MCP client to try it with an assistant | | **PRJEDU_gauge** | Python beside the engine: a ` ``` | Field | Meaning | |---|---| | `id` | **The module name.** Its functions are reached as `main.foo()` — not the file name | | `xdb_fnc="[SCR.prs]"` | Binds the loader. **Without it the script is never loaded and no error is given** | | `src` | The file. Path macros work: `%DB%`, `%PRJ%`, `%BIN%`, `%ROOT%`. Leave it out and the element's own text is the source | | `language` | `JScript` is the default. `Python` runs through the engine's Python module (`AyLIB_Python`, in `bin\win-x64`) and needs an installed 64-bit Python 3.11 — see `PRJEDU_gauge` (numpy and camera frames), and `PRJEDU_pycall` for calling it (below) | The file must be **UTF-8**. A small helper does not need a file at all: ```xml ``` ## How the XML calls it There is no `main()`. Script runs in exactly three situations: **1 — A name in a `*_fnc` chain.** ```xml ``` **2 — A life-cycle hook on the document root.** These are written as an expression — a call, not just a name: ```xml ``` `xdb_init` runs while the document is being set up (this is where you write configuration such as a connection string); `xdb_start` runs once everything is up. **3 — Inside an address**, which is how a web front end reaches your code: ``` doc('HIS/S00')/api.recent(10) ``` > **The `api` module is a convention with teeth.** A call arriving from > outside the engine can reach only the `api.` scope; other modules are > reserved for local calls with an engineer-level role. Put the functions the > outside world may call in a module named `api`. ## The function you write ```js function read(evt) { var n = evt.obj.cnt + 1; // the element this event is on evt.obj.cnt = n; var v = 20.0 + 8.0 * Math.sin(0.05 * n); evt.SetValue("id('ALARM')/@val", v); return 0; } ``` Every function called through a chain receives one argument, `evt`: | | | |---|---| | `evt.obj` | The element this event is on | | `evt.valn` / `evt.valp` | The value being written / the previous one | | `evt.stan` / `evt.stap` | The same for `sta` | | `evt.atr` | Which attribute is being written | | `evt.user_role` | Who caused it (0–7 user, 8 document, 9 system) | | Method | | |---|---| | `evt.GetValue('adr')` | Read | | `evt.SetValue('adr', v)` | Write | | `evt.MskValue` / `XorValue` / `AddValue` | Masked / bitwise / arithmetic update | | `evt.GetElement` / `AddElement` / `DelElement` | Element operations | | `evt.ForEach(adr, 'handler(evt, idx, prm)', prm)` | Loop over elements | | `evt.QueValue(adr, v, pri, delay_ms, syn, thr)` | Do it later, or on another thread | | `evt.Log(level, str)` | `0` PANIC · `3` ERROR · `4` WARN · `6` INFO · `7` DEBUG | Adding an element built from text takes three steps, and the first two are easy to miss: parse it with `-1` (the XDB fields on — without it `id()` never finds the new element), then take it out of the little document it was parsed into, because `AddElement` only takes a node that belongs to no document (it returns 0 otherwise). `PRJPER_adsb` does it once per aircraft: ```js var x = new XML("", -1); var ac = x['ac']; x.removeChild(ac); evt.AddElement("id('SKY')", ac); // 1 = added evt.DelElement("id('AC_4B1805')"); // and gone again ``` An element added this way lives in **this engine only**: replication does not carry it, so a mirror or a spare never sees it. When the list is part of a replicated document, let the list make its own elements with `ELM.new` — write what the new element should hold into the list's `@new`, and a script of yours (`new_elm_new_eval`) fills it in. `PRJEDU_cluster` keeps its batch list that way: ```xml ``` `new_elm_new_max="B10"`: ids `B01` … `B10`, ten kept. In `fill_item`, `evt.obj` is the new element and `evt.valn` what was written; return 1 to keep it. ## Addresses An address names an element and an attribute: ```js evt.GetValue("id('LEVEL')/@val") // same document evt.GetValue("doc('PRC/P01')/id('LEVEL')/@val") // another document evt.GetValue("doc('PRC/P01')/count(data/var)") // a function over it evt.GetValue("count(id('L_RECENT')/item)") // same document, counted ``` `id('X')` is indexed — it does not walk the tree, so it is cheap even in a document with thousands of elements. > **`doc()` comes first, the function after it.** > `doc('X')/count(data/var)` works; `count(doc('X')/data/var)` silently > answers `0`. The same order applies to `xml(...)` and `json(...)`. ## Seven places this is not JavaScript The language is close enough to JavaScript to be readable and different enough to bite. These seven are the ones that actually cost people an afternoon. **1 — Arithmetic is typed, and the LEFT operand decides.** ```js c / 12 // integer division if c is an integer c / 12.0 // what you meant 20 + 0.22 // 20 Math.round(x) / 100.0 // still an integer — Math.round returned one ``` There is no error and no log line. Write the decimal point. **2 — `x == ""` does not catch a missing value.** A missing attribute, a missing JSON key, an element that is not there and an SQL NULL all arrive as `undefined`. Use `!x` for "missing or empty", `x == null` for "missing". **3 — Ordering operators with `undefined` are not JS.** `undefined < 1` is true. Do not rely on them; test for the missing value first. **4 — Hex values stay hex.** Arithmetic on a hex-subtype value (such as `evt.stan`) produces a hex result, and comparing it with `1` fails: ```js if (evt.stan == "0x0303") { ... } // correct ``` **5 — `switch`, classes, `async`/`await`, template strings, destructuring, spread and real regular expressions do not exist.** `var`/`let`/`const`, `if`/`for`/`while`/`do`, `for (x in obj)`, `try`/`catch`/`finally`, `throw`, `?:`, `??`, `?.`, arrow functions and object/array literals all do. A regex literal is accepted only for plain text and is turned into a string at compile time; a metacharacter in one is a **compile error** with a line and column, not a silent wrong answer. **6 — String indices are characters, not bytes.** `String.length` counts characters, `String.size` counts bytes. The difference only shows up outside ASCII — which means it shows up in production and not in your test. **7 — An integer attribute can be unsigned.** In `val_typ` the class `0x01..` is UNSIGNED and `0x02..` signed: `0x0104` is an unsigned 32-bit integer, `0x0204` a signed one, `0x0108` / `0x0208` the 64-bit pair. Read an unsigned value and subtract, and the result does not go negative, it wraps: ```js // U32 is val="3" val_typ="0x0104", I32 the same with val_typ="0x0204" get(evt, "U32") - 5 // 4294967294 get(evt, "I32") - 5 // -2 parseInt("" + get(evt, "U32")) - 5 // -2 ``` Keep time stamps and anything you subtract in a signed type (`0x0208` for milliseconds), or read it through `parseInt`. `Time.GetTimeMs()` itself is signed from engine 4.3.4.466 on; before that it was unsigned and `now - later` wrapped the same way — `0.0 + Time.GetTimeMs()` is the old remedy, and it still works. ## What is built in | Scope | | |---|---| | `Math` | `abs floor ceil round trunc sign min max pow sqrt log exp sin cos tan atan2 random` | | `String` | search, slice, case, trim, split, replace, pad, `fromCharCode` | | `Array` `Number` `Object` `JSON` `Buffer` | the usual, with the differences above | | `Time` | `now` `GetTimeStr` `FmtTimeStr` `GetTimeMs` `ParseTimeMs` `AddDurationMs` `SleepMs` | | `File` | `open close read gets write puts remove rename` | | `CODEC` | `B64_enc/dec` `Z64_enc/dec` `HEX_enc/dec` `JWT_sign/verify/dec` | | `SQL` | see below | | `FILE` | `list exists mkdir rmdir move zip` — directories and ZIP archives, from the module `AyLIB_net`; Engineer and above (`PRJEDU_net`) | | `IMG` `CAM` | image functions on blobs and the camera list, from the module `AyLIB_vision` (`PRJEDU_gauge`, `PRJEDU_vision`) | `FILE`, `IMG` and `CAM` live in engine modules beside the engine in `bin`; the first call attaches the module, no `xdb_lib` needed. `FILE` is refused to a script started from a page that did not log in: it runs with the page's role, and the engine's own timers run as the engine (`PRJEDU_net` makes its report on a timer for that reason). There is no `Date`. `Time.now()` (engine 4.3.4.530 and later) is the instant as a time value — it prints as `2026-10-01T12:00:00.123+03:00`, and `t2 - t1` is milliseconds. The text stamp most examples log is `Time.GetTimeStr(0)` → `2026-09-20 22:31:04`. **Every scope documents itself**, and the key is always lower-case `help`: ```js evt.Log(4, Math.help); evt.Log(4, SQL.help); evt.Log(4, evt.help); evt.Log(4, Object.keys(Math)); ``` That is the live list. If anything here disagrees with it, believe the engine. ## The engine functions you put in chains These are normally written in **XML**, not called from script. The names you will meet in the examples: | Scope | What it does | Seen in | |---|---|---| | `ATR.rou` `ATR.lmt` | round to `val_rou`, clamp to `val_min`/`val_max` | 101, modbus | | `VAL.put` `VAL.cnv` | `pro` → `val`, and scale it | modbus | | `ALM.onlvl` | compare with the four limits, set the alarm bits in `sta` | 101, modbus, rep | | `PRO.pol` `PRO.sta` `PRO.inp` | poll, state, distribute a block | modbus | | `EVT.sta` `THR.sta` | let `sta` start and stop a timer or thread | all | | `XDB.evt` `THR.cre` `SCR.prs` | create the timer / thread / script object | all | | `SQL.qry` | run the SQL in `@qry` and turn the rows into elements | sqlite | | `IF.*` | cut the rest of the chain on a condition | — | | `REP.cfg` | work out this node's end of a replication link | rep, cluster | | `CLT.cfg` `CLT.cmd` | the same for an election link; a hand-over written to `pro_clt_cmd` | cluster | | `ELM.new` | make an element from what was written to `@new` | cluster | | `PRO.out` | send what was written to `pro` to the device; the value is not stored and the chain ends there — the device's answer comes back into the same `pro` and runs the functions after `PRO.out` (4.3.4.559 and later) | snmp, mqtt, net | | `PRO.outc` | the same, but the chain goes on with the value written — for a point nothing ever reads back | — | ## Script can also speak a protocol A connection whose protocol is `PRO.raw` has none of its own: the engine moves the bytes and `pro_onparse` hands them to a function of yours, which decides where one frame ends and may answer on the same socket. ```js function onparse(exc) { var n = exc.buf_recv.len - exc.buf_recv.pos; var b = exc.buf_recv.copy(exc.buf_recv.pos, n); // bytes; b[0] is a number exc.buf_recv.pos = exc.buf_recv.len; // consumed exc.buf_send.remove(); exc.buf_send.append(CODEC.HEX_dec("300b0201 00")); // text -> bytes exc.send(); return RES.OKY; // frame done } ``` `CODEC.HEX_dec` and `HEX_enc` are how a script builds and reads binary: an array of numbers is **not** a byte source, and a byte in a copied buffer cannot be written through its index. Returning `RES.INF_SUB_CONTINUE` after setting `exc.buf_recv.nxt` asks for more bytes instead. `PRJEDU_snmp` does this for real: its device end is an SNMP v2c agent in script, because `PRO.snmp` is a manager and does not answer. Reach for it when the protocol is not one the engine knows — and expect to keep the parsing small; a format that turns out to matter belongs in a plug-in later. ## Script can also write a web page A file under an HTTP connection's `pro_http_root` whose name ends in `.ayp` is a **server page**: the engine runs it on every request and answers with what it writes. It is a script with HTML around it, in the syntax ASP, JSP and EJS use: ```html <% var t = main.tank(evt, "T1"); -%>

<%= t.name %>: <%= t.level %> %

<% if (t.alarm > 0) { %>

<%= t.text %>

<% } %> ``` A page sees `evt`, `req` (`req.getForm("sp")` reads a form field), `res` (`res.status`, `res.type`, `res.header`, `res.redirect`), `ses` and your modules by name — `main.tank()` above is an ordinary function in the document's script. Everything in this guide holds inside a page: typed arithmetic, `evt.GetValue` addresses, `return` to stop early. Statements need their `;` (there are no automatic semicolons). A page that fails answers **500** and logs why, with the template's line number; its source is never sent. `PRJEDU_pages` is built this way, and `PRJEDU_oee` adds a printable shift report (`www\report.ayp`) to an existing project. A page runs with the visitor's rights — see `06_Access_and_Safety.md`. ## Calling Python A Python script (`language="Python"`) runs on the engine's one Python thread. A function in it marked `@xdb_fnc` becomes callable from the XML and from JScript, under the script's `id`: ```python @xdb_fnc def analyse(evt): # one parameter named evt: an EVENT call blk = json.loads(evt.valn) xdb.SetValue("id('RMS')/@val", rms) @xdb_fnc def forecast(limit, stamp): # anything else: an ARGUMENT call xdb.SetValue("id('FORECAST')/@val", json.dumps(out)) ``` ```xml ``` ```js var q = vib.forecast(4.5, Time.GetTimeMs()); // q = 1: queued ``` **Both are asynchronous.** The call goes into the Python thread's queue and the caller goes on; JScript gets `1` ("queued"), never the function's result. The result comes back the way every value does: Python writes it into a point, and that point runs its own `val_fnc` — rounding, alarms — like any measurement. In an event call the engine stores the attribute itself after the function returns. `PRJEDU_pycall` measures both calls on every call, and shows what happens when Python is slower than the data. ## Pictures and other large values: blobs A camera frame does not belong in a string. An attribute declared as a blob (`val_typ="0x050E"`, `0x150E` with its `val_fnc`) holds large binary data by reference: writing it, reading it, handing it to a `val_fnc` or to Python never copies the bytes. MQTT and JSON show only a descriptor, and a saved XML leaves the attribute empty (a blob is not persistent): ``` {"blob":1,"len":13890,"fmt":"jpeg","seq":24,"tim":0} ``` ```js var b = Blob.from(buf, { fmt: "jpeg", seq: n }); // bytes in: one copy, numbered b.len b.fmt b.seq b.w b.h b.dtype // read-only; no b[i] var s = IMG.stats(frame, { x: 60, y: 60, w: 120, h: 120 }); // AyLIB_vision, in C var j = IMG.encode(frame, "jpeg", 80); ``` From Python a numpy array becomes a blob with one copy, and a blob comes back as a read-only numpy view with none. A `PRO.http` connection with `pro_blob="/blob"` serves the bytes from memory, once (`GET /blob/DOC/NODE/ID/val`) or as an MJPEG stream (`?stream=1`). The `IMG.*` functions come from the engine module `AyLIB_vision`, named on the root document (`xdb_lib="AyLIB_vision"`; up to 4.3.4.562 the plug-in `AyPRO_img`). `PRJEDU_gauge` uses all of it, and keeps the old way (a base64 PNG in a string on MQTT) one button away, to compare. ## SQL from script The shape never changes: ```js var con = SQL.attach('CN_SQL'); // borrow from the 's pool var rs = SQL.exec_direct(con, sql); // or prepare / parameter / execute while (SQL.fetch(rs)) { ... SQL.getdata(rs, 1) ... } SQL.close(rs); // always ``` You never disconnect `con` — it belongs to the connection's pool and goes back by itself. Closing the **result set** is what you owe. * Use `prepare` / `parameter` / `execute` the moment a **value** goes into the statement. A bound value is data and never becomes SQL. `PRJEDU_sqlite` proves it by storing a tag called `x'; DROP TABLE reading; --` every tenth reading; the table is still there. * **`SQL.getdata(rs, n)` counts from 1**, not from 0. * **An SQL NULL comes back as `undefined`** — see difference 2 above. * `SQL.qry` is the other way in: write SQL into a `` element's `@qry` and every row becomes an `` whose attributes are named after the columns. After that the rows are addressable: `id('L_RECENT')/item[1]/@val`. ## Debugging There are no breakpoints and no stepping, **on purpose**: a script runs on a connection, timer or worker thread which may be holding a session lock, and stopping it would not debug the system, it would freeze it. Instead: | | | |---|---| | `evt.Log(4, ...)` | The first tool. The examples use nothing else | | `SCR.trace` | Statement trace | | `SCR.prof` | Function timings | | `SCR.dump` | Dump a scope | A syntax error is reported with line, column and the source line. A runtime error is reported with a call stack — **unless it is inside a `try`/`catch`**, which swallows it. If you expect an error in the log and there is none, look for the `catch`. ## Checklist 1. Does the ` ``` ```python from XDB import * def init(xdb_in): # once, after the file is compiled global xdb, xdb_evt xdb = xdb_in xdb_evt = [["IMG", "val", cam_init, cam_loop, cam_done, {}]] def cam_loop(evt, st): # every thr_per; a frame when one is due true_value = float(xdb.GetValue("id('TRUE')/@val")) ... xdb.SetValue("id('FRAME')/@val", frame) # a numpy array becomes a blob view = xdb.GetValue("id('FRAME')/@val") # and comes back as a read-only view ... xdb.SetValue("id('PRESS')/@val", value) # runs PRESS's val_fnc ``` `xdb.GetValue` / `xdb.SetValue` are the same calls a JScript function makes, and a write runs the element's `val_fnc` chain: `PRESS` is rounded and alarmed by `ATR.rou` and `ALM.onlvl`, whoever wrote it. The chain runs on the thread that wrote: `main.on_frame` and `main.encode` are JScript, but they run on Python's thread, not on the document's, which keeps the image work off the document's timers. **A measurement is more than a number.** `PRESS` is only written when the reader trusts the frame. `CONF` has its own low alarms, and `READ` says in words why a frame was not used. Downstream nothing knows the value came from a picture. That is the point of a retrofit. **Calibration is configuration.** `CAL_V0`, `CAL_V1`, `CAL_A0`, `CAL_A1`, `CAL_R` and `CONF_MIN` are ``s that Python reads every frame. For a different gauge you change them in the XML, not in the code. They describe the gauge to the **reader** only. The camera draws the real dial from its own constant, so a wrong calibration gives a wrong reading, as it would in a plant. **One frame, two readers.** `main.on_frame` measures the light on the dial with `IMG.stats` in C, and Python reads the needle with numpy, from the same bytes: the page shows the address the script saw, and `SAME` is 1 when Python's view starts at that address. Behind a numpy view of a blob sits the engine's `XDB.Blob`, which keeps the frame alive while the view lives. numpy puts a `memoryview` between the two (`view.base` is the memoryview, `view.base.obj` the blob), so `holder()` in `db_GAU.py` follows the chain. Hand that blob, not the array, back to `SetValue`, and the engine gets the same frame again, not a copy. **Every number handed to the engine is a plain `float` or `int`.** A numpy scalar is stored as a number only in a typed attribute, so the script writes `float(x)`. An array is different: it becomes a blob. ## Try this | Do | See | |---|---| | **SIZE 1**, then the three **PATH** buttons | "What the picture costs": base64 moves the whole picture through MQTT; the blob paths leave MQTT with the values and the descriptors only | | open **http://127.0.0.1:8091/blob/GAU/S00/PIC/val?stream=1** in a second tab | the same pictures as an MJPEG stream, no page at all (PATH 1 or 2). Each write of `PIC` is one part | | press **3 shadow** and watch the red ✕ marks on the trend | the frames the reader refused. `PRESS` holds its value through them | | **3 shadow**, then **1 darkest ray** | the first version of the reader. Wait for a shadow: now and then the read line jumps several bar and is **not** marked ✕ | | set `CONF_MIN` to `0` in `db\db_GAU.xml` and restart | nothing is refused any more. At NOISE 3 a shadow now and then gives a reading several bar off: the refused frames were the bad ones | | set `CAL_A1` to `130` and restart | the reader believes the scale ends 5 degrees early. The error grows with the pressure, from 0 at the bottom of the scale to about +0.15 bar at the top. That is what a badly measured retrofit looks like. The camera still draws the real gauge (`GAUGE` in `db_GAU.py`) | | set `JPG_Q` to `30` and restart | a picture of about 5 KB instead of 14 KB (measured at 240 × 240), and blocky. The reader does not care: it reads the raw frame, not the JPEG | | untick "show what the reader found" | the picture as the camera sees it. Try reading it yourself at NOISE 3 | ## A real gauge Replace `draw_frame()` with a real picture: a grey `numpy` array, the dial roughly in the middle of the 240 × 240 region the reader looks at. With OpenCV installed (`py -3.11 -m pip install opencv-python`), for example: ```python cap = cv2.VideoCapture(0) # once, in cam_init ok, frame = cap.read() # every frame, in cam_loop frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) # uint8 [h, w]: hand it to FRAME as it is ``` and set `ROI_BIG` in `db_GAU.py` to where the dial is in the camera's picture. Then measure your gauge. Where is the needle at the first and last mark (`CAL_A0`, `CAL_A1`), what do they read (`CAL_V0`, `CAL_V1`), and how many pixels is the dial's radius (`CAL_R`)? The reader expects a light face with a dark needle. For a black face with a white needle, invert the picture (`frame = 255 - frame`). This part is **not tested here**: it needs a camera and a gauge. ## If there is no picture The page says "waiting for the first frame". Look in `log\GAU-*.log` and `log\XDB-*.log`: | Line | Meaning | |---|---| | `AyLIB: 'AyLIB_vision' not loaded: ...AyLIB_vision.dll (Win32 error 126)`, then `engine will not start` | the image module is missing: copy `AyLIB_vision.dll` (`libAyLIB_vision.so` on Linux) with the rest of `bin`. It needs an engine of the same build, 4.3.4.564 or later | | `AyPLG: 'AyPRO_img' is now the engine module 'AyLIB_vision' (C563) -- loaded under its new name; rename it in xdb_lib` | a project written for an older engine: it still works; write `xdb_lib="AyLIB_vision"` and the line goes | | `AyLIB: 'AyLIB_Python' not loaded ... error 126` (Windows) | no 64-bit Python 3.11 found on the `PATH`. Install one "for all users" with "Add to PATH" ticked | | `AyLIB: 'AyLIB_Python' not loaded ... libpython3.11.so.1.0` (Linux) | no Python 3.11 on the machine: `sudo apt install python3.11` | | `GAU: numpy could not be imported - ... No module named 'numpy'` | not installed for **this** Python: `py -3.11 -m pip install numpy`, or `python3.11 -m pip install --user numpy` on Linux (not `pip`, which may be another version) | | `init failed [ERR_COMMON_INVTYPE]` | `AyLIB_Python` and `AyPRO` are from different builds. Copy them together | The `numpy could not be imported` line carries numpy's own explanation, and the next line lists every directory this Python looked in. ## Licence Two connections, two documents: well inside FREE. The Python module and the image plug-in report themselves as free. ============================================================================== PAGE: PRJEDU_hardening — harden a plant that was delivered weak URL: http://docs.aymon.ca/PRJEDU_hardening/ ============================================================================== # PRJEDU_hardening — harden a plant that was delivered weak A small plant — a tank, a pump, a high-level trip, a secret recipe — with a page and live values. It works. It was also set up in a hurry, and its access configuration has **seven weaknesses**. Your task: find them and fix them in `db/db_PLT.xml`, until the plant's own checks say so. ```bat run.bat ``` On Linux: `./run.sh`. Then open **http://127.0.0.1:8101/**. Three people have an account: | user | password | group | role | should be able to | |---|---|---|---|---| | `mon` | `mon123` | monitor | 2 | watch | | `opr` | `opr123` | operator | 3 | + set the setpoint, start and stop the pump | | `eng` | `eng123` | engineer | 6 | + set the high-level trip, read and keep the recipe | Nobody else should see or change anything. (The passwords are in this README because this is a lab; the page's access test uses them too.) ## The task The seven questions — each has a wrong answer in the configuration as delivered: 1. **Who may open a connection at all?** Every listener can name the addresses it accepts. 2. **Does anybody have to log in?** And what role does a connection have before it has? 3. **What happens to an element nobody thought about?** A plant has hundreds; the safe answer for the ones you forgot is "nobody". 4. **Who may move the high-level trip?** It is the last thing between the tank and the floor. 5. **Who may read the recipe?** 6. **How are the passwords stored, and who may read the account list?** 7. **How many guesses does a password get?** Two tools on the page tell you where you are: - **Self-check** — the plant reads its own configuration every five seconds and says what it can see there: questions 1, 2, 3, 6 and 7. - **Access test** — the page logs in as four identities (no user name, `mon`, `opr`, `eng`) and tries what each may and may not do: questions 2, 3, 4 and 5. It also checks the other way: that your fix did not lock out the people who need the access (the rows marked 0). Every value it changes, it puts back. Edit `db/db_PLT.xml`, stop the engine (CTRL-C), `run.bat` again, reload the page, log in, look. A weakness is fixed when both tools agree. Log in as different people and try the controls by hand, too: the page says whether the plant **took** a write. MQTT has no "refused" answer — the page looks whether the value changed — and the engine writes every refusal into `log\PLT-*.log`. ## What you have to work with All of it is in `db/db_PLT.xml`, and all of it is ordinary engine configuration: | Attribute | On | Says | |---|---|---| | `con_acl` | a listener (``) | the addresses that may connect: `127.0.0.1`, `10.40.1.*`, a list separated by commas. Checked when the connection is accepted, before anything else | | `con_auth` | a listener | which `` logs its sessions in. Without it nobody logs in | | `con_role` | a listener | the role a session has before (or without) logging in | | `xdb_acl` | the document, any element | who may do what: four bits per role, role 0 in the lowest hex digit — `0x76543210`. Bit 1 read, 2 write, 4 create, 8 delete. `0xFF113110`: roles 7 and 6 everything, 3 read and write, 5, 4, 2 and 1 read, 0 nothing. An element without its own takes its parent's; **no `xdb_acl` anywhere means no access control at all** | | `usr_pwd` | a `` in `` | the password — in clear text, or as a hash `$pbkdf2-sha256$$$` | | `auth_rehash` | `` | 1 = a password found in clear text is replaced by its hash at the first login (in memory — the file keeps what you wrote) | | `auth_try`, `auth_lock` | `` | this many wrong passwords for one user from one address lock that user, from that address, for this many seconds | Roles: 0 Guest, 1 Customer, 2 Monitor, 3 Operator, 4 Supervisor, 5 Manager, 6 Engineer, 7 Admin. A hash for `usr_pwd`, with nothing but Python: ```python import hashlib, os, base64 b = lambda x: base64.urlsafe_b64encode(x).decode().rstrip("=") salt = os.urandom(16) print("$pbkdf2-sha256$100000$" + b(salt) + "$" + b(hashlib.pbkdf2_hmac("sha256", b"opr123", salt, 100000, 32))) ``` (or `AUTH.hash("opr123")` in the engine's maintenance shell — see [PRJEDU_telnet](../PRJEDU_telnet/README.md)). ## What it measured Engine 4.3.4.467, on a Windows 11 laptop and on Linux (WSL, Ubuntu 22.04): | | as delivered (`run.bat`) | hardened (`run_solution.bat`) | |---|---|---| | self-check | 5 of 5 weak | 5 of 5 OK | | access test | 3 of 8 — only the "0" rows pass | 8 of 8 | | no user name | accepted, as an Operator | refused, CONNACK 4 | | `opr` with a wrong password | accepted | refused, CONNACK 4 | | `mon` writes the setpoint | taken | not taken; `SETPOINT.val: xdb_acl yazma reddetti (rol 2)` in the log | | `opr` writes the trip level | taken | not taken; the same line with `TRIP_LEVEL`, role 3 | | `mon` subscribes | receives the recipe | never receives it | **Why it matters, measured on the plant as delivered:** a page connected *without a password* set the trip level to 120 and the pump to STOP. The tank filled, nothing closed the inlet, and it overflowed — the plant's own message "the tank overflows". **On the hardened plant:** - five wrong passwords for `mon` → the right one refused too (CONNACK 5, `REFUSED: locked - 60 s left`); `eng` from the same address unaffected; `mon` in again about a minute later. A good login in between resets the count — the access test can be run again and again (measured six times in a row) without locking anyone out. - every login leaves a line in the log: `mqtt 'opr'@127.0.0.1 ACCEPTED, role 3 via local`, `REFUSED: wrong password`, `REFUSED: unknown user`. - a refused login is a FAIL line, and the document's own health rises with it: `sta alarm NORMAL -> ALARM` (sta `0x034B`) — the plant's status shows that someone tried. - a refused login also leaves an ERROR line from the MQTT layer (`MQTT_CONNECT_IN : ... res = INF_OKY exception error`) and, as the socket closes, a line or two about the WebSocket closing frame. They are noise, not a second fault; the `AUTH(...)` line is the one that says what happened. ## The solution `db/db_PLT_solution.xml` is one way to fix all seven; every change is marked `FIX n`. `run_solution.bat` (`./run_solution.sh`) runs it. Try it yourself first — and then compare: there is more than one right answer to 1 and 3. ## Not in this lab - **The wire.** The login crosses `ws://` in clear text. On 127.0.0.1 nobody else sees it; on a network it would be `wss://` — [PRJEDU_web](../PRJEDU_web/README.md) shows how. - **The other ways in**: the maintenance shell ([PRJEDU_telnet](../PRJEDU_telnet/README.md)) and an AI assistant over MCP ([PRJEDU_mcp](../PRJEDU_mcp/README.md)) go through the same `con_acl`, `con_auth` and `xdb_acl`. ## Licence **Two connections** (the page and its live values) and one document: inside the **FREE** edition. ============================================================================== PAGE: PRJEDU_mcp — an AI assistant talks to the plant URL: http://docs.aymon.ca/PRJEDU_mcp/ ============================================================================== # PRJEDU_mcp — an AI assistant talks to the plant A small tank plant that runs on its own, and two **MCP** endpoints that let an AI assistant — Claude Code, Claude Desktop, or any MCP client — read it and, within limits, change it. Two windows: ```bat run.bat ``` ```bat try.bat every call an assistant would make, without an assistant ``` On Linux: `./run.sh`, then `./try.sh` in a second terminal (`try.py`, Python 3). Stop the engine with CTRL-C. While it runs, open **http://127.0.0.1:8109/** — the plant as the person beside the assistant sees it: the level, the set point, the pump, the maintenance mode, a trend, and a list of every change the page saw. Run `try.bat` (or ask the assistant to change the set point) and watch the changes arrive. **MCP** (Model Context Protocol) is how an assistant calls tools on another system: JSON-RPC 2.0 over HTTP POST. `PRO.mcp` makes the engine an MCP server. It already knows what an element, a value, an alarm state and a log are, so the built-in tools need no code — the example adds configuration, two small project tools, and nothing else. ## The page `PLT/S00` serves it with two more connections: `CN_WEB` (HTTP, `www\`) and `CN_LIVE` (MQTT inside a WebSocket on the same port, `/mqtt`). It only reads, and it reads as a Guest (role 0): the document's `xdb_acl="0xFF111111"` gives that role read — the same rule that answers the read-only endpoint. So `VENDOR_NOTE` (`0xF0000000`, the administrator's) never reaches the page, and the page says so. Measured with `try.bat`: the page listed the set point going 60 → 70 → 90 → 60 and the pump starting, and `VENDOR_NOTE` never arrived. ## The two endpoints | | URL | Key | Access | Runs as | |---|---|---|---|---| | `CN_MCP` | `http://127.0.0.1:8095/mcp` | `demo-read-key` | read only | role 2, Monitor | | `CN_MCPW` | `http://127.0.0.1:8096/mcp` | `demo-write-key` | read and write | role 3, Operator | Both keys are in this public repository. They are examples, not secrets. ## Connecting an assistant **Claude Code** — this directory carries a `.mcp.json` that names both endpoints. Start Claude Code **in this directory** while `run.bat` is running, approve the two servers when asked, and ask away. Or add one by hand: ```bash claude mcp add --transport http aymon-demo http://127.0.0.1:8095/mcp --header "Authorization: Bearer demo-read-key" ``` **Claude Desktop** speaks to local processes, not to URLs, so it needs the `mcp-remote` bridge (Node.js 18 or later). In `%APPDATA%\Claude\claude_desktop_config.json`: ```json { "mcpServers": { "aymon-demo": { "command": "cmd", "args": ["/c", "npx", "-y", "mcp-remote", "http://127.0.0.1:8095/mcp", "--header", "Authorization:${AUTH_HEADER}", "--transport", "http-only"], "env": { "AUTH_HEADER": "Bearer demo-read-key" } } } } ``` The header goes through `AUTH_HEADER` because Windows splits an argument at the space in `Bearer demo-read-key`. Quit Claude Desktop from the tray and start it again after editing the file. Then ask, for example: * *"What AyMON documents can you see, and what is in them?"* * *"How is the tank doing?"* * *"Raise the setpoint to 90 and tell me when the level alarm shows."* * *"Set the tank level to 10."* — and watch it be refused * *"Is there anything in the log I should know about?"* An assistant caches the tool list when it connects. If you change the tools (`db\db_AI_mcp.js`) or an endpoint, restart the client as well, or it keeps calling with the old list. ## What you should see `try.bat`, shortened: ``` == tools/list on the READ endpoint (8095) - no write tools at all system_info, browse, find_points, read_points, get_alarms, get_events, query_history, tree, get_summary, get_metrics, get_connections, aggregate, read_logs, plant_status == tools/list on the WRITE endpoint (8096) - set_value ... and the project's set_setpoint ... read_logs, set_value, add_value, xor_value, msk_value, ask_value, api_call, ack_alarm, plant_status, set_setpoint == a wrong key HTTP 401 == system_info - only PLT/S00 is listed; AI/S00 has no mcp_pub {"server":"AyMON", ... "docs":[{"id":"PLT/S00","desc":"Tank plant: one tank ...","sta":"0x0303","sta_txt":"RUN"}], ... == read_points - VENDOR_NOTE is 'not found' for role 2, and acl_hidden says one was filtered {"doc":"PLT/S00","acl_hidden":1,"points":[{"id":"LEVEL", ... "val":"62.9","sta":"0x0343","sta_txt":"RUN new"}, ... {"id":"VENDOR_NOTE","error":"not found"}]} == plant_status - a tool the PROJECT declared, in db\db_AI_mcp.js {"level_pct":62.9,"level_state":"normal","setpoint_pct":60,"pump":"stopped","mode":"automatic","step":155} == set_value on the read endpoint - the tool does not exist there error -32601: Method not found == set_value SETPOINT = 150 - OKY, and the plant's own limit makes it 90 {"op":"set_value","adr":"doc('PLT/S00')/id('SETPOINT')/@val","matched":1,"res":"OKY"} {"doc":"PLT/S00","points":[{"id":"SETPOINT", ... "val":"90", ...}]} == set_value LEVEL = 10 - a measurement; role 3 may not write it isError {"op":"set_value", ... "res":"refused","reason":"xdb_acl refused the write ..."} == one address, two elements: SETPOINT written, LEVEL refused = PARTIAL {"op":"set_value","adr":"doc('PLT/S00')//var[@val_unt='%']/@val","matched":2,"res":"PARTIAL", ...} == set_setpoint 150 - the project tool refuses instead of clamping {"ok":0,"error":"setpoint 150 is outside 20..90 %; nothing was written"} ``` ## The files | File | What is in it | |---|---| | `db\AyXDB.xml` | the **root document**: names the engine, loads the two children | | `db\db_PLT.xml` | `PLT/S00`, the plant — **published** to MCP, with its `xdb_acl` | | `db\db_PLT.js` | the simulator and the level controller; nothing in it knows about MCP | | `db\db_AI.xml` | `AI/S00`, the two MCP endpoints — **not** published | | `db\db_AI_mcp.js` | the project's own tools, `plant_status` and `set_setpoint` | | `db\db_AI_api.js` | the `api` scope: `api.help()`, the text `system_info` hands the assistant | | `try.ps1` / `try.bat` | the raw JSON-RPC calls, no assistant needed | | `.mcp.json` | both endpoints, for Claude Code | ## What it teaches * **Publishing is per document.** `mcp_pub="1"` on a document's root puts it on the whitelist; `mcp_desc` is the sentence the assistant reads to decide whether it wants that document. `AI/S00` has neither, so the assistant cannot see even the configuration of the door it came in through. * **An endpoint has a role, and the role is the whole answer.** `pro_mcp_role` says who the endpoint *is*; `xdb_acl` on each element says what that role may do. MCP does not decide anything of its own — the same element answers a web user, a telnet session or a replication link the same way. The write endpoint's tool list says `set_value`, and it still cannot write `LEVEL`. * **`ro` removes the tools, not just the permission.** On the read endpoint the write tools are not in the list; calling one anyway is `Method not found`. An assistant cannot try what it has not been shown. * **Hidden looks like absent — but never silently.** A point the role may not read answers `not found`, not `denied`: that it exists is half of what the ACL keeps. Every answer that left something out carries `acl_hidden: N`, so a filtered list is never mistaken for a complete one. * **Four answers to a write, and they mean different things.** `OKY` — done. `refused` — the role may not; nothing was written. `PARTIAL` — some of the matched elements were written and some were refused; *do not repeat the call*, or `add_value` would add twice. `dry_run: true` — nothing written, here is what would be. * **The plant's own limits hold for an AI too.** `SETPOINT` has `val_min="20" val_max="90"` and `ATR.lmt`, so 150 lands as 90. That is configuration on the element, not a check in the MCP layer, and it applies to every writer. * **A project adds its own tools in script.** `pro_mcp_tools="mcp.tools"` names a function that returns tool declarations; the engine merges them into the list. `plant_status` answers "how is the plant" in one call; `set_setpoint` refuses an out-of-range value *with a reason*, where `set_value` clamps it. Whatever a handler returns is the answer. A handler runs with the endpoint's role, not as the engine, so it is no way around `xdb_acl`. `write: true` lists a tool on `rw` endpoints only. * **Tell the assistant what it is looking at.** If the `api` scope has a `help()` function, `system_info` hands its text to the assistant as `api_help` — the place to say in plain words what this installation is. It is optional; an engine older than 4.3.4.449 logs, on every `system_info`, that it looked for `api.help` and did not find it. The `api` scope is also what `api_call` may reach on a `rw` endpoint, so keep it to what every caller may run. * **The log is opt-in.** `pro_mcp_log="4"` turns on `read_logs` up to WARN. The log is free text and is not covered by `mcp_pub`, so it is a separate decision; without the attribute the tool does not exist. ## Who may do what | | Read endpoint, role 2 | Write endpoint, role 3 | |---|---|---| | see `PLT/S00` | yes | yes | | see `AI/S00` | no — not published | no — not published | | read `LEVEL`, `PUMP`, `MAINT` | yes | yes | | read `VENDOR_NOTE` | not found | not found | | write `SETPOINT` | no write tools | yes, clamped to 20..90 | | write `LEVEL`, `PUMP`, `MAINT` | no write tools | refused | The ACLs are in `db\db_PLT.xml`: the root carries `0xFF111111` (read only below Engineer) and every element without its own ACL inherits it; `SETPOINT` carries `0xFF333111` (Operator and up may write); `VENDOR_NOTE` carries `0xF0000000` (Admin only). One nibble per role, role N in bits 4N..4N+3, written high role first — `1` read, `2` write, `4` create, `8` delete. A refused write is in the engine log, at ALERT, in the document of the element it tried to change, and a wrong key at FAIL in `AI/S00`'s log — some in Turkish, as the engine's own messages are: ``` AI/S00:CN_MCP:PRO.mcp(1).auth: yetkisiz istek reddedildi "unauthorised request refused" PLT/S00:LEVEL.val: xdb_acl yazma reddetti (rol 3) "write refused, role 3" PLT/S00:PLT/S00: '//var[@val_unt='%']/@val' -- xdb_acl refused 1 of 2 matched nodes (role 3), the others were written ``` After `try.bat` these three kinds are the only error-level lines in the logs. ## Addresses The write tools take an address in either of two forms: | Form | Example | | |---|---|---| | topic | `PLT/S00/SETPOINT/val` | document / element id / attribute | | topic, many | `PLT/S00/var[**]/val` | every `` in the document | | XPath | `doc('PLT/S00')//var[@val_unt='%']/@val` | anything containing `doc(`, `@` or `//` | Try a bulk address with `dry_run: true` first; `pro_mcp_wmax="10"` on the write endpoint refuses any write that would touch more than ten elements. Addresses starting with `#` are refused outright — MCP cannot run free-form script. ## Known limits * **`find_points` finds nothing here.** It searches `` elements — the tools were shaped by field plants, where every point sits under a polled device. A simulated plant cannot use ``: a `` with no connection behind it is never raised to running, and its points do not run their `val_fnc` chain (no rounding, no alarm). So the plant is ``, `find_points` answers `total: 0`, and `plant_status` is how the assistant gets the picture instead. On a Modbus or MQTT plant (`PRJEDU_modbus`, `PRJEDU_mqtt`) the points are `` and `find_points` works. * **`get_alarms`, `get_events` and `query_history`** need an alarm list, an event list and a history database; this example has none of them, so they answer `address not configured` / `sqlcon not configured` (`pro_mcp_alm`, `pro_mcp_evt`, `pro_mcp_sqlcon` on the endpoint name them). * **One identity per key.** Everybody behind the same key is the same role. The engine can resolve a person per request (`pro_mcp_onuser` and an `X-AyMON-User-Token` header) — that is for a real installation with a login. * **No push.** The server answers when asked; it does not tell the assistant that an alarm has just appeared. ## Opening it to another machine Both endpoints are on `127.0.0.1` and `con_acl="127.0.0.1"` refuses everyone else at accept time. Before you change that: * [ ] replace both keys. `pro_mcp_key` **left empty turns authentication off altogether** — the engine says so in the log, and then anyone who can reach the port is in * [ ] put TLS in front: `[CON.tls,PRO.mcp]` with `con_ssl_*`, as `PRJEDU_telnet` does for its shell * [ ] widen `con_acl` only as far as needed — `10.40.1.*`, not blank * [ ] give the endpoint the **lowest** `pro_mcp_role` that does the job, and keep `pro_mcp_acc="ro"` unless writing is the point * [ ] put `xdb_acl` on what matters; a role means nothing against elements that carry none * [ ] publish only the documents the assistant needs — never one that holds users or passwords * [ ] a licence that carries `mcp` — see below ## Licence MCP is **not in FREE**, and not in the education licence in `bin\lic` either — the start-up log says so: `features off: cluster,rep,rpc,mcp`. It runs here anyway because an unlicensed feature is a **local** feature: on a loopback address it works with no licence at all. Move `con_host` to a LAN address and the connection is refused until a licence carries `mcp`. Four connections (two for the page) and three documents are well inside FREE's numbers. ============================================================================== PAGE: PRJEDU_modbus — Modbus TCP, both ends, no hardware URL: http://docs.aymon.ca/PRJEDU_modbus/ ============================================================================== # PRJEDU_modbus — Modbus TCP, both ends, no hardware One engine, one command, real Modbus frames on a loopback socket: ```bat run.bat ``` On Linux the same example starts with `./run.sh` (the engine in `../bin/linux-x64`; see "On Linux" in the top README). Then open **http://127.0.0.1:8103/** — the monitoring side's page: the tank with its limits, the three bits of the status register as lamps, a trend, and every point as the wire sent it (`pro`) next to what it means (`val`) and its alarm state. Two documents in the same process talk to each other: | Document | Role | What it is | |---|---|---| | `PLC/S00` | Modbus **slave** | the plant. Listens on `127.0.0.1:1502`, answers for unit 1, serves four holding registers that a timer keeps moving. | | `MON/S00` | Modbus **master** | the monitoring side. Polls those registers once a second and turns them into engineering values with limits and alarms. | Read `db\db_PLC.xml` first if you are putting AyMON **where a device used to be**, `db\db_MON.xml` first if you are **reading a device that already exists**. They are the same four levels seen from opposite ends. ## What it teaches * the four levels every AyMON project has — ``, ``, ``, `` * that the **block** is the unit of communication and the **point** is the unit of meaning: 50 registers in one `` is one frame, 50 separate `` is 50 frames for the same data * **what `pro_len` counts**: elements of the block's own `pro_typ`. With `pro_typ="0xD412"` (a buffer of 16-bit units) `pro_len="4"` is four *registers*; with `pro_typ="0xD411"` (bytes) the same block is `pro_len="8"` and every `` address below becomes a byte offset too * **bit addressing**: `pro_adr="2.1"` is register 2, bit 1 — the same unit as `pro_len`, so absolute bit 2×16+1 of the block. Three points in this example take three bits out of one status register with no script * that `pro` is the **wire** value and `val` is the **meaning**; nothing crosses between them until `pro_fnc="[VAL.put]"` says so * that scaling, rounding, clamping and alarms are **configuration** (`VAL.cnv`, `ATR.rou`, `ATR.lmt`, `ALM.onlvl`), not script * that plain Modbus is not a licensed feature — all of this runs unlicensed ## The page `MON/S00` serves it itself, with two more connections: `CN_WEB` (HTTP, the files in `www\`) and `CN_LIVE` (MQTT inside a WebSocket on the same port, `/mqtt`). The page subscribes to `MON/S00/+/val`, `/pro` and `/sta`; the engine answers with the current value of every `` at once and then sends each change. It only reads: `xdb_acl="0xFFFFFFF1"` on the document gives a page that did not log in (a Guest) read and nothing else. Both listen on `127.0.0.1` only (`con_acl`). `www\kit.js` is the small toolkit the example pages share (connect, subscribe, what a `sta` word says, a trend chart); `wwwpp.js` is this page. ## The register map Unit 1, holding registers, served by `PLC/S00`: | Register | Meaning | Raw | Engineering | |---|---|---|---| | 0 | tank level | 0..1000 | 0..100 % (×0.1) | | 1 | temperature | 160..340 | 16..34 C (×0.1) | | 2 | status word | bit0 pump, bit1 valve, bit2 high temp | three `` | | 3 | pump hours | counts up | h | Scaled integers, because a Modbus register is 16 bits and has no decimal point. Turning them back into engineering values is `val_cnv_dlt` in `db\db_MON.xml` — one attribute per point. ## What the log looks like ``` PLC/S00:PLC: Modbus TCP slave on 127.0.0.1:1502, unit 1, holding registers 0..3 MON/S00:MON: polling unit 1 at 127.0.0.1:1502, holding registers 0..3, once a second MON/S00:MON: pump STARTED MON/S00:MON: level 79 %, temp 33.3 C, pump on, valve shut, hot 1, level sta 0x0303 ``` Turn `xdb_log` up to `7` on the `` root **and** on the ``, `` and `` you care about, and the frames themselves are in the log — both sides of them: ``` MON/S00:TANK_BLK:PRO.modbus(1).M-Q[...]: 00 04 00 00 00 06 01 03 00 00 00 04 PLC/S00:CN_PLC:PRO.modbus(1).SLV-I[...]: 00 04 00 00 00 06 01 03 00 00 00 04 PLC/S00:modbus.MAS-Q[...]: 00 04 00 00 00 0B 01 03 08 02 63 01 34 00 01 00 05 MON/S00:TANK_BLK:PRO.modbus(1).M-R[...]: 00 04 00 00 00 0B 01 03 08 02 63 01 34 00 01 00 05 ``` The file threshold and the element detail are two different settings: the root's `xdb_log` decides what the file accepts, each element's `xdb_log` decides how much that element produces. Raising only one of them changes nothing. ## Pointing it at a real device Change `con_host` and `con_port` on `` in `db\db_MON.xml`, and `pro_adr` to its unit id. Then set `sta="0x0000"` on `` in `db\AyXDB.xml` so the simulated plant stays out of the way. Nothing else in `db_MON.xml` is about the simulator. The other direction works as well: leave `PLC/S00` running and point any Modbus master (a SCADA package, `mbpoll`, a PLC) at `127.0.0.1:1502` — change `con_host` to `0.0.0.0` first if it is on another machine. ============================================================================== PAGE: PRJEDU_mqtt — MQTT, both ends, nothing to install URL: http://docs.aymon.ca/PRJEDU_mqtt/ ============================================================================== # PRJEDU_mqtt — MQTT, both ends, nothing to install One engine, one command, real MQTT packets on a loopback socket: ```bat run.bat ``` On Linux the same example starts with `./run.sh` (the engine in `../bin/linux-x64`; see "On Linux" in the top README). Then open **http://127.0.0.1:8104/** — the monitoring side's page: the four points with their alarm state, a trend, and a set point you send to the field and watch arrive (the page shows each step with its time). Two documents in the same process talk to each other through a broker: | Document | MQTT role | What it is | |---|---|---| | `FLD/S00` | **broker** | the field: a small tank, and AyMON as the broker on `127.0.0.1:1884`. Every value that changes is published as `FLD/S00//`. | | `MON/S00` | **client** | the monitoring side: connects to that broker, subscribes, turns messages into points with limits and alarms, and writes a set point back. | Read `db\db_FLD.xml` first if AyMON is to **be** the broker other things connect to. Read `db\db_MON.xml` first if AyMON is to connect to a broker that **already exists**: mosquitto, a cloud service, another AyMON. The port is 1884 rather than 1883 so the example never collides with a mosquitto already running on this machine. ## What it teaches * **the broker publishes its own document**. `FLD/S00` has no list of topics. Every attribute that changes is on the wire as `//`. Adding a point is adding a ``. * **a write is a PUBLISH to `.../set`**. `FLD/S00/SETPOINT/val/set` writes `id('SETPOINT')/@val` through that element's own `val_fnc`: rounded, clamped (send `250`, get `100`) and logged. The stored value is then published like any other change, so the writer learns what was kept. * **what stays home**: an attribute whose type carries the `0x4000` (LOCAL) bit is never published. `STEPS` changes every second and is never seen outside, and neither is the timer's own `evt`. * **retain, the AyMON way**. The broker keeps no store of retained messages; the document is the store. With `pro_retain="1"` a new subscription is answered at once with the current values (`pro_retain_adr` picks the elements, `pro_retain_atr` the attributes), so a client that connects late does not wait for the next change. * **the client side is the same four levels as Modbus**, with a different meaning at each level: * `` is the broker connection * `` is one publisher, named by its topic prefix * `` is one subscription; its `pro_adr` is a topic filter (`FLD/S00/+/val`) * `` is one topic, one point * **no timer on the block**. Modbus asks, so it needs a clock. MQTT is told: the point moves when the message arrives. * **`pro` is the wire, `val` is the meaning**. The payload is text (`"47.3"`). `pro_typ` says how to read it, and `pro_fnc="[VAL.put]"` copies it into `val`. Leave `VAL.put` out and every message arrives while `val` never moves. * **limits and alarms belong to the monitoring side**. The field publishes plain numbers; `MON/S00` decides what "too high" means. * **writing back from the client**: `pro_fnc="[PRO.out,VAL.put]"` on the `SETPOINT` point. A write to its `pro` goes out as `FLD/S00/SETPOINT/val/set`. The value that comes back from the field is protocol input, and `PRO.out` does not send protocol input out again, so there is no echo. * **script can publish anything**: `MQTT.publish("CN_MQ", "MON/S00/summary", JSON.stringify(...), 0, 0)`. The broker relays it to every subscriber, because it is also an ordinary broker, whether or not the topic is an element. ## What the log looks like ``` FLD/S00:FLD: MQTT broker on 127.0.0.1:1884, publishing FLD/S00// FLD/S00:CN_BRK:PRO.mqtt(1).retain: 'FLD/S00/+/val' için 4 değer yayınlandı MON/S00:VALUES.mqtt(0).connect: bi: topic('FLD/S00/+/val') MON/S00:MON: level 51.5 % (set point 50), temp 18 C, pump on, level sta 0x0303 MON/S00:MON: asking the field for set point 40 % FLD/S00:FLD: SETPOINT 50 -> 40 % (written over MQTT) ... MON/S00:MON: level WARNING (sta 0x0303 -> 0x0347) MON/S00:MON: level ALARM (sta 0x0347 -> 0x034B) MON/S00:MON: level WARNING (sta 0x034B -> 0x0347) MON/S00:MON: level normal (sta 0x0347 -> 0x0343) ``` A client that goes away without saying DISCONNECT — a `mosquitto_sub` closed with its window, say — leaves one line behind: `CN_BRK:PRO.mqtt(n).S-Q[...|ERR_CONNECT_WANTCLOSE]`. The engine drops that session, and there is nothing else to do. The `retain` line counts four values: `LEVEL`, `TEMP`, `PUMP` and `SETPOINT`. `STEPS` is local, so it is not among them. The line itself is one of the engine's own messages and is in Turkish ("4 values published for ..."). The set point moves every 30 seconds, through 40, 70 and 95 %. At 95 the level climbs past both high limits, so the alarm path runs without anyone touching anything. ## The page `MON/S00` serves it with two more connections: `CN_WEB` (HTTP, `www\`) and `CN_LIVE` — MQTT again, but the other way round: `CN_MQ` is this document as a **client** of the field's broker, `CN_LIVE` is this document as a **broker** for the page, inside a WebSocket on the page's port (`/mqtt`). The page writes exactly one thing, `MON/S00/SETPOINT/pro/set`: writing a point's `pro` runs its `PRO.out`, which publishes `FLD/S00/SETPOINT/val/set` to the field. `xdb_acl="0xFFFFFFF1"` on the document lets a page that did not log in (a Guest) read everything and write nothing; `SETPOINT`'s own `xdb_acl="0xFFFFFFF3"` adds that one write. Measured: the page's set point reached `MON/S00` in 70 ms and the field's pump reacted 0.6 s after the click; a write to `TEMP` from the page was not taken. ## Watching it with mosquitto The broker speaks plain MQTT 3.1.1, so any client works. With the mosquitto command-line tools (`mosquitto_sub`, `mosquitto_pub`) in another window: ```bat mosquitto_sub -h 127.0.0.1 -p 1884 -t "FLD/#" -t "MON/#" -v ``` You get the current values at once (that is the retain replay), then every change, the set-point writes, and `MON/S00/summary` as one JSON message every ten seconds. Write a set point yourself: ```bat mosquitto_pub -h 127.0.0.1 -p 1884 -t "FLD/S00/SETPOINT/val/set" -m 85 ``` Two outside clients can also talk to each other through the engine on any topic at all, `mosquitto_pub -t hello/world -m hi` included. No `-i` needed: a client that sends an empty client id is given one by the broker (`auto-CN_BRK-1`, `auto-CN_BRK-2`, …). This needs engine **4.3.4.442** or later. Earlier engines refuse such a client. ## Using mosquitto (or any broker) instead To make `MON/S00` a client of a real broker: 1. In `db\db_MON.xml`, set `con_host` / `con_port` on `` to the broker, for example `127.0.0.1` / `1883`. Add `pro_usr` / `pro_pwd` if the broker wants a login. 2. In `db\AyXDB.xml`, set `sta="0x0000"` on `` so the engine's own broker and tank stay out of the way. 3. Feed the broker the topics MON expects: ```bat mosquitto_pub -h 127.0.0.1 -p 1883 -t "FLD/S00/LEVEL/val" -m 84.5 mosquitto_pub -h 127.0.0.1 -p 1883 -t "FLD/S00/LEVEL/val" -m 93 ``` The second one puts `LEVEL` into ALARM on the monitoring side. Nothing in `db_MON.xml` knows it is talking to a different broker. Or keep both documents and point a real device at `FLD/S00`: anything that publishes `FLD/S00//val` feeds the tank's elements. Add `/set` to the topic to write to them. ## Opening it to other machines Everything listens on `127.0.0.1`, and `con_acl="127.0.0.1"` refuses anyone else at accept. There is no login: any client that can connect may read every value and write every one that has no `xdb_acl`. Before you change `con_host`: * widen `con_acl` only as far as needed (`192.168.1.*`) * give the broker a login. `pro_oncon="main.oncon(exc,cid,usr,pwd);"` is the simplest form: a script that returns 0 to accept. * use `[CON.tls,PRO.mqtt]` with a certificate for anything that crosses a network you do not own (PRJEDU_web shows the certificate attributes) ## Licence Four connections (two for the page) and three documents: well inside FREE. MQTT is not a licensed feature. ============================================================================== PAGE: PRJEDU_net — the engine writes mail and files, nothing to install URL: http://docs.aymon.ca/PRJEDU_net/ ============================================================================== # PRJEDU_net — the engine writes mail and files, nothing to install The plant talks to the office world. **Mail**: a tank's level alarm writes an e-mail when it comes and another when it is gone. **Files**: a shift report is written, zipped and uploaded to an FTP server. One engine, one command, real SMTP and FTP on loopback sockets: ```bat run.bat ``` On Linux the same example starts with `./run.sh` (the engine in `../bin/linux-x64`; see "On Linux" in the top README). Then open **http://127.0.0.1:8114/** — the tank, three buttons (*Overfill once*, *Send a test mail*, *Make and upload a shift report*), what the engine sent with the server's answer to each mail and each transfer, the mails as the mail server received them, and the files the file server holds. | Document | Role | What it is | |---|---|---| | `PLT/S00` | the **plant** | a tank with a level alarm, a mail connection (`CN_MAIL`, `PRO.smtp`) and the page | | `MBX/S00` | the **mail server** | a small SMTP server **written in script** on `127.0.0.1:2525`: it takes every message and hands it to the page | | `FTP/S00` | the **file server** | a small FTP server **written in script** on `127.0.0.1:2121`: it takes every upload, keeps its name and size, and lists them | Read `db\db_PLT.xml` and `db\db_PLT.js` if you came for mail or file transfer — that is what you would write for a real plant. Read `db\db_MBX.js` and `db\db_FTP.js` if you ever have to write a server for a line protocol. Needs engine **4.3.4.568** or later and its module **`AyLIB_net`** (in `bin`; `AyLIB_net.dll`, `libAyLIB_net.so` on Linux). The module is attached the first time a `` names `PRO.smtp` or `PRO.ftp`, or a script calls `FILE.*` — no `xdb_lib` needed. ## A mail is a write to a point ```xml ``` ```js evt.SetValue("id('MAIL')/@pro", JSON.stringify({ id: "M7", to: "operator@example.org", subject: "ALARM: tank level 91.6 %", body: "The tank level reached ..." })); ``` * **The connection is made for the message.** Nothing is dialled at start; the first mail connects, says EHLO and sends; later mails use the open session; after `con_idletmo` quiet seconds the engine says QUIT. * **One attribute, both directions.** The message goes out through `PRO.out` and is never stored. The server's answer comes back **into the same `pro`** as protocol input, passes `PRO.out` and runs `main.mail_done`: ```json {"seq":2,"id":"M2","code":250,"ok":1,"text":"250 2.0.0 Ok: queued as 2"} ``` `code` 250 = taken, 4xx = not now, 5xx = no, **0 = it never came to an answer** (no connection, refused before sending, the licence's allowance). `ok` = recipients the server accepted. The handler's first line asks whether the value really came from the protocol — the chain also runs once at start. This is the rule every protocol's `pro` follows: a value written to a Modbus register goes out the same way and the device's answer comes back the same way. * **Sending waits for the server**, so it never runs on the thread that runs the tank. Whoever wants a mail writes `NOTE` (`{subject, body}`); `NOTE`'s `val_fnc` runs queued on `TH_MAIL` (`val_typ="0x3401"`) and writes the point. * **When a mail goes out** is the project's own decision, in script: `main.on_alarm` is the `sta_fnc` of `LEVEL` and asks for a mail when the level reaches the alarm (90 %) and when it is back to normal after one. The warning (80 %) writes no mail — it is on the page. ## A file transfer is a write too ```xml ``` ```js evt.SetValue("id('FTP')/@pro", JSON.stringify({ id: "F3", cmd: "put", src: "%PRJ%/out/shift_20261005_215222.zip", dst: "shift_20261005_215222.zip", tmp: 1, move: "%PRJ%/out/sent" })); ``` The same rules as mail: the job is the JSON written to a point, the server's answer comes back **into the same point**: ```json {"seq":3,"id":"F3","cmd":"put","code":226,"ok":1,"text":"226 Transfer complete, 327 bytes","bytes":327} ``` * **`put` with `tmp`** uploads as `.part` and renames it when it is complete — a reader of the server's directory never sees half a file; **`move`** puts the local file into `out/sent` afterwards. * **`list`** answers the directory in the reply (`"list":[{name, size, ..}]`). `main.ftp_done` asks for it after every upload — on the OTHER point, `FTP_LS`, with the reply's "from the protocol" mark taken off first: a write made inside a reply's chain would otherwise not be sent. * **Who does the work.** The page only writes `REPORT` (the time of the press). The tank's own timer sees it and makes the report: the samples of the last minutes into a CSV (`File.open` / `File.puts`), `FILE.mkdir` and `FILE.zip` (module `AyLIB_net`). Those two are for an Engineer and above, and a script started from the page runs with the page's role (Guest); the engine's own timer runs as the engine. * The upload is handed to `TH_MAIL` through `FTPJOB`, like a mail: a transfer waits for the server and never runs on the tank's thread. * Other jobs: `get`, `stat`, `del`, `mkdir`, `rmdir`, `rename` — the field note *File Transfer (FTP)* of the licensed product lists them. ## Why the mail server is script So that the example needs no mail account and nothing installed. It is built out of the same pieces as `PRJEDU_snmp`'s agent, plus one: | | | |---|---| | `PRO.raw` | a connection with no protocol of its own | | `pro_onaccept` | called **once per accepted connection, before anything is read**. SMTP's server speaks first (`220 ...`) and the client waits for it — without this hook (4.3.4.565) a script server could only answer, never begin | | `pro_onparse` | called with what arrived: the script cuts it into lines (`CR LF`), answers each command and collects the message between `DATA` and the line holding one dot. What is not a whole line yet waits in `exc.ses.mem` for the next arrival | | `con_role="3"` | an accepted connection is a Guest (role 0), and its scripts run with that role; the server hands each message to the page's document, whose `INBOX` only an Operator (3) may write | It answers EHLO, HELO, MAIL, RCPT, DATA, RSET, NOOP and QUIT, decodes the base64 body and the encoded subject, and keeps the last ten messages. It does not relay, authenticate or encrypt — it is a mail **sink**, for this example only. ## Why the file server is script, and what FTP needs from it FTP uses **two kinds of connection**: commands and answers on one, and every listing and every file on a connection of its own, which the client opens to the port the server named in its answer to `EPSV`. In stream mode **a file ends where its connection ends** — so the server has to hear the close. That is the third hook (4.3.4.568): | | | |---|---| | `pro_onaccept` | a connection arrived. Just after an `EPSV` it is the data connection: taken, nothing said. Otherwise a client: `220 ...` | | `pro_onparse` | a command line, or the bytes of a file (counted; the first four kept, to show a ZIP is a ZIP) | | `pro_onclose` | a connection ended. The data connection of an upload: the file is complete, answer `226` on the command connection | A listing goes the other way: the server writes it on the data connection and closes it itself — the client reads until the close. **One listener for both.** `EPSV` names the command port itself (2121), so a client's command session and its data session are two sessions of the same `` and run on the same thread: the script keeps its state in its own variables and never has two threads touching them. A real server names a range of ports instead. It serves one client at a time, keeps names and sizes — not the files — and does no TLS: a **sink**, like the mail server. ## The licence: 20 operations a day Mail and file transfer are the licence feature **`net`**. Without it — FREE, and the education licence of these examples — **20 operations a day are free**, mails and FTP jobs together; the next one is not refused by address but answered `code 0` ("not sent"), and one `LIC:` line goes to the root log. The count is kept in memory and starts again with the engine. The start-up log says it: ``` XDB:LIC: net is not licensed: mail and file transfer (PRO.smtp / ftp / sftp) work at any address and are COUNTED, 20 operations a day ``` One *Overfill* costs two mails, *Send a test mail* one, a shift report two FTP jobs (the upload and the listing). ## What the log looks like ``` XDB:AyLIB: 'AyLIB_net' attached (module abi 1.3, core 1.3) MBX/S00:MBX: SMTP mail sink on 127.0.0.1:2525 (PRO.raw + pro_onaccept + pro_onparse) PLT/S00:PLT: tank + mail to operator@example.org through CN_MAIL; open http://127.0.0.1:8114/ PLT/S00:PLT: overfill started at 57 % PLT/S00:PLT: level WARNING (82 %) PLT/S00:PLT: level ALARM (92 %) PLT/S00:DV_MAIL:PRO.smtp(1).connect: ready -- 127.0.0.1, PLAIN TEXT (pro_smtp_tls="none"), no AUTH MBX/S00:MBX: message 1 from plant@example.org to operator@example.org (503 bytes): ALARM: tank level 92 % PLT/S00:PLT: mail M1 sent to 1 recipient(s): ALARM: tank level 92 % PLT/S00:PLT: level WARNING (89.5 %) PLT/S00:PLT: level normal (79.5 %) MBX/S00:MBX: message 2 from plant@example.org to operator@example.org (477 bytes): Cleared: tank level back to normal PLT/S00:PLT: mail M2 sent to 1 recipient(s): Cleared: tank level back to normal PLT/S00:PLT: shift report shift_20261005_215216.zip - 704 bytes of samples in 305 bytes PLT/S00:DV_FILES:PRO.ftp(1).connect: ready -- 127.0.0.1, PLAIN TEXT (pro_ftp_tls="none"), user anonymous FTP/S00:FTP: received shift_20261005_215216.zip.part (305 bytes, a ZIP archive) PLT/S00:PLT: uploaded F1 (305 bytes): 226 Transfer complete, 305 bytes PLT/S00:PLT: the file server holds 1 file(s) ``` Raise `xdb_log` to `7` on `CN_MAIL` or `CN_FILES` and every command and answer is in the log (`C: ..` / `S: ..`); a password, when there is one, shows as `***`. ## Sending to a real mailbox In `db\db_PLT.xml`, on ``: | Field | Set it to | |---|---| | `con_host` / `con_port` | your provider's submission server, e.g. `smtp.example.org` / `587` | | `pro_smtp_tls` | **remove it**: the engine then insists on STARTTLS and refuses a server that does not offer it. Port 465 instead: `xdb_fnc="[CON.tls,PRO.smtp]"` | | `pro_usr` / `pro_pwd` | the account; many providers want an **app password**, not your own | | `pro_smtp_from` | an address the account may send as | | `con_ssl_cacert` | the server's CA chain as a PEM file — **on Windows the system certificate store is not read** | and set `MAIL_TO` to your address. Leave `MBX/S00` running or set `sta="0x0000"` on its `` in `db\AyXDB.xml`. If nothing arrives, the reply on the page says why: | Reply | Meaning | |---|---| | `code 0`, `no connection to ` | the server cannot be reached or the login failed — the `connect:` line above it in `PLT.log` has the reason | | `code 0` … `does not offer STARTTLS` | port 465 wants `CON.tls`; a plain relay on your own network wants `pro_smtp_tls="none"` | | `code 0` … `certificate verification failed` | `con_ssl_cacert` missing or not the server's CA | | `code 535` | account or password | | `code 0` … `daily allowance` | the 20 mails of the day are used — see *The licence* above | ## Uploading to a real FTP server In `db\db_PLT.xml`, on ``: | Field | Set it to | |---|---| | `con_host` / `con_port` | the server, port `21` | | `pro_ftp_tls` | **remove it**: the engine then insists on `AUTH TLS` (FTPS) and refuses a server that does not offer it; commands AND data are encrypted. Implicit FTPS on port 990: `xdb_fnc="[CON.tls,PRO.ftp]"` | | `pro_usr` / `pro_pwd` | the account (none = anonymous) | | `pro_ftp_dir` | the remote directory the jobs may use; names cannot leave it | | `con_ssl_cacert` | the server's CA chain as a PEM file | Leave `FTP/S00` running or set `sta="0x0000"` on its ``. For SFTP (SSH) the same jobs go through `[CON.ssh,PRO.sftp]`, with the server's key fingerprint in `con_ssh_hostkey` — only the `` changes. ## Opening it to other machines Every listener — the page, the mail sink, the file sink — is on `127.0.0.1`, and `con_acl="127.0.0.1"` refuses everyone else at accept. The sinks check nothing: do not put them on a network. ## Licence Six connections (mail and file connections, the two sinks, two for the page), four documents, two devices: inside FREE. Mail and file transfer are counted as above. ============================================================================== PAGE: PRJEDU_odbc — one plant, any SQL database, chosen by a connection string URL: http://docs.aymon.ca/PRJEDU_odbc/ ============================================================================== # PRJEDU_odbc — one plant, any SQL database, chosen by a connection string ```bat run.bat ``` On Linux the same example starts with `./run.sh` (the engine in `../bin/linux-x64`; see "On Linux" in the top README). Then open **http://127.0.0.1:8107/** — the history's page: the reading written a second ago, what the database says every ten seconds, the last five rows, and a box that counts the rows of any tag you type — with the tag **bound**, so typing `x'; DROP TABLE reading; --` finds the rows that carry that text and the table stays. Out of the box this runs at once: the database is the embedded SQLite and the file `log\plant.db` appears by itself. Stop it with CTRL-C and start it again — the readings are still there. This is the twin of [PRJEDU_sqlite](../PRJEDU_sqlite). The plant, the timers and the SQL calls are the same. What is new is **where the database comes from**: the `` has an empty `con_host`, and the connection string is picked from a short list in the root document when the history document starts. Move the plant to SQL Server or PostgreSQL by swapping two ids — the script does not change. ## What it teaches * **the connection string is the whole choice** — `DRIVER=SQLite3i;...` reaches the embedded SQLite, anything no driver claims goes to ODBC. Both drivers are plug-ins (`AyPRO_sqlite.dll`, `AyPRO_odbc.dll`) that attach themselves on first use; neither is named anywhere in the XML * **connection strings kept in one place** — `` in `AyXDB.xml`, read by every document that needs the database, written into `CN_SQL/@con_host` in `xdb_init`, before the connection exists * **path macros are yours to expand** — `con_host` is handed to the driver exactly as written, so the script turns `%PRJ%` into a path with `xdb.normalizePath()` * **one script, three databases** — every statement is written once, in SQL that SQLite, SQL Server and PostgreSQL all accept; the single exception (does the table exist?) is chosen by the entry's `eng` * **a write that did not happen is not counted** — `SQL.execute(rs)` answers 1 when the row is stored and 0 when the database refused it (the reason is in the log); the sequence number moves only on 1. This needs an engine of 4.3.4.440 or later — older ones answered 0 in both cases * the lessons of PRJEDU_sqlite still hold: bound parameters are never SQL (the tag `x'; DROP TABLE reading; --` is stored as text, on every database), `SQL.qry` turns rows into elements, an SQL NULL is `undefined` ## Choosing the database `db\AyXDB.xml`: ```xml ``` The entry whose id is exactly **`ConnStr`** is the one in use; the others are parked. To use SQL Server, rename `ConnStr` to `ConnStrSQLITE` and `ConnStrMSSQL` to `ConnStr`. Then create the empty database the string names: | Database | Create `aymon_edu` once | |---|---| | SQL Server (Express is enough) | `sqlcmd -S .\SQLEXPRESS -E -Q "CREATE DATABASE aymon_edu"` | | PostgreSQL | `createdb -U postgres aymon_edu` — and put your user and password in the string | The script creates the table itself. For PostgreSQL, use the driver name *ODBC Data Sources (64-bit)* shows on your machine; installers name it differently (`PostgreSQL Unicode(x64)`, `PostgreSQL ODBC Driver(UNICODE)`). A password in a connection string is written to the log as `PWD=***`, but it is in `AyXDB.xml` in clear text — that file is as secret as the password. Windows authentication (`Trusted_Connection=Yes`) keeps it out altogether. ## Why the string is set in `xdb_init` A document starts in a fixed order: ```