From 012223791ea71de74dcdd43d31faecc15bcc2169 Mon Sep 17 00:00:00 2001 From: teo1747 Date: Sat, 31 Oct 2026 01:17:57 +0100 Subject: [PATCH 1/3] feat(workq): optional per-queue statistics (KRN-WQ-011) CONFIG_EMB_WORKQ_STATS (off by default) completes KRN-WQ-011. emb_workq_get_stats() reports, per queue, the worst submit-to-start latency, the longest handler with its function and the deepest backlog. emb_workq_reset_stats() restarts the maxima. Both are thread and ISR safe, O(1), with API IDs 143 and 144. Times are kernel-clock ticks. Each work item gains a submit timestamp. A resubmission while the handler runs starts the latency of the run it queues, and flush markers are not counted. native-timeslice now also builds the option and runs the new conformance case, which was also checked under QEMU Cortex-M4 with the option set. A65 records the contract. Signed-off-by: teo1747 --- CHANGELOG.md | 2 + CMakePresets.json | 2 +- configs/native-timeslice.conf | 2 + docs/STATUS.md | 2 +- docs/requirements/KRN-WQ.md | 4 +- docs/specs/AMENDMENTS-M2.md | 1 + .../SPEC-006-notifications-and-work-queues.md | 2 +- docs/user/configuration.md | 3 +- docs/user/tasks.md | 2 +- include/emb/api_ids.h | 2 + include/emb/work.h | 52 +++++++++++ kernel/Kconfig | 10 +++ kernel/include/embk/work.h | 10 +++ kernel/work/workq.c | 88 +++++++++++++++++++ tests/conformance/test_workq.c | 34 ++++++- tools/fault/api-ids.json | 12 +++ tools/fault/api-registry.json | 12 +++ 17 files changed, 232 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 123bbdc..251be76 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,8 @@ User-visible changes are recorded here. There are no released versions or releas ### Added +- Work-queue statistics (`CONFIG_EMB_WORKQ_STATS`, off by default; KRN-WQ-011, A65): `emb_workq_get_stats()` reports the worst submit-to-start latency, the longest handler with its function and the deepest backlog per queue, in ticks; `emb_workq_reset_stats()` restarts them. + - Static definitions for the other object types (A64): `EMB_SEM_DEFINE`, `EMB_MUTEX_DEFINE`, `EMB_EVENT_DEFINE`, `EMB_COND_DEFINE`, `EMB_MSGQ_DEFINE`, `EMB_WORKQ_DEFINE` and, for interrupt-context callbacks, `EMB_TIMER_DEFINE` declare storage (with the ring or stack) and the handle and construct the object in `emb_kernel_init()`. - Round-robin time slicing (`CONFIG_EMB_TIME_SLICING`, off by default): a thread at or below `CONFIG_EMB_TIMESLICE_PRIORITY` that runs for `CONFIG_EMB_TIMESLICE_TICKS` without blocking moves behind its READY equal-priority peers; without a peer its quantum is renewed (KRN-SCH-007, 026, 037; A63). The quantum is a deadline in the tickless computation and a comparison in the periodic tick. The reference model implements it, the new `native-timeslice` preset checks it through the differential bridge, and `native-timeslice-tickless` runs the conformance cases with the tickless clock. List scheduler only. diff --git a/CMakePresets.json b/CMakePresets.json index ec290c8..1ac0360 100644 --- a/CMakePresets.json +++ b/CMakePresets.json @@ -39,7 +39,7 @@ }, { "name": "native-timeslice", - "displayName": "native, GCC, base profile, checked, round-robin time slicing", + "displayName": "native, GCC, base profile, checked, time slicing and work-queue statistics", "inherits": "native-gcc", "binaryDir": "${sourceDir}/build/native-timeslice", "cacheVariables": { diff --git a/configs/native-timeslice.conf b/configs/native-timeslice.conf index 855e623..349f755 100644 --- a/configs/native-timeslice.conf +++ b/configs/native-timeslice.conf @@ -5,6 +5,8 @@ CONFIG_EMB_TIME_SLICING=y CONFIG_EMB_TIMESLICE_TICKS=2 CONFIG_EMB_TIMESLICE_PRIORITY=20 +# Work-queue statistics (KRN-WQ-011), also off by default, are built and tested here too. +CONFIG_EMB_WORKQ_STATS=y # Periodic clock: the quantum then counts delivered ticks exactly, as the model does. In # tickless builds it starts from the free-running counter, which the model cannot see. # CONFIG_EMB_TICKLESS is not set diff --git a/docs/STATUS.md b/docs/STATUS.md index 1be11b0..8441e0a 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -13,7 +13,7 @@ This page is the authoritative inventory of implementation and available evidenc | Synchronization | Semaphores, mutexes with inheritance, ceiling, recursion and owner-death policy; event flags with ANY/ALL/CLEAR waits and binding (M2-05); condition variables with the mandatory relock (M2-06) | Conformance and native differential bridge; inheritance and deadlock detection have a configured depth limit | | Communication | Copy-by-value message queues: direct hand-off to a waiting receiver, slot hand-off from a waiting sender, send to front, peek, `EMB_NO_WAIT` calls from ISRs and a receive-side binding (M2-07) | Conformance and native differential bridge; no pipes, buffer pools or zero-copy ownership queues; the masked copy time is bounded by configuration, not measured on hardware | | Software timers | One-shot and drift-free periodic timers in their own deadline list (ADR-047), with callbacks on a work queue or in interrupt context, closed-form catch-up and overrun counts, stop states and `stop_sync` (M2-09) | Conformance and native differential bridge; list scheduler only (not tiny); arming overhead and jitter not measured on hardware | -| Deferred work | Work queues on a kernel thread with FIFO execution, idempotent and ISR-safe submit, resubmission while running, cancel and synchronous cancel, flush, destroy joining the thread, a system queue linked only when referenced (M2-08, ADR-048), and delayed work items over internal timers (M2-09) | Conformance and native differential bridge; list scheduler only (not tiny); no statistics | +| Deferred work | Work queues on a kernel thread with FIFO execution, idempotent and ISR-safe submit, resubmission while running, cancel and synchronous cancel, flush, destroy joining the thread, a system queue linked only when referenced (M2-08, ADR-048), and delayed work items over internal timers (M2-09) | Conformance and native differential bridge; list scheduler only (not tiny); optional per-queue statistics (A65), conformance-tested on native and spot-checked on QEMU Cortex-M4 | | Notifications | Per-thread bits; bindings from semaphores, event flags and message queues to a thread, with unbind; a bound thread cannot be destroyed (ADR-046) | Binding takes a one-bit mask; pipes and pools do not exist to bind; restart generations need the isolated profile | | Objects | Caller-owned generated storage and POINTER handles; per-type active-object lists for every handle object type (`CONFIG_EMB_OBJ_LIST`) used by init to refuse live storage (ADR-045) | No INDEXED or TABLE handles, capability rights, partition restart, system freeze, `emb_obj_foreach()` or debug-descriptor export | | Native port | POSIX host-thread gate, deterministic IRQ injection, virtual time | Base/tiny development workflow; no current wall-clock, asynchronous signal stress, simulated isolated profile or Win32 backend | diff --git a/docs/requirements/KRN-WQ.md b/docs/requirements/KRN-WQ.md index f3bffd5..3190d4f 100644 --- a/docs/requirements/KRN-WQ.md +++ b/docs/requirements/KRN-WQ.md @@ -81,6 +81,6 @@ KRN-WQ-001 to 004 originate in `docs/architecture/03-kernel-architecture.md` §6 ### KRN-WQ-011 Trace and statistics **Statement.** Submit, start with submit-to-start latency, end with duration, cancel, and delayed scheduling shall be trace events; maximum FIFO length, worst latency, and longest handler with its address shall be optional statistics per queue. **Rationale.** Work queue latency is the latency of every timer callback and bottom half. -**Status.** Accepted 2026-10-07; trace records implemented (A52); statistics are not. -**Verification.** Test: trace decoder (not written); statistics API test (no API). +**Status.** Accepted 2026-10-07; trace records implemented (A52); statistics implemented with `CONFIG_EMB_WORKQ_STATS` (A65). +**Verification.** Test: `workq_stats_record_depth_latency_and_longest_handler` on `native-timeslice`; trace decoder (not written). **Trace.** SPEC-006 §7; OBS-005 diff --git a/docs/specs/AMENDMENTS-M2.md b/docs/specs/AMENDMENTS-M2.md index 72c4536..e41e68f 100644 --- a/docs/specs/AMENDMENTS-M2.md +++ b/docs/specs/AMENDMENTS-M2.md @@ -42,3 +42,4 @@ Corrections found while preparing the M2 primitives, where an accepted specifica | A62 | KRN-SCH-036, 03 §2.1 | The priority count's lower bound is 2, as `CONFIG_EMB_PRIORITY_COUNT` has always allowed, not 4: idle plus one application level is a valid tiny configuration. Accepted by the project owner 2026-10-31. | `kernel/Kconfig` | | A63 | SPEC-003 §8, KRN-SCH-007, KRN-SCH-026, KRN-SCH-037, KRN-SCH-041, KRN-TIM-029 | Time slicing is implemented with one quantum (`CONFIG_EMB_TIMESLICE_TICKS`) and one threshold (`CONFIG_EMB_TIMESLICE_PRIORITY`, 255 for every level). It requires the list scheduler: `CONFIG_EMB_TIME_SLICING` depends on `!CONFIG_EMB_SCHED_TABLE`, since a table level holds one thread. The quantum restarts at every dispatch, also for a preempted thread resuming ahead of its peers, and a yield without a peer keeps it running. Expiry renews the quantum and, with a READY peer, sets a per-CPU rotation flag that makes the next decision place the thread behind its peers. A thread raised above the threshold stops being sliced at its next expiry; one lowered below it starts at its next dispatch. The reference model implements the same rules; the differential bridge checks them with the periodic clock only (`native-timeslice`), because a tickless quantum starts from the free-running counter, which can run ahead of the ticks the model counts. Implemented 2026-10-31 at the project owner's request; the details await review. | `kernel/time/time.c`, `kernel/sched/sched.c`, `kernel/Kconfig` | | A64 | SPEC-001 §6.4, SPEC-009 §5.2, API-021, A40 | The deferred cross-type `_DEFINE` change: `EMB_SEM_DEFINE(sym, initial, max)`, `EMB_MUTEX_DEFINE(sym, protocol, ceiling, flags)`, `EMB_EVENT_DEFINE(sym, initial, flags)`, `EMB_COND_DEFINE(sym, flags)`, `EMB_MSGQ_DEFINE(sym, item_size, capacity, flags)` with its ring, and `EMB_WORKQ_DEFINE(sym, priority, stack_size)` with its stack. They take positional fields like `EMB_THREAD_DEFINE` rather than an attribute initializer, because a partial initializer would zero fields whose defaults are not zero (a semaphore's `max`, a work queue's priority). The table entry stays one init-wrapper pointer, as for threads, not `{init_fn, &storage, &attr, &handle}`. `EMB_TIMER_DEFINE(sym, fn, arg)` defines interrupt-context timers only: a work-queue timer needs its queue at construction and table entries, the system queue's included, run in link order, while `EMB_INIT_AFTER` is not implemented. The object name is the symbol where `CONFIG_EMB_OBJ_NAMES` keeps names, else none. A construction failure faults through `emb_check_status()`. | `include/emb/*.h` | +| A65 | SPEC-006 §7, KRN-WQ-011, A52 | Work-queue statistics are `CONFIG_EMB_WORKQ_STATS` (off by default): `emb_workq_get_stats()` fills `emb_workq_stats_t` (worst submit-to-start latency, longest handler and its function, items queued now and at most), and `emb_workq_reset_stats()` restarts the maxima, the pending maximum from the items queued now. Both are `thread isr` and O(1). Times are kernel-clock ticks, not cycles, so a handler shorter than a tick reads as 0. A submit while the item's handler runs starts the latency of the run it queues, and flush markers are not items. Each work item gains a submit timestamp. The per-thread count of notification sets is not implemented. | `kernel/work/workq.c`, `include/emb/work.h` | diff --git a/docs/specs/SPEC-006-notifications-and-work-queues.md b/docs/specs/SPEC-006-notifications-and-work-queues.md index 7482532..250678c 100644 --- a/docs/specs/SPEC-006-notifications-and-work-queues.md +++ b/docs/specs/SPEC-006-notifications-and-work-queues.md @@ -195,7 +195,7 @@ Handler latency is the sum of the queue thread's dispatch latency at its priorit ## 7. Observability -Trace events: `notify_set(target, bits, woke)`, `notify_wait(thread, mask, mode, result)`, `bind_signal(object, thread, bit)`, `work_submit(queue, item, result)`, `work_start(queue, item, latency)`, `work_end(queue, item, duration)`, `work_cancel(item, state)`, `dwork_schedule(item, deadline)`. Statistics (optional): per queue, maximum FIFO length, worst submit-to-start latency, longest handler with its address; per thread, notification sets received. The debug descriptor (ADR-011) exports the work item layout so a debugger can list a queue's pending items. +Trace events: `notify_set(target, bits, woke)`, `notify_wait(thread, mask, mode, result)`, `bind_signal(object, thread, bit)`, `work_submit(queue, item, result)`, `work_start(queue, item, latency)`, `work_end(queue, item, duration)`, `work_cancel(item, state)`, `dwork_schedule(item, deadline)`. Statistics (optional): per queue, maximum FIFO length, worst submit-to-start latency, longest handler with its address, read with `emb_workq_get_stats()` and restarted with `emb_workq_reset_stats()` under `CONFIG_EMB_WORKQ_STATS` (A65); per thread, notification sets received (not implemented). The debug descriptor (ADR-011) exports the work item layout so a debugger can list a queue's pending items. ## 8. Tiny profile diff --git a/docs/user/configuration.md b/docs/user/configuration.md index aeea325..bfe5b74 100644 --- a/docs/user/configuration.md +++ b/docs/user/configuration.md @@ -13,7 +13,7 @@ The configuration pipeline resolves Kconfig defaults seeded by a port manifest, | `native-clang` | Clang compiler coverage; requires Clang | | `native-asan` / `native-tsan` | Sanitizer configurations; require supported compiler runtimes | | `native-coverage` | GCC `--coverage` build; after CTest, `tools/coverage/report.py` prints kernel statement and branch coverage | -| `native-timeslice` | GCC, checked base with round-robin time slicing (quantum 2 ticks, levels up to 20) and a periodic clock, so the differential bridge checks rotation | +| `native-timeslice` | GCC, checked base with round-robin time slicing (quantum 2 ticks, levels up to 20) and a periodic clock, so the differential bridge checks rotation; also builds the work-queue statistics | | `native-timeslice-tickless` | The same slicing with the tickless clock; conformance only | Use the same preset name for configure, build and CTest. A tiny result does not verify base-only features; a release result does not exercise checked-build faults. Embedded presets additionally require their cross-compiler, libc and emulator; do not interpret a build with no registered tests as conformance. @@ -31,6 +31,7 @@ The later [timer-delivery validation](../engineering/m2-01-timer-correction.md) | `CONFIG_EMB_MSGQ`, `CONFIG_EMB_MSGQ_MAX_ITEM` | y; 64 bytes (16 on tiny) | message queues; the item bound limits the copy made with interrupts masked | | `CONFIG_EMB_IPC_SMALL` | n (y on tiny) | 8-bit queue counts, at most 255 items | | `CONFIG_EMB_WORKQ` | y | work queues and, with timers, delayed work; not available with the tiny profile's table scheduler | +| `CONFIG_EMB_WORKQ_STATS` | n | per-queue maximum backlog, worst latency and longest handler (`emb_workq_get_stats()`) | | `CONFIG_EMB_SYSTEM_WORKQ`, `_PRIO`, `_STACK` | y; 0 (second-highest priority); 2048 bytes | the system work queue, linked only when `emb_workq_system()` is referenced | | `CONFIG_EMB_TIMER` | y | software timers; not available with the table scheduler | | `CONFIG_EMB_OBJ_LIST` | y (n on tiny) | exact refusal of init on live storage | diff --git a/docs/user/tasks.md b/docs/user/tasks.md index 1feb438..f3be224 100644 --- a/docs/user/tasks.md +++ b/docs/user/tasks.md @@ -123,7 +123,7 @@ emb_work_init(&job.work, process); (void)emb_work_submit(jobs, &job.work); /* EMB_EEXIST if it is already queued */ ``` -An item is never queued twice and never runs concurrently with itself; submitting it while its handler runs queues it once more for after the handler returns. `emb_work_cancel()` reports whether the item was pending (now removed), running (it completes) or idle. Before reusing an item's storage, call `emb_work_cancel_sync()`, which also waits for a running handler. `emb_workq_flush()` waits until everything queued before the call has run. Calling either from the queue's own handlers would wait forever and is misuse, as is returning from a handler with a mutex held. The queue's thread belongs to the kernel: thread calls on it are refused and it cannot be notified directly. `emb_workq_destroy()` needs an idle queue and returns after the thread has exited. Referencing `emb_workq_system()` links in a shared system queue at a high priority; keep its handlers short. +An item is never queued twice and never runs concurrently with itself; submitting it while its handler runs queues it once more for after the handler returns. `emb_work_cancel()` reports whether the item was pending (now removed), running (it completes) or idle. Before reusing an item's storage, call `emb_work_cancel_sync()`, which also waits for a running handler. `emb_workq_flush()` waits until everything queued before the call has run. Calling either from the queue's own handlers would wait forever and is misuse, as is returning from a handler with a mutex held. The queue's thread belongs to the kernel: thread calls on it are refused and it cannot be notified directly. `emb_workq_destroy()` needs an idle queue and returns after the thread has exited. Referencing `emb_workq_system()` links in a shared system queue at a high priority; keep its handlers short. With `CONFIG_EMB_WORKQ_STATS`, `emb_workq_get_stats()` shows whether they are: it reports each queue's worst submit-to-start latency, its longest handler and which function that was, and its deepest backlog. For work that should run later, an `emb_dwork_t` adds a delay: `emb_dwork_init(&dw, fn)`, then `emb_dwork_schedule(queue, &dw, EMB_MS(50))` submits it when the delay expires. Scheduling it again while the delay runs returns `EMB_EEXIST`; `emb_dwork_reschedule()` replaces the delay instead. `emb_dwork_cancel()` stops the delay or removes the queued item, and `emb_dwork_cancel_sync()` also waits for a running handler. The handler receives `&dw.work`. diff --git a/include/emb/api_ids.h b/include/emb/api_ids.h index 05f9ca3..e7e8e92 100644 --- a/include/emb/api_ids.h +++ b/include/emb/api_ids.h @@ -146,5 +146,7 @@ #define EMB_API_ID_DWORK_RESCHEDULE 0x008Cu #define EMB_API_ID_DWORK_CANCEL 0x008Du #define EMB_API_ID_DWORK_CANCEL_SYNC 0x008Eu +#define EMB_API_ID_WORKQ_GET_STATS 0x008Fu +#define EMB_API_ID_WORKQ_RESET_STATS 0x0090u #endif /* EMB_API_IDS_H */ diff --git a/include/emb/work.h b/include/emb/work.h index 668e493..0c1a418 100644 --- a/include/emb/work.h +++ b/include/emb/work.h @@ -131,6 +131,58 @@ emb_status_t emb_workq_destroy(emb_workq_t workq); */ emb_status_t emb_workq_flush(emb_workq_t workq, emb_timeout_t timeout); +#if CONFIG_EMB_WORKQ_STATS +/* A queue's statistics (KRN-WQ-011): maxima since init or the last reset. */ +typedef struct emb_workq_stats { + emb_duration_t max_latency; /* worst time from a submit to its handler's start */ + emb_duration_t max_duration; /* longest handler run */ + emb_work_fn_t max_duration_fn; /* the handler that took it; NULL before the first run */ + uint16_t pending; /* items queued now */ + uint16_t max_pending; /* most items queued at once */ +} emb_workq_stats_t; + +/** + * emb_workq_get_stats() - Read a queue's statistics. + * @workq: Initialized work queue. + * @out_stats: Receives the statistics. + * + * @ctx thread isr + * @blocks no + * @time O(1) + * @owns none + * @config CONFIG_EMB_WORKQ_STATS + * @req KRN-WQ-011 + * @since 0.3 + * @stable no + * + * Times are kernel-clock ticks. A submit while the item's handler runs starts the latency + * of the run it queues. Flush markers are not items. + * + * Return: EMB_OK; EMB_EINVAL for a bad handle or a NULL @out_stats. + */ +emb_status_t emb_workq_get_stats(emb_workq_t workq, emb_workq_stats_t *out_stats); + +/** + * emb_workq_reset_stats() - Restart a queue's maxima from now. + * @workq: Initialized work queue. + * + * @ctx thread isr + * @blocks no + * @time O(1) + * @owns none + * @config CONFIG_EMB_WORKQ_STATS + * @req KRN-WQ-011 + * @since 0.3 + * @stable no + * + * The latency and duration maxima return to zero and the pending maximum to the items + * queued now. + * + * Return: EMB_OK; EMB_EINVAL for a bad handle. + */ +emb_status_t emb_workq_reset_stats(emb_workq_t workq); +#endif + #if CONFIG_EMB_SYSTEM_WORKQ /** * emb_workq_system() - The system work queue. diff --git a/kernel/Kconfig b/kernel/Kconfig index 6e20c4f..54d2a09 100644 --- a/kernel/Kconfig +++ b/kernel/Kconfig @@ -279,6 +279,16 @@ config EMB_WORKQ A kernel thread per queue runs submitted items in order. The tiny profile's table scheduler has no work queues in M2. +config EMB_WORKQ_STATS + bool "Per-queue work statistics (KRN-WQ-011)" + depends on EMB_WORKQ + default n + help + Each queue records the most items it held at once, the worst submit-to-start + latency and the longest handler with its address, read with + emb_workq_get_stats(). Costs a timestamp per work item and two clock reads per + handler run. + config EMB_SYSTEM_WORKQ bool "The system work queue, linked only when referenced (ADR-048)" depends on EMB_WORKQ diff --git a/kernel/include/embk/work.h b/kernel/include/embk/work.h index cfd58e1..ba84af9 100644 --- a/kernel/include/embk/work.h +++ b/kernel/include/embk/work.h @@ -22,6 +22,9 @@ typedef struct embk_work { uint8_t tag; /* EMBK_WORK_TAG once initialized */ uint8_t state; /* EMB_WORK_IDLE | EMB_WORK_PENDING | EMB_WORK_RUNNING */ uint8_t flags; /* EMBK_WORK_RESUBMITTED | EMBK_WORK_MARKER */ +#if CONFIG_EMB_WORKQ_STATS + emb_tick_t submitted; /* the last submit: the start of its latency */ +#endif } embk_work_t; typedef struct embk_workq { @@ -32,6 +35,13 @@ typedef struct embk_workq { embk_work_t *running; /* the item whose handler runs, or NULL */ uint8_t stopping; /* destroy has begun: the thread exits at an empty FIFO */ uint8_t timer_refs; /* timers whose callbacks run here: destroy is refused */ +#if CONFIG_EMB_WORKQ_STATS + uint16_t pending; /* items queued now, flush markers excluded */ + uint16_t max_pending; /* KRN-WQ-011 statistics from here */ + emb_tick_t max_latency; /* worst submit to start */ + emb_tick_t max_duration; /* longest handler */ + void (*max_duration_fn)(struct emb_work *work); /* the handler that took it */ +#endif #if CONFIG_EMB_OBJ_LIST embk_list_node_t all_node; /* in embk_workq_all from init to destroy */ #endif diff --git a/kernel/work/workq.c b/kernel/work/workq.c index 4174f29..886567a 100644 --- a/kernel/work/workq.c +++ b/kernel/work/workq.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include @@ -62,9 +63,33 @@ static void wake_done_locked(embk_workq_t *q, const embk_work_t *w) } } +#if CONFIG_EMB_WORKQ_STATS +/* An item joined @q's FIFO. Lock held. */ +static void stats_queued(embk_workq_t *q) +{ + if (q->pending != UINT16_MAX) { + q->pending++; + } + if (q->pending > q->max_pending) { + q->max_pending = q->pending; + } +} + +static void stats_reset(embk_workq_t *q) +{ + q->max_pending = q->pending; + q->max_latency = 0u; + q->max_duration = 0u; + q->max_duration_fn = NULL; +} +#endif + /* Unlink a pending item: it will not run. Lock held. */ static void unlink_locked(embk_work_t *w) { +#if CONFIG_EMB_WORKQ_STATS + w->queue->pending--; +#endif embk_list_remove(&w->node); w->state = EMB_WORK_IDLE; w->queue = NULL; @@ -74,6 +99,10 @@ static void unlink_locked(embk_work_t *w) static void workq_main(void *arg) { embk_workq_t *q = (embk_workq_t *)arg; +#if CONFIG_EMB_WORKQ_STATS + emb_tick_t start; + void (*fn)(struct emb_work *work); +#endif for (;;) { emb_irq_key_t key = embk_lock_object(q); embk_list_node_t *n = embk_list_pop_first(&q->fifo); @@ -98,11 +127,26 @@ static void workq_main(void *arg) } w->state = EMB_WORK_RUNNING; q->running = w; +#if CONFIG_EMB_WORKQ_STATS + q->pending--; + start = embk_time_ticks_locked(); + if ((emb_tick_t)(start - w->submitted) > q->max_latency) { + q->max_latency = start - w->submitted; + } + fn = w->fn; +#endif EMBK_TRACE(EMB_TRACE_WORK_RUN, embk_thread_index(&q->thread), w, 0u); embk_unlock_object(q, key); w->fn((emb_work_t *)(void *)w); key = embk_lock_object(q); EMBK_TRACE(EMB_TRACE_WORK_RUN, embk_thread_index(&q->thread), w, 1u); +#if CONFIG_EMB_WORKQ_STATS + start = embk_time_ticks_locked() - start; /* now the handler's duration */ + if (start > q->max_duration) { + q->max_duration = start; + q->max_duration_fn = fn; + } +#endif #if CONFIG_EMB_MUTEX if (q->thread.owned != NULL) { /* the next handler would inherit the lock (SPEC-006 §6) */ @@ -115,6 +159,9 @@ static void workq_main(void *arg) w->flags &= (uint8_t)~EMBK_WORK_RESUBMITTED; w->state = EMB_WORK_PENDING; embk_list_append(&q->fifo, &w->node); +#if CONFIG_EMB_WORKQ_STATS + stats_queued(q); /* its latency runs from the resubmitting call */ +#endif } else { w->state = EMB_WORK_IDLE; w->queue = NULL; @@ -176,6 +223,10 @@ emb_status_t emb_workq_init(emb_workq_storage_t *storage, const emb_workq_attr_t q->running = NULL; q->stopping = 0u; q->timer_refs = 0u; +#if CONFIG_EMB_WORKQ_STATS + q->pending = 0u; + stats_reset(q); +#endif emb_thread_attr_default(&ta); ta.name = attr->name; ta.stack = attr->stack; @@ -306,6 +357,36 @@ emb_status_t emb_workq_flush(emb_workq_t workq, emb_timeout_t timeout) return st; } +#if CONFIG_EMB_WORKQ_STATS +emb_status_t emb_workq_get_stats(emb_workq_t workq, emb_workq_stats_t *out_stats) +{ + embk_workq_t *q = workq_from_handle(workq); + emb_irq_key_t key; + EMBK_API_REQUIRE(EMB_API_ID_WORKQ_GET_STATS, q != NULL, EMB_FAULT_API_HANDLE, EMB_EINVAL, 1u); + EMBK_API_REQUIRE(EMB_API_ID_WORKQ_GET_STATS, out_stats != NULL, EMB_FAULT_API_ARGUMENT, + EMB_EINVAL, 2u); + key = embk_lock_object(q); + out_stats->max_latency.ticks = q->max_latency; + out_stats->max_duration.ticks = q->max_duration; + out_stats->max_duration_fn = q->max_duration_fn; + out_stats->pending = q->pending; + out_stats->max_pending = q->max_pending; + embk_unlock_object(q, key); + return EMB_OK; +} + +emb_status_t emb_workq_reset_stats(emb_workq_t workq) +{ + embk_workq_t *q = workq_from_handle(workq); + emb_irq_key_t key; + EMBK_API_REQUIRE(EMB_API_ID_WORKQ_RESET_STATS, q != NULL, EMB_FAULT_API_HANDLE, EMB_EINVAL, 1u); + key = embk_lock_object(q); + stats_reset(q); + embk_unlock_object(q, key); + return EMB_OK; +} +#endif + void embk_work_init(embk_work_t *w, void (*fn)(struct emb_work *work)) { embk_list_node_init(&w->node); @@ -333,6 +414,9 @@ emb_status_t embk_work_submit_locked(embk_workq_t *q, embk_work_t *w) } else if (w->state == EMB_WORK_RUNNING) { if (w->queue == q) { w->flags |= EMBK_WORK_RESUBMITTED; /* queued again when the handler returns */ +#if CONFIG_EMB_WORKQ_STATS + w->submitted = embk_time_ticks_locked(); +#endif } else { st = EMB_EBUSY; /* an item belongs to one queue at a time */ } @@ -340,6 +424,10 @@ emb_status_t embk_work_submit_locked(embk_workq_t *q, embk_work_t *w) w->state = EMB_WORK_PENDING; w->queue = q; embk_list_append(&q->fifo, &w->node); +#if CONFIG_EMB_WORKQ_STATS + w->submitted = embk_time_ticks_locked(); + stats_queued(q); +#endif embk_notify_set_locked(&q->thread, EMB_NOTIFY_K_WORK); } EMBK_TRACE(EMB_TRACE_WORK_SUBMIT, embk_thread_index(&q->thread), w, st); diff --git a/tests/conformance/test_workq.c b/tests/conformance/test_workq.c index 06968aa..270cc07 100644 --- a/tests/conformance/test_workq.c +++ b/tests/conformance/test_workq.c @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: Apache-2.0 */ /* Copyright 2026 Junior Deogracias */ -/* Work queue conformance (SPEC-006 §4; KRN-WQ-001, 002, 004 to 006, 008 to 010). */ +/* Work queue conformance (SPEC-006 §4; KRN-WQ-001, 002, 004 to 006, 008 to 011). */ #include #define IRQ_W 2u /* software interrupt slot */ @@ -348,3 +348,35 @@ EMB_TEST(workq_define_constructs_in_init_table) emb_test_mark(1); EMB_ASSERT_MARKS(4, 1); } + +#if CONFIG_EMB_WORKQ_STATS +EMB_TEST(workq_stats_record_depth_latency_and_longest_handler) +{ + emb_workq_stats_t st; + EMB_TEST_REQ("KRN-WQ-011"); + make(2u); + items[0].sleep_ticks = 2u; /* the others wait behind it */ + emb_sched_lock(); + EMB_ASSERT_OK(emb_work_submit(wq, &items[0].work)); + EMB_ASSERT_OK(emb_work_submit(wq, &items[1].work)); + EMB_ASSERT_OK(emb_work_submit(wq, &items[2].work)); + EMB_ASSERT_OK(emb_workq_get_stats(wq, &st)); + EMB_ASSERT_EQ(st.pending, 3u); + emb_sched_unlock(); + EMB_ASSERT_OK(emb_workq_flush(wq, EMB_WAIT_FOREVER)); + EMB_ASSERT_OK(emb_workq_get_stats(wq, &st)); + EMB_ASSERT_EQ(st.pending, 0u); + EMB_ASSERT_EQ(st.max_pending, 3u); /* the flush marker is not an item */ + emb_test_check_elapsed(__LINE__, (uint32_t)st.max_latency.ticks, 2u); + emb_test_check_elapsed(__LINE__, (uint32_t)st.max_duration.ticks, 2u); + EMB_ASSERT_TRUE(st.max_duration_fn == handler); + EMB_ASSERT_OK(emb_workq_reset_stats(wq)); + EMB_ASSERT_OK(emb_workq_get_stats(wq, &st)); + EMB_ASSERT_EQ(st.max_pending, 0u); + EMB_ASSERT_EQ(st.max_latency.ticks, 0u); + EMB_ASSERT_EQ(st.max_duration.ticks, 0u); + EMB_ASSERT_TRUE(st.max_duration_fn == NULL); + EMB_ASSERT_MISUSE(emb_workq_get_stats(wq, NULL), EMB_EINVAL); + EMB_ASSERT_OK(emb_workq_destroy(wq)); +} +#endif diff --git a/tools/fault/api-ids.json b/tools/fault/api-ids.json index 73acc67..100654e 100644 --- a/tools/fault/api-ids.json +++ b/tools/fault/api-ids.json @@ -853,6 +853,18 @@ "name": "emb_dwork_cancel_sync", "header": "work.h", "state": "active" + }, + { + "id": 143, + "name": "emb_workq_get_stats", + "header": "work.h", + "state": "active" + }, + { + "id": 144, + "name": "emb_workq_reset_stats", + "header": "work.h", + "state": "active" } ] } diff --git a/tools/fault/api-registry.json b/tools/fault/api-registry.json index b618c6a..4656fee 100644 --- a/tools/fault/api-registry.json +++ b/tools/fault/api-registry.json @@ -852,6 +852,18 @@ "name": "emb_dwork_cancel_sync", "header": "work.h", "state": "active" + }, + { + "id": 143, + "name": "emb_workq_get_stats", + "header": "work.h", + "state": "active" + }, + { + "id": 144, + "name": "emb_workq_reset_stats", + "header": "work.h", + "state": "active" } ] } From 9ee0945d9d398c942429583b157995b7ae2954c7 Mon Sep 17 00:00:00 2001 From: teo1747 Date: Sat, 31 Oct 2026 01:19:43 +0100 Subject: [PATCH 2/3] ci: run the experimental EmbCC-kernel native presets A new embcc job installs the EmbCC 0.0.1-beta Linux x86-64 release, checked against a pinned SHA-256. It then configures, builds and tests native-embcc, -release, -tiny and -periodic: an EmbCC-built kernel with a GCC-built POSIX host (ADR-041). The merge gate requires it. Signed-off-by: teo1747 --- .github/workflows/ci.yml | 36 ++++++++++++++++++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 2177e4b..5333658 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -5,7 +5,7 @@ # AVR with avr-gcc run under QEMU and on EmbSIM's Arduino Uno, the Cortex-M4F suite on EmbSIM's # NUCLEO-F446RE (SIM-002), formatting, and the footprint gate (TEST-011). # Target order: Cortex-M and RISC-V first, AVR secondary (ADR-038). -# The EmbCC legs join when the compiler is available to CI (09 §11). +# The experimental EmbCC-kernel / GCC-host native legs use a pinned EmbCC release (09 §11). name: ci on: @@ -371,10 +371,42 @@ jobs: path: case-evidence-rv32-c3-embsim.json if-no-files-found: warn + # The experimental EmbCC-kernel / GCC-host native build (ADR-041), with the compiler pinned + # to a release and its digest (docs/engineering/embcc-native-linux.md). + embcc: + runs-on: ubuntu-24.04 + strategy: + fail-fast: false + matrix: + preset: [native-embcc, native-embcc-release, native-embcc-tiny, native-embcc-periodic] + env: + EMBCC_VERSION: 0.0.1-beta + EMBCC_SHA256: 984b25acf1b93d61416887530c9589be96b43437a8de8d61ffd07cf6a856551a + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Install tools + run: | + sudo apt-get update + sudo apt-get install -y ninja-build python3-yaml + - name: Install EmbCC + run: | + archive="embcc-${EMBCC_VERSION}-linux-x86_64.tar.gz" + curl -fsSL -o "$archive" "https://github.com/EmbLinkOs/EmbCC/releases/download/v${EMBCC_VERSION}/${archive}" + echo "${EMBCC_SHA256} ${archive}" | sha256sum -c - + mkdir -p "$HOME/embcc" + tar -xzf "$archive" -C "$HOME/embcc" --strip-components=1 + "$HOME/embcc/bin/embcc" --version + - name: Configure + run: cmake --preset ${{ matrix.preset }} -DEMB_KERNEL_COMPILER="$HOME/embcc/bin/embcc" + - name: Build + run: cmake --build --preset ${{ matrix.preset }} + - name: Test + run: ctest --preset ${{ matrix.preset }} --output-on-failure --no-tests=error + merge_gate: name: CI merge gate if: ${{ always() }} - needs: [native, cortex_m, riscv, avr, avr-embsim, cm4-embsim, rv32-embsim, lint] + needs: [native, cortex_m, riscv, avr, avr-embsim, cm4-embsim, rv32-embsim, embcc, lint] runs-on: ubuntu-24.04 steps: - name: Require every matrix and lint job to succeed From 459fc281f805ba78de07e45112c38ab6bcac6971 Mon Sep 17 00:00:00 2001 From: teo1747 Date: Sat, 31 Oct 2026 01:22:36 +0100 Subject: [PATCH 3/3] docs(embcc): record the pinned EmbCC CI leg The embcc job's first run passed 20, 20, 20 and 17 CTest targets for the checked, release, periodic and tiny presets with an EmbCC 0.0.1-beta kernel. The Linux integration report, STATUS and CHANGELOG now say that the leg exists, is merge-gated and is pinned by version and digest. Signed-off-by: teo1747 --- CHANGELOG.md | 2 ++ docs/STATUS.md | 4 ++-- docs/engineering/embcc-native-linux.md | 6 +++++- 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 251be76..7af5da5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,8 @@ User-visible changes are recorded here. There are no released versions or releas ### Added +- A CI job runs the experimental EmbCC-kernel / GCC-host native presets (`native-embcc`, `-release`, `-tiny`, `-periodic`) with the EmbCC `0.0.1-beta` release, pinned by version and SHA-256; the merge gate requires it. + - Work-queue statistics (`CONFIG_EMB_WORKQ_STATS`, off by default; KRN-WQ-011, A65): `emb_workq_get_stats()` reports the worst submit-to-start latency, the longest handler with its function and the deepest backlog per queue, in ticks; `emb_workq_reset_stats()` restarts them. - Static definitions for the other object types (A64): `EMB_SEM_DEFINE`, `EMB_MUTEX_DEFINE`, `EMB_EVENT_DEFINE`, `EMB_COND_DEFINE`, `EMB_MSGQ_DEFINE`, `EMB_WORKQ_DEFINE` and, for interrupt-context callbacks, `EMB_TIMER_DEFINE` declare storage (with the ring or stack) and the handle and construct the object in `emb_kernel_init()`. diff --git a/docs/STATUS.md b/docs/STATUS.md index 8441e0a..d229871 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -18,7 +18,7 @@ This page is the authoritative inventory of implementation and available evidenc | Objects | Caller-owned generated storage and POINTER handles; per-type active-object lists for every handle object type (`CONFIG_EMB_OBJ_LIST`) used by init to refuse live storage (ADR-045) | No INDEXED or TABLE handles, capability rights, partition restart, system freeze, `emb_obj_foreach()` or debug-descriptor export | | Native port | POSIX host-thread gate, deterministic IRQ injection, virtual time | Base/tiny development workflow; no current wall-clock, asynchronous signal stress, simulated isolated profile or Win32 backend | | Embedded ports | ATmega328P; Cortex-M3/M4F; RV32IMAC and RV32IMC machine mode; boards Arduino Uno, NUCLEO-F446RE, ESP32-C3-DevKitM-1, QEMU's MPS2 and RV32 virt | QEMU jobs and EmbSIM jobs (Uno, NUCLEO-F446RE, ESP32-C3-DevKitM-1) defined in CI; no real-board timing or peripheral/HIL validation recorded here | -| Configuration/build | Kconfig subset, port manifests, CMake/Ninja, layout probe, footprint tool | GCC/Clang native CI and cross-GCC QEMU jobs; ten legacy EmbCC capabilities reviewed with scoped local checks, but current RTOS compiler CI/qualification remains incomplete | +| Configuration/build | Kconfig subset, port manifests, CMake/Ninja, layout probe, footprint tool | GCC/Clang native CI and cross-GCC QEMU jobs; an EmbCC-kernel/GCC-host native CI leg on a pinned release; ten legacy EmbCC capabilities reviewed with scoped local checks, but embedded EmbCC builds and compiler qualification remain incomplete | | Observability | M1 trace sink and fault reporting; trace records for every outcome of the M2 waits and for queue, work and timer events (IDs 31 to 44) | CTF rings/metadata, deferred log decoder, retained crash records, debug descriptor and worst-case monitors remain planned | | Documentation | Sphinx/MyST manuals and native tutorials, Doxygen/Breathe C API, Sphinx Needs design catalog, JSON/CSV downloads and annotation audit | Strict local HTML build, complete extracted function annotations, source-preservation and catalog-integrity checks; merged `37887c9` Pages deployment and served evidence-workflow page independently observed successful | | Repository process | Issue forms, PR checklist, changelog, release/M2 work checklists, stable CI merge checks, authored-commit DCO checker and a main ruleset policy | Local syntax and DCO/gate failure checks; GitHub's effective main rules report PRs, resolved conversations, the three strict CI checks and deletion/non-fast-forward protection; no remote rejection experiment claimed | @@ -66,7 +66,7 @@ The user-guide pass also configured native/base with stack statistics and two TL ## Known limits and unresolved evidence -The [EmbCC layout-marker correction](engineering/embcc-layout-marker.md), source `3578006`, supplies the portability macro; storage/assertion/kernel-archive compilation passed on eight configurations. The subsequent [Linux native integration](engineering/embcc-native-linux.md), source `df32ec2`, adds explicit experimental EmbCC-kernel/GCC-host presets and a build-time layout guard. Actual checked/release/tiny/periodic mixed builds passed all twelve CTest targets each and 11,200 seeded model comparisons in total, with zero divergences. GCC compiles the POSIX port, board, applications and harness and links the real Linux runtime; this is not an all-EmbCC host build. The latter still fails on glibc header compatibility after an explicit system-header/link experiment. Compiler CI currently checks the marker/launcher contracts and the existing matrix; the EmbCC executions are scoped local evidence. +The [EmbCC layout-marker correction](engineering/embcc-layout-marker.md), source `3578006`, supplies the portability macro; storage/assertion/kernel-archive compilation passed on eight configurations. The subsequent [Linux native integration](engineering/embcc-native-linux.md), source `df32ec2`, adds explicit experimental EmbCC-kernel/GCC-host presets and a build-time layout guard. Actual checked/release/tiny/periodic mixed builds passed all twelve CTest targets each and 11,200 seeded model comparisons in total, with zero divergences. GCC compiles the POSIX port, board, applications and harness and links the real Linux runtime; this is not an all-EmbCC host build. The latter still fails on glibc header compatibility after an explicit system-header/link experiment. CI now runs the four mixed presets with the pinned EmbCC `0.0.1-beta` release on every pull request (merge-gated; 20, 20, 20 and 17 CTest targets passed on the first run); deeper campaigns remain scoped local evidence. - Hardware timing and total memory use remain to be demonstrated on the primary Cortex-M board. Map-derived kernel subtotals exclude port, board, runtime and stack costs. - The Cortex-M base TCB figures in the port page exceed R-003 T4's target; that is a disclosed target miss, not a failed native setup. diff --git a/docs/engineering/embcc-native-linux.md b/docs/engineering/embcc-native-linux.md index 2be8db1..ac316bd 100644 --- a/docs/engineering/embcc-native-linux.md +++ b/docs/engineering/embcc-native-linux.md @@ -57,6 +57,10 @@ python3 -I tools/model/bridge.py \ The [evidence archive](https://github.com/EmbLinkOs/EmbLinkRTOS/tree/main/tools/toolchain/evidence/embcc-native-2026-10-15) retains actual compile commands, resolved configurations, layouts, runner/source/toolchain identities, raw and readable logs, and initial POSIX/negative-check inputs. Its manifest records hashes and commands. Readable logs strip trailing whitespace; `raw-logs.json` retains original output. Compiler warnings for unsupported diagnostic options remain visible; no warning suppression was added. +## CI leg + +Since 2026-10-31 the `embcc` CI job runs the four presets on every pull request and push to main, and the merge gate requires it. It installs the EmbCC `0.0.1-beta` Linux x86-64 release archive, checked against a SHA-256 pinned in `.github/workflows/ci.yml`, and passes its `embcc` as `EMB_KERNEL_COMPILER`. On the first run ([PR #45](https://github.com/EmbLinkOs/EmbLinkRTOS/pull/45)) the checked, release and periodic presets passed 20 of 20 CTest targets each and tiny 17 of 17, including the differential bridge at CTest depth (6 runs per scenario, seed 1). The release is a newer compiler than the source revision measured above; updating it means changing the version and the digest together. + ## Remaining scope -CI runs the launcher/storage contracts and existing GCC/Clang/embedded matrix. The actual EmbCC executions above are local evidence, not a pinned EmbCC CI leg. Other hosts, compiler revisions, PIE/LTO, sanitizing the EmbCC kernel, embedded images and physical timing remain unqualified. AVR flash/registration integration, RTOS hardware-import adaptation, embedded C++ checks and complete compiler/port conformance remain separate tasks. A subsequent all-EmbCC host build must resolve the actual POSIX header/runtime/link boundaries rather than inheriting the mixed build's result. +CI runs the launcher/storage contracts, the existing GCC/Clang/embedded matrix and the pinned EmbCC-kernel leg. Deeper campaigns with the EmbCC kernel remain the local evidence above. Other hosts, compiler revisions, PIE/LTO, sanitizing the EmbCC kernel, embedded images and physical timing remain unqualified. AVR flash/registration integration, RTOS hardware-import adaptation, embedded C++ checks and complete compiler/port conformance remain separate tasks. A subsequent all-EmbCC host build must resolve the actual POSIX header/runtime/link boundaries rather than inheriting the mixed build's result.