Repository navigation
libc: POSIX, EmbLinkOS's userland on it, and C++ there by EmbCC - #166
Merged
Merged
Conversation
…tory once The driver appended <binary's directory>/include as a system directory ahead of the defaults paths.c computes -- a line from before paths.c existed. That directory holds the compiler's declarations-only <string.h>, which paths.c deliberately puts LAST so it cannot shadow the C library's; added early, it did, and every build-tree compile without -I lost the POSIX half of <string.h> (strtok_r, strlcpy, ...). Run from $PATH, argv[0] has no '/', and the directory was ./include -- wherever the user happened to be. It is now added only when paths.c found no support files, which is what it was for. With it gone, a directory named both by -I and by the defaults was searched twice in a row, and #include_next from the C library's <limits.h> found that same file again, include guard set, so INT_MAX was never defined (the libc build itself broke). GCC searches such a directory once, where -I put it, as a system directory; so does EmbCC now, by plat_same_dir: device and inode where the host has them (macOS, Linux), spelling elsewhere -- EmbLinkOS's newlib stat has no inodes. tests/golden/install.sh compiles <string.h> + <limits.h> with and without -I lib/libc/include; the first case fails on the old driver and the second on one without the dedupe. build.ebm regenerated for platform_posix.c's <sys/stat.h>.
The first of the POSIX layer EmbLinkOS needs to leave newlib behind --
the part that asks the operating system nothing new.
<string.h> strtok_r, strerror_r (XSI's), strsep, strlcpy, strlcat,
memmem; strcasestr, memrchr, strchrnul, explicit_bzero
<stdlib.h> setenv, unsetenv (declared before, never defined), putenv,
clearenv; the mkstemp family, mkdtemp, mktemp; realpath;
posix_memalign, memalign, reallocarray; qsort_r; getsubopt;
random/srandom/initstate/setstate, the *rand48 family and
rand_r, each giving glibc's sequence for a seed
modules <libgen.h>, <fnmatch.h>, <getopt.h> (getopt_long, and
permuting as glibc's and newlib's do), <search.h> (AVL
tsearch, a growing hsearch, the _r forms)
<features.h> decides what a C header declares beyond ISO C: everything
unless the program asked for ISO C alone, and GNU's additions hidden
too when it names a POSIX version and nothing more, as glibc does.
The environment owns what it allocates: the startup vector is copied on
the first change, setenv's strings are freed when replaced, putenv's
are the caller's and never are. realpath resolves a component at a time,
so ".." after a symlink leaves the link's target. The entropy behind
temporary names is one function now (__tmp_bits), which stdio's tmpnam
will share.
tests/exec/posix-strings.c holds the values glibc 2.36 printed for the
same calls, run against both on a Linux kernel; where they differ the
test says why (tree shape, a growing hash table). Its tree checks catch
an AVL delete that skips rebalancing.
<stdio.h> gains fdopen, fileno, fmemopen, open_memstream, getline, getdelim, dprintf, asprintf and their v forms, fseeko, ftello, ctermid, flockfile/ftrylockfile/funlockfile and the _unlocked functions, and GNU's fopencookie, setbuffer and setlinebuf. <err.h> is new; its messages begin with program_invocation_short_name, which Linux's startup now sets from argv[0]. What it took inside: a FILE may be something other than a descriptor -- four functions and a cookie, laid out as glibc's cookie_io_functions_t -- and the seven places stdio reached the backend go through one dispatcher each; the stream lock NESTS. flockfile(stdout) and then printf takes it twice, and the old lock, which is not recursive, deadlocked -- and on a target with no futex spun forever, every single-threaded program; freopen reopens the stream it is given. It closed it and called fopen, which handed back a different FILE, so freopen(.., stdout) left stdout closed; past FOPEN_MAX, streams come in chunks from the heap that are never freed, so the flush at exit cannot walk into a closed one. fmemopen follows glibc since 2.22 (where a NUL goes); open_memstream's length moves only when a write passes it, which is POSIX's rule and not glibc 2.36's. tests/exec/posix-stdio.c and posix-files.c (mkstemp, realpath through symlinks, freopen of stdout) run on our libc and, posix-stdio at least, on glibc, on a Linux kernel; linux-libc-exec.sh runs both, and posix-strings, with no -I.
EmbLD wrote every image it linked with the mode a plain file gets, 0644 under the usual umask -- so `embcc hello.c -o hello` on a Linux host made a program that could not be run, and a test that installed one in a guest as /bin/sh got EACCES from the kernel's exec. It now gets 0777 less the umask, as GNU ld gives its output: ELF executables, script-linked images and EMBX alike, through plat_make_executable (chmod where the host has modes, nothing in the ISO host layer).
os/backend.h gains POSIX's half: about a hundred primitives, each
shaped like the call it serves (dup, pipe, fcntl, fstat, the *at forms,
fork/execve/spawn/waitpid, sigaction and the masks, clocks, mmap, ppoll,
sockets, termios, ptys, uname and sysconf's questions), with Linux's
numbers for everything that needs one. A backend that cannot do one
still defines it, failing with ENOSYS: os/absent.inc is the list,
guarded per primitive, which the bare-metal and posixlike backends
include whole.
os/linux/posix.c implements every one as syscalls on both 64-bit ABIs,
the numbers read out of the kernel's own uapi headers: open's flags
translated where aarch64 moves them, the kernel's shorter termios and
utsname converted, a signal restorer placed as bytes (checked against
both assemblers), posix_spawn as fork + actions + execve with the
child's error returned over a close-on-exec pipe.
Above them, src/posix: <unistd.h>, <fcntl.h>, <sys/stat.h>,
<dirent.h> (`.` and `..` returned, positions kept), the *at functions,
the exec family with PATH search and the script rule, <sys/wait.h>,
<spawn.h>, system and popen on posix_spawn (so they work where fork
does not), <sys/mman.h>, poll and select, sockets, inet_pton/ntop,
getaddrinfo over the hosts file, the OS's resolver or a DNS client
(resolv.conf, search domains, TCP for truncated answers), termios and
ptys, the user and group files, sysconf and the rest. Every public
POSIX name is weak -- none is reserved to the implementation, and a
program's own `read` or `getline` must win. sys/types.h takes hosted
widths (64-bit ino_t and dev_t, 32-bit uids).
Signals are two models: the kernel's where it delivers (Linux) and
lib/libc's own elsewhere -- a handler table, a mask and a pending set,
so a blocked raise is delivered on unblock -- and abort now raises
SIGABRT first, ending the program even if a handler returns.
tests/exec/posix-{fd,process,signals,net}.c hold what glibc 2.36 gave
for the same calls on the same kernel (they were run against both);
posix-process is its own /bin/sh and child. linux-libc-exec.sh runs
all 19 of its programs on x86-64 and aarch64 Linux: 76 runs exit 42.
The EmbLinkOS backend now defines every primitive of os/backend.h. It
carries over what EmbLinkOS's newlib adapter (user/lib/syscalls.c,
spawn_compat.inc) worked out POSIX means on that kernel, under its
three honest categories -- REAL, VACUOUS (true whatever is done: a
mask where nothing is delivered), ABSENT (ENOSYS, never pretended):
paths normalized here ("..", ".", the 256-byte limit), the
working directory this library's, a child's named in its
spawn or by PWD
files O_CLOEXEC translated (the kernel's is 0x100000),
O_DIRECTORY and O_NOFOLLOW honoured by looking, rename the
kernel's atomic one -- it was unlink-then-rename, which a
crash between the two turned into a lost destination
directories a snapshot that grows until the kernel's silent
truncation is ruled out -- it was a fixed 512 entries
descriptors dup, dup2, pipes, close-on-exec and the status flags;
record locks ENOSYS
processes posix_spawn on the native actions (descriptor snapshot,
cwd, kept status), attributes refused; waitpid on owned
child handles, a cancelled wait EINTR; fork and exec
ENOSYS
time the RTC in seconds, uptime in milliseconds, a thread's CPU
time, each at the precision it has
memory mmap's refusals where the kernel cannot honour a request
network IPv4 sockets and the kernel's resolver; poll on fd_poll
terminals termios and the window size through terminal control
tests/golden/libc-backends.sh reads the primitives out of os/backend.h
and requires every backend -- bare metal (Cortex-M and AVR), posixlike,
Linux on both ABIs, EmbLinkOS -- to define all 139; it fails naming the
one a backend left out.
…efuse time: - tz.c reads TZ as POSIX writes it (quoted names, Mm.w.d / Jn / n rules, times past 24h or negative), and zone names as TZif files (v1-3, with the footer rule) under TZDIR or /usr/share/zoneinfo; no TZ is /etc/localtime, an unusable one is UTC. tzset sets tzname/timezone/daylight. - struct tm carries tm_gmtoff and tm_zone; strftime's %z and %Z read them. - mktime reads tm_isdst as glibc does on the mornings the clocks change. - strptime: the "C" locale's conversions and glibc's readings -- digits stop where one more could not fit, %I joins %p, %C with %y, and the fields a date implies (wday, yday, and a date from %U/%W + weekday). Every value refereed against glibc 2.36 on Linux; tests/exec/posix-time.c holds them and runs in linux-libc-exec on x86-64 and AArch64. spawn: - posix_spawn_file_actions_t and posix_spawnattr_t are handles: init allocates, destroy frees, NULL is EINVAL. - __os_spawn_flags names the attribute flags a backend honours; setflags refuses the rest with ENOTSUP instead of a spawn that ignores them. - posix_spawnp: the system's directories without PATH, and a file the kernel cannot run (ENOEXEC) is run by /bin/sh. EmbLinkOS backend: embk_empty_env for its crt0; libc-emblinkos also makes an empty libm.a, so -lm finds no other library's. libc.sh: the harness's kill (newlib's floor, always EINVAL) is left out, so posix-signals reaches the library's own -- all 366 programs pass.
What EmbLinkOS's own tests asked for, building every program it has against this library (LIBC=emblibc in EmbLinkOS): - kill of another process is ENOSYS where the kernel cannot signal one (ESRCH claimed there was none; mrsh's job control hung on it). - spawn: POSIX's EBADF for a descriptor at or past OPEN_MAX from the add call itself, and __os_spawn_check, a new primitive, lets a backend refuse at that moment what its spawn can never do (EmbLinkOS: more than 61 actions, a path past 255 bytes, fchdir, an open flag it has no bit for). - EmbLinkOS backend: getrlimit(RLIMIT_NOFILE) is the kernel's 64, and fcntl(F_SETFL) with a flag it cannot set is EINVAL, as that OS says. - getpass, with echo off; <malloc.h> (malloc_usable_size, mallinfo2, malloc_stats, valloc, pvalloc -- QuickJS sizes its heap with the first); <string.h> brings <strings.h> and <stdlib.h> <alloca.h>, as glibc's do; ARG_MAX gives way to a build's own. C++ on EmbLinkOS: make libcxx-emblinkos, and libc-emblinkos now also makes librt.a with lib/rt/unwind.c (-D__EMBCC_RT_UNWIND) -- its programs start with -nostartfiles, so nothing registers tables for libgcc's. Everything there is built with --unwind-tables: for x86_64-elf C code has none by default, and the unwinder's own frames need them or every throw ends where it starts. EMBLINKOS_ARCH=aarch64 builds the other set. Fixed on the way: - <filesystem> kept its own copy of __os_fileinfo, shorter than the backend's since the POSIX fields: every status() wrote past it (a hang). libc-backends.sh now compares libcxx's copies of the contract with it, structs and all 45 prototypes; it fails on the old copy. - 16-bit int: tz.c's 1 << 18 buffer and an unsigned-short compare that made week 0 a huge day number on AVR; malloc's 64 KB chunk constant. Tests: malloc-gnu.c (refereed against glibc), posix-process's EBADF; libc.sh 367, linux-libc-exec 84 runs, libcxx-std's struct tm is 56 bytes now, glibc's.
- newlib's limits.h guards itself with _LIBC_LIMITS_H_, the word this library's used both as its guard and to tell the compiler's half it was there. With newlib's headers first (tools/build-libstdcxx.sh puts them there) its #include_next reached ours, ours was already "done", and the compiler's half -- INT_MAX -- was never read: libsupc++ and the libstdc++ tests failed to compile. Ours has its own guard now, sets the shared word only when it is the first C library's half, and always passes the chain on. - posix-process read argc/argv before knowing it ran where a kernel passes them: on Xtensa and the ESP32-C3, whose startup promises no argc, argv[1] was a wild load. The /bin/sh behaviour is under EMBCC_TEST_HAS_FS with the rest of what needs a filesystem. Full matrix: 840/860; these were the two real failures. The others -- timeouts and one killed run, all on the busy machine -- pass alone, as do these two: libc-limits, libm-long-double, rt, sroa, stack-args, frameless, cxx-libstdcxx, cxx-libsupcxx, xtensa-exec, embsim-esp32c3, install, libc-system, big-functions, linux-libc-exec, exec-boards.
teo1747
changed the title
teo1747
marked this pull request as ready for review
October 30, 2026 04:08
The bare-metal and posix-like backends implement __os_write by calling write, __os_sbrk by calling sbrk: those names are the program's or the board's (weak defaults it replaces), newlib-style. The POSIX layer defined them too, weakly, calling the primitives back, so whichever an image linked first won -- and when proc.o came first on Xtensa and MIPS, sbrk called __os_sbrk called sbrk until the stack ran out. open, close, read, write, lseek, isatty, _exit and sbrk move to os/posix-names.c, built for Linux and EmbLinkOS only.
…'s does 604db0f moved read, write, open, close and _exit there, out of the portable half this golden builds; without it the archive it checks for closure needs them from nowhere.
This was referenced Oct 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
EmbCC's C library becomes a POSIX library, and the C library EmbLinkOS builds its userland on (
make LIBC=emblibcthere, in place of newlib).What is in it
Part 1: C and POSIX without an OS. POSIX's string, stdlib and stdio additions (memory streams,
getline,fdopen, stream locks that nest), getopt (permuting like glibc), fnmatch, libgen, search, err,<features.h>gating. The driver puts the C library's headers before the compiler's, each directory once.Part 2: POSIX's operating system.
lib/libc/os/backend.hgrows from the C seam into POSIX's: 141__os_*primitives with Linux's numbers.os/absent.incgivesENOSYSfor what a backend has not got.tests/golden/libc-backends.shholds every backend to the whole contract, and lib/libcxx's copies of it to it too.*atformfork/exec,posix_spawnwith handle-based actions,wait,system,popenTZand TZif files);strptimemmap,poll/select, sockets andgetaddrinfowith a DNS resolversysconf/uname/rlimits<malloc.h>,getpasslibcxx-emblinkos, andlibrt.awith lib/rt's unwinder.Bugs found on the way, not only in new code
<filesystem>kept a short copy of__os_fileinfo, so everystatus()overwrote the stack..eh_framewithout--unwind-tables, so the unwinder could not walk its own frames.limits.hshares our include guard; with newlib's headers first,INT_MAXvanished.write/sbrk/... and the bare-metal backend's weak hooks of the same names called each other in a loop whenever the linker met the POSIX one first (604db0f; those names are now built only for Linux and EmbLinkOS).killof another pid saidESRCHwhere the honest answer isENOSYS(mrsh hung on it).Tested
EMBCC_VERIFY=1, clean.