Skip to content

document shared memorysize for buffers and working memory #235

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pfent merged 1 commit into from
Nov 6, 2026
Merged

document shared memorysize for buffers and working memory #235

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 26 additions & 12 deletions content/references/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,7 @@
cat ~/.cedardb/config
# Comment only line
"verbosity" = "debug5" # Inline comment
"buffersize" = "1G" # Buffer pool size (cf. Memory Usage section)
"workmemsize" = "3G"
"memorysize" = "4G" # Total memory for buffers and working memory (cf. Memory Usage section)
"license.key" = "<your_key>"
```

Expand All @@ -50,8 +49,7 @@

```text
export VERBOSITY=debug5
export BUFFERSIZE=1G
export WORKMEMSIZE=3G
export MEMORYSIZE=4G
export LICENSE_KEY=<your_key>
```

Expand All @@ -65,7 +63,7 @@
statement, without editing the file by hand:

```sql
ALTER SYSTEM SET debug.buffersize = '8G';
ALTER SYSTEM SET debug.memorysize = '8G';
```

As with the config file, these changes only take effect after a restart.
Expand Down Expand Up @@ -129,16 +127,32 @@

## Memory Usage

By default, CedarDB uses 45% of available system memory for its buffer manager and 45% as working memory during query processing.
If you are running applications beside CedarDB on your machine (e.g., your IDE or your browser)
while doing heavy query processing on big datasets, you may want to reduce this amount.
By default, CedarDB uses 90% of available system memory (or of the cgroup memory limit in containers).
The buffer manager and the working memory for query processing share this memory dynamically.
For 10&nbsp;GB system memory, CedarDB uses up to 9&nbsp;GB for cached table data when queries need little working memory.
Memory-intensive queries can take up to three quarters of it (6.75&nbsp;GB), evicting cached data if necessary.
If queries need even more working memory, CedarDB spills to disk.

If you are running applications beside CedarDB on your machine (e.g., your IDE or your browser) while doing heavy query

Check failure on line 136 in content/references/configuration.md

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[Google.Latin] Use 'for example' instead of 'e.g.'.

Raw Output:
{"message": "[Google.Latin] Use 'for example' instead of 'e.g.'.", "location": {"path": "content/references/configuration.md", "range": {"start": {"line": 136, "column": 65}}}, "severity": "ERROR"}
processing on big datasets, you may want to reduce `memorysize`.
Usually, `memorysize` is the only memory setting you need, but you can additionally cap the buffer manager or the
working memory individually.

These settings must be set before starting CedarDB.

| **Setting Name** | **Description** | **Unit** | **Default** |
|------------------|--------------------------------------------------|-----------------------------------------|-------------------------|
| `buffersize` | Buffer manager pool size | Size with unit suffix (5G, 256M, 1024K) | 45% of available memory |
| `workmemsize` | Amount of memory to be used before spooling disk | Same as above | 45% of available memory |
| **Setting Name** | **Description** | **Unit** | **Default** |
|------------------|---------------------------------------------------------------------|-----------------------------------------|-------------------------|
| `memorysize` | Total memory shared by the buffer manager and the working memory | Size with unit suffix (5G, 256M, 1024K) | 90% of available memory |
| `buffersize` | Maximum size of the buffer manager pool | Same as above | `memorysize` |
| `workmemsize` | Maximum working memory for query processing before spilling to disk | Same as above | 75% of `memorysize` |

The buffer manager always keeps at least the memory that the working memory cannot take, i.e.,

Check failure on line 149 in content/references/configuration.md

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[Google.Latin] Use 'that is' instead of 'i.e.'.

Raw Output:
{"message": "[Google.Latin] Use 'that is' instead of 'i.e.'.", "location": {"path": "content/references/configuration.md", "range": {"start": {"line": 149, "column": 90}}}, "severity": "ERROR"}
`memorysize - workmemsize` (by default 25% of `memorysize`).

{{% callout type="info" %}}
Before release v2026-11-04, CedarDB used static, separate pools with 45% of available memory each for `buffersize` and `workmemsize`.
Existing configurations with explicit `buffersize` and `workmemsize` keep working, but now act as upper limits for a shared pool.
{{% /callout %}}

## Degree of parallelism

Expand Down
6 changes: 3 additions & 3 deletions content/references/sessions/altersystem.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Changes take effect the next time CedarDB starts.
Usage example:

```sql
ALTER SYSTEM SET debug.buffersize = '8G';
ALTER SYSTEM SET debug.memorysize = '8G';
```

The new value is only written to CedarDB's configuration file and applies after the next restart.
Expand All @@ -20,7 +20,7 @@ To inspect the current value of a setting, use [`SHOW`](/docs/references/session
To restore a setting to its default value and delete the explicitly set value from configuration file, use `RESET`:

```sql
ALTER SYSTEM RESET debug.buffersize;
ALTER SYSTEM RESET debug.memorysize;
```

Like `ALTER SYSTEM SET`, this takes effect after the next restart.
Expand All @@ -38,7 +38,7 @@ but does not affect running sessions.

`ALTER SYSTEM` accepts CedarDB server settings that are listed in `pg_settings`, such as `verbosity` and `compilationmode`.
Session settings, such as `statement_timeout`, fail with `ALTER SYSTEM of session settings not supported`.
Startup-only settings, such as `buffersize`, `workmemsize`, and `parallel`, require a `debug.` prefix.
Startup-only settings, such as `memorysize`, `buffersize`, `workmemsize`, and `parallel`, require a `debug.` prefix.

## Permissions

Expand Down
Loading