This page is a work in progress and will have more detail in the coming months. If you have specific questions, feel free to file a GitHub issue.
Materialize stores the majority of its state in-memory, and works best when the streamed data can be reduced in some way. For example, if you know that only a subset of your rows and columns are relevant for your queries, it helps to avoid materializing sources or views until you’ve expressed this to the system (we can avoid stashing that data, which can in some cases dramatically reduce the memory footprint).
To minimize the chances that Materialize runs out of memory in a production environment, we recommend you make additional memory available to Materialize via a SSD-backed swap file or swap partition.
This is particularly important in Linux and in Docker, where swap may not be automatically setup for you:
By default, a
container has no resource constraints and can use as much of a given resource as the host’s
kernel scheduler allows, unless you have overridden this with the
--memory or the
Most Linux distributions do not enable swap by default. However, you can enable it quite easily:
Create a swapfile: The general syntax is:
fallocate [-n] [-o offset] -l length filename
(ie, for a 1GB swapfile):
sudo fallocate -l <swap size> /swapfile
Make the swapfile only accessible to
chmod 600 /swapfile
Mark the file as swap space:
Enable the swap file:
We can now verify the swap is available:
(Optional): To make the swap file permenant, add this to
cat '/swapfile none swap sw 0 0' >> /etc/fstab