Monday, March 5, 2012

Erlang VM memory size and ERL_MAX_PORTS

Erlang VM has the ports to communicate with other programs and objects (files, drivers, etc.) The ports are seen as the Erlang processes. The OS environment variable ERL_MAX_PORTS sets the default value of maximum number of ports. (See the manual for the function erlang:open_port/2 for the further details.)

I discovered by accident that Erlang VM of R15B on FreeBSD/amd64 9.0 in my test environment sets the value of maximum file descriptors assigned to the OS process as the default value for ERL_MAX_PORTS. This means ERL_MAX_PORTS is set to 800000, far larger than 1024, specified in the manual. This excessive value of the ERL_MAX_PORTS leads into unnecessary memory consumption of the Erlang VM. An easy way to test this is to execute the following code on the shell:

env ERL_MAX_PORTS=[given value] erl -noinput -eval 'io:format("~p~n",[erlang:memory(system)]).' -s erlang halt

How memory consumption changes just after the Erlang VM startup, as {ERL_MAX_PORTS, memory size in bytes}: {1024, ~65M}, {102400, ~124M}, {800000, ~543M}.

So I decided to set the ERL_MAX_PORTS for my home Yaws server to 4096. This significantly reduced the memory consumption.

I suggest you to tune the value of ERL_MAX_PORTS to optimize the number of concurrently opening ports of an Erlang VM, especially when you run a server on a memory-restricted host.

Update 11-MAR-2013: on R16B, the amount of memory consumed by the ports will be significantly reduced at the Erlang VM startup.

References