Erlang VM memory size and ERL_MAX_PORTS

Erlang VM has the ports to communicate with other programs and objects (files, drivers, etc.) The ports are seen as the Erlang processes. The OS environment variable ERL_MAX_PORTS sets the default value of maximum number of ports. (See the manual for the function erlang:open_port/2 for the further details.)

I discovered by accident that Erlang VM of R15B on FreeBSD/amd64 9.0 in my test environment sets the value of maximum file descriptors assigned to the OS process as the default value for ERL_MAX_PORTS. This means ERL_MAX_PORTS is set to 800000, far larger than 1024, specified in the manual. This excessive value of the ERL_MAX_PORTS leads into unnecessary memory consumption of the Erlang VM. An easy way to test this is to execute the following code on the shell:

env ERL_MAX_PORTS=[given value] erl -noinput -eval 'io:format("~p~n",[erlang:memory(system)]).' -s erlang halt

How memory consumption changes just after the Erlang VM startup, as {ERL_MAX_PORTS, memory size in bytes}: {1024, ~65M}, {102400, ~124M}, {800000, ~543M}.

So I decided to set the ERL_MAX_PORTS for my home Yaws server to 4096. This significantly reduced the memory consumption.

I suggest you to tune the value of ERL_MAX_PORTS to optimize the number of concurrently opening ports of an Erlang VM, especially when you run a server on a memory-restricted host.

Update 11-MAR-2013: on R16B, the amount of memory consumed by the ports will be significantly reduced at the Erlang VM startup.

References

• Manual of erlang:memory/0
• [erlang-questions] About Environment variable ERL_MAX_PORTS
• [erlang-questions] Max open files and erlang memory

Kernel module functions for IPv6 on Erlang/OTP R15B

I've been reading the source code of Erlang/OTP R15B to find out how it handles the IPv6-related functions.

Erlang VM has the most functions embedded into the inet_drv linked-in driver, so programmers don't really have to worry about many things except for the address tuple format difference between IPv4 ({192, 168, 0, 1}) and IPv6 ({8193,3512,4660,22136,51966,47806,57005,48879} for 2001:db8:1234:5678:cafe:babe:dead:beef). You can find a lot of gems by reading the kernel module source code.

Here is a list of notable functions inside the kernel module for IPv6 handling on Erlang/OTP R15B. I suggest you to read these functions before writing your own ones.

(Note: this is a work-in-progress article of my Erlang Factory SF Bay Area 2012 presentation.)

%%% A partial list of IPv6 keywords in Erlang/OTP R15B source code

%%% atom "inet6" is an important keyword

inet_parse:address/1
inet_parse:ipv6_address/1
inet_parse:ipv6strict_address/1 % review RFC4291
inet_parse:ntoa/1

inet_parse:hosts/2 % internal, now ignoring "%if" suffix

-type inet:ip6_address()

inet:getifaddrs/0
inet:getifaddrs/1
inet:gethostbyaddr/1
inet:getaddrs/2 % second arg: inet6
inet:gethostbyname_string/2 % second arg: inet6

inet:i/0
inet:i/1

%%% interesting module results
%%% (no significant difference other than inet/inet6 flags):
%%%     diff --unified inet_tcp.erl inet6_tcp.erl
%%%     diff --unified inet_udp.erl inet6_udp.erl
%%%     diff --unified inet_sctp.erl inet6_sctp.erl
%%%     diff --unified inet_tcp_dist.erl inet6_tcp_dist.erl

inet_config:set_hostname/0 % internal, dependent on inet_udp:open/2

%%% inet_db.erl has tcp/udp/sctp module config params
%%% for the gen_server callbacks

inet_db:set_inet6/1
inet_db:dn_ip6_int/8 % internal, for 8 elements of an IPv6 address tuple
inet_db:make_hostent(Name, Addrs, Aliases, ?S_AAAA)
inet_db:handle_call({add_host, IPv6address, Names})
inet_db:handle_call({del_host, IPv6address})

inet_dns.erl: ?S_AAAA related lines

inet_dns.hrl: --define(T_AAAA), --define(S_AAAA)

inet_int.hrl: --define(INET_AF_INET6), --define(SCTP_FLAG_PMTUD_*),
              --define(ip6(A,B,C,D,E,F,G,H)

-type inet_res:res_option({inet6, boolean()})
-type inet_res:dns_data() % inet:ip6_address()

inet_res:gethostbyaddr/1
inet_res:gethostbyaddr_tm/2
inet_res:gethostbyname/1
inet_res:gethostbyname_tm/3
inet_res:udp_open/2 % internal (record #sock)
inet_res:udp_connect/2 % internal (record #sock)

% There will be more...