Sign Up
Log In
Log In
or
Sign Up
Places
All Projects
Status Monitor
Collapse sidebar
home:Ledest:erlang:23
erlang
5091-Add-erl-option-for-reading-config-from-fil...
Overview
Repositories
Revisions
Requests
Users
Attributes
Meta
File 5091-Add-erl-option-for-reading-config-from-file-descript.patch of Package erlang
From 7e88dc8e447d46e030971b2f9310c9ee1aa0223a Mon Sep 17 00:00:00 2001 From: Kjell Winblad <kjellwinblad@gmail.com> Date: Fri, 11 Dec 2020 17:36:19 +0100 Subject: [PATCH] Add erl option for reading config from file descriptor Make it possible for users to specify the option "-configfd FD" when executing the erl command. When this option is given the system will try to read and parse configuration parameters from the file descriptor. At the moment the only configuration format that is supported is the format that is supported when using the -config option. Motivation ---------- Allow users to configure an Erlang system without writing files to disk (which can be an issue in some container environments with read-only file systems). --- erts/doc/src/erl_cmd.xml | 71 ++++- erts/emulator/nifs/common/prim_file_nif.c | 75 +++-- erts/emulator/nifs/common/prim_file_nif.h | 6 + erts/emulator/nifs/unix/unix_prim_file.c | 36 ++- erts/emulator/nifs/win32/win_prim_file.c | 95 +++++- erts/etc/common/erlexec.c | 15 +- erts/preloaded/ebin/atomics.beam | Bin 3308 -> 3324 bytes erts/preloaded/ebin/counters.beam | Bin 3112 -> 3132 bytes erts/preloaded/ebin/erl_init.beam | Bin 2312 -> 2332 bytes erts/preloaded/ebin/erl_prim_loader.beam | Bin 52372 -> 52392 bytes erts/preloaded/ebin/erl_tracer.beam | Bin 2220 -> 2240 bytes erts/preloaded/ebin/erlang.beam | Bin 110620 -> 110644 bytes erts/preloaded/ebin/erts_code_purger.beam | Bin 10920 -> 10940 bytes .../erts_dirty_process_signal_handler.beam | Bin 2764 -> 2788 bytes erts/preloaded/ebin/erts_internal.beam | Bin 23388 -> 23408 bytes .../ebin/erts_literal_area_collector.beam | Bin 3268 -> 3284 bytes erts/preloaded/ebin/init.beam | Bin 50616 -> 52592 bytes erts/preloaded/ebin/persistent_term.beam | Bin 1864 -> 1884 bytes erts/preloaded/ebin/prim_buffer.beam | Bin 3616 -> 3632 bytes erts/preloaded/ebin/prim_eval.beam | Bin 1552 -> 1572 bytes erts/preloaded/ebin/prim_file.beam | Bin 28720 -> 29164 bytes erts/preloaded/ebin/prim_inet.beam | Bin 79584 -> 79640 bytes erts/preloaded/ebin/prim_net.beam | Bin 5504 -> 5520 bytes erts/preloaded/ebin/prim_socket.beam | Bin 27116 -> 27152 bytes erts/preloaded/ebin/prim_zip.beam | Bin 22348 -> 22376 bytes erts/preloaded/ebin/socket_registry.beam | Bin 5592 -> 5692 bytes erts/preloaded/ebin/zlib.beam | Bin 20208 -> 20228 bytes erts/preloaded/src/init.erl | 76 ++++- erts/preloaded/src/prim_file.erl | 13 + lib/kernel/doc/src/config.xml | 119 ++++--- lib/kernel/src/application_controller.erl | 232 +++++++++++--- lib/kernel/test/application_SUITE.erl | 293 +++++++++++++++++- .../testconfigfd1.config | 1 + .../testconfigfd2.config | 1 + 34 files changed, 903 insertions(+), 130 deletions(-) create mode 100644 lib/kernel/test/application_SUITE_data/testconfigfd1.config create mode 100644 lib/kernel/test/application_SUITE_data/testconfigfd2.config diff --git a/erts/doc/src/erl_cmd.xml b/erts/doc/src/erl_cmd.xml index 614a3262db..9e4f4f64f5 100644 --- a/erts/doc/src/erl_cmd.xml +++ b/erts/doc/src/erl_cmd.xml @@ -175,7 +175,7 @@ and lets the shell start in parallel with the rest of the system.</p> </item> - <tag><c><![CDATA[-boot File]]></c></tag> + <tag><marker id="boot" /><c><![CDATA[-boot File]]></c></tag> <item> <p>Specifies the name of the boot file, <c><![CDATA[File.boot]]></c>, which is used to start the system; see @@ -207,14 +207,79 @@ <p>Not recommended; use <seecom marker="erlc"><c>erlc</c></seecom> instead.</p> </item> - <tag><c><![CDATA[-config Config [Config]]]></c></tag> + <tag><marker id="config" /><c><![CDATA[-config Config [Config ...]]]></c></tag> <item> <p>Specifies the name of one or more configuration files, <c><![CDATA[Config.config]]></c>, which is used to configure applications; see <seefile marker="kernel:app"><c>app(4)</c></seefile> and <seeerl marker="kernel:application"> - <c>application(3)</c></seeerl>.</p> + <c>application(3)</c></seeerl>. See the documentation for + the <seefile marker="kernel:config">configuration file + format</seefile> for a description of the configuration + format and the order in which configuration parameters are + read.</p> + </item> + <tag><marker id="configfd" /><c><![CDATA[-configfd FD [FD ...]]]></c></tag> + <item> + <p>Specifies the name of one or more file descriptors (called + configuration file descriptors from here on) with + configuration data for applications; see <seefile + marker="kernel:app"><c>app(4)</c></seefile> and + <seeerl marker="kernel:application"> + <c>application(3)</c></seeerl>. See the documentation for + the <seefile marker="kernel:config">configuration file + format</seefile> for a description of the configuration + format and the order in which configuration parameters are + read. + </p> + <p> + A configuration file descriptor will be read until its end + and will then be closed. + </p> + <p> + The content of a configuration file + descriptor is stored so that it can be reused when <seemfa + marker="erts:init#restart/0"><c>init:restart/0</c></seemfa> + or <seemfa + marker="erts:init#restart/1"><c>init:restart/1</c></seemfa> + is called. + </p> + <p> + The parameter <c><![CDATA[-configfd 0]]></c> implies <c><![CDATA[-noinput]]></c>. + </p> + <note> + <p> + It is not recommended to use file descriptors 1 (standard + output), and 2 (standard error) together with + <c><![CDATA[-configfd]]></c> as these file descriptors are + typically used to print information to the console the program + is running in. + </p> + </note> + <p>Examples (Unix shell):</p> +<pre> +$ <input>erl \ +-noshell \ +-configfd 3 \ +-eval \ +'io:format("~p~n",[application:get_env(kernel, logger_level)]),erlang:halt()' 3< \ +<(echo '[{kernel, [{logger_level, warning}]}].')</input> +{ok,warning} +</pre> +<pre> +$ <input>echo '[{kernel, [{logger_level, warning}]}].' > test1.config</input> +$ <input>echo '[{kernel, [{logger_level, error}]}].' > test2.config</input> +$ <input>erl \ +-noshell \ +-configfd 3 \ +-configfd 4 \ +-eval \ +'io:format("~p~n",[application:get_env(kernel, logger_level)]),erlang:halt()' \ +3< test1.config 4< test2.config</input> +{ok,error} +</pre> + </item> <tag><marker id="connect_all"/><c><![CDATA[-connect_all false]]></c></tag> <item> diff --git a/erts/emulator/nifs/common/prim_file_nif.c b/erts/emulator/nifs/common/prim_file_nif.c index d26be2bc3e..c01ad3616c 100644 --- a/erts/emulator/nifs/common/prim_file_nif.c +++ b/erts/emulator/nifs/common/prim_file_nif.c @@ -101,11 +101,17 @@ static ERL_NIF_TERM read_file_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM a static ERL_NIF_TERM open_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM close_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]); +static ERL_NIF_TERM file_desc_to_ref_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]); + /* Internal ops */ static ERL_NIF_TERM delayed_close_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM get_handle_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM altname_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]); +/* Helper functions */ + +static ERL_NIF_TERM create_ref_or_error_tuple(ErlNifEnv *env, efile_data_t *d); + /* All file handle operations are passed through a wrapper that handles state * transitions, marking it as busy during the course of the operation, and * closing on completion if the owner died in the middle of an operation. @@ -205,6 +211,7 @@ static ErlNifFunc nif_funcs[] = { {"get_handle_nif", 1, get_handle_nif}, {"delayed_close_nif", 1, delayed_close_nif, ERL_NIF_DIRTY_JOB_IO_BOUND}, {"altname_nif", 1, altname_nif, ERL_NIF_DIRTY_JOB_IO_BOUND}, + {"file_desc_to_ref_nif", 1, file_desc_to_ref_nif, ERL_NIF_DIRTY_JOB_IO_BOUND}, }; ERL_NIF_INIT(prim_file, nif_funcs, load, NULL, upgrade, unload) @@ -472,28 +479,10 @@ static enum efile_modes_t efile_translate_modelist(ErlNifEnv *env, ERL_NIF_TERM return modes; } -static ERL_NIF_TERM open_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) { - posix_errno_t posix_errno; - efile_data_t *d; - +static ERL_NIF_TERM create_ref_or_error_tuple(ErlNifEnv *env, efile_data_t *d) { ErlNifPid controlling_process; - enum efile_modes_t modes; ERL_NIF_TERM result; - efile_path_t path; - - ASSERT(argc == 2); - if(!enif_is_list(env, argv[1])) { - return enif_make_badarg(env); - } - - modes = efile_translate_modelist(env, argv[1]); - - if((posix_errno = efile_marshal_path(env, argv[0], &path))) { - return posix_error_to_tuple(env, posix_errno); - } else if((posix_errno = efile_open(&path, modes, efile_resource_type, &d))) { - return posix_error_to_tuple(env, posix_errno); - } - + enif_self(env, &controlling_process); if(enif_monitor_process(env, d, &controlling_process, &d->monitor)) { @@ -521,6 +510,52 @@ static ERL_NIF_TERM open_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[] return enif_make_tuple2(env, am_ok, result); } +static ERL_NIF_TERM open_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) { + posix_errno_t posix_errno; + efile_data_t *d; + + enum efile_modes_t modes; + efile_path_t path; + + ASSERT(argc == 2); + if(!enif_is_list(env, argv[1])) { + return enif_make_badarg(env); + } + + modes = efile_translate_modelist(env, argv[1]); + + if((posix_errno = efile_marshal_path(env, argv[0], &path))) { + return posix_error_to_tuple(env, posix_errno); + } else if((posix_errno = efile_open(&path, modes, efile_resource_type, &d))) { + return posix_error_to_tuple(env, posix_errno); + } + + return create_ref_or_error_tuple(env, d); +} + +static ERL_NIF_TERM file_desc_to_ref_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) { + posix_errno_t posix_errno; + efile_data_t *d; + + int fd; + + ASSERT(argc == 1); + + if(!enif_is_number(env, argv[0])) { + return enif_make_badarg(env); + } + + if(!enif_get_int(env, argv[0], &fd)) { + return enif_make_badarg(env); + } + + if((posix_errno = efile_from_fd(fd, efile_resource_type, &d))) { + return posix_error_to_tuple(env, posix_errno); + } + + return create_ref_or_error_tuple(env, d); +} + static ERL_NIF_TERM close_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) { enum efile_state_t previous_state; efile_data_t *d; diff --git a/erts/emulator/nifs/common/prim_file_nif.h b/erts/emulator/nifs/common/prim_file_nif.h index 1cf5d52192..d4f18fd494 100644 --- a/erts/emulator/nifs/common/prim_file_nif.h +++ b/erts/emulator/nifs/common/prim_file_nif.h @@ -32,6 +32,8 @@ enum efile_modes_t { EFILE_MODE_DIRECTORY = (1 << 7), + EFILE_MODE_FROM_ALREADY_OPEN_FD = (1 << 8), + EFILE_MODE_READ_WRITE = EFILE_MODE_READ | EFILE_MODE_WRITE }; @@ -160,6 +162,10 @@ int efile_truncate(efile_data_t *d); posix_errno_t efile_open(const efile_path_t *path, enum efile_modes_t modes, ErlNifResourceType *nif_type, efile_data_t **d); +posix_errno_t efile_from_fd(int fd, + ErlNifResourceType *nif_type, + efile_data_t **d); + /** @brief Closes a file. The file must have entered the CLOSED state prior to * calling this to prevent double close. * diff --git a/erts/emulator/nifs/unix/unix_prim_file.c b/erts/emulator/nifs/unix/unix_prim_file.c index d110ead1e5..1efde4c0cf 100644 --- a/erts/emulator/nifs/unix/unix_prim_file.c +++ b/erts/emulator/nifs/unix/unix_prim_file.c @@ -123,13 +123,8 @@ static int open_file_is_dir(const efile_path_t *path, int fd) { return error == 0 && S_ISDIR(file_info.st_mode); } -posix_errno_t efile_open(const efile_path_t *path, enum efile_modes_t modes, - ErlNifResourceType *nif_type, efile_data_t **d) { - - int mode, flags, fd; - - flags = 0; - +static int get_flags(enum efile_modes_t modes) { + int flags = 0; if(modes & EFILE_MODE_READ && !(modes & EFILE_MODE_WRITE)) { flags |= O_RDONLY; } else if(modes & EFILE_MODE_WRITE && !(modes & EFILE_MODE_READ)) { @@ -160,6 +155,15 @@ posix_errno_t efile_open(const efile_path_t *path, enum efile_modes_t modes, flags |= O_SYNC; #endif } + return flags; +} + +posix_errno_t efile_open(const efile_path_t *path, enum efile_modes_t modes, + ErlNifResourceType *nif_type, efile_data_t **d) { + + int mode, flags, fd; + + flags = get_flags(modes); if(modes & EFILE_MODE_DIRECTORY) { mode = DIR_MODE; @@ -209,6 +213,24 @@ posix_errno_t efile_open(const efile_path_t *path, enum efile_modes_t modes, return errno; } +posix_errno_t efile_from_fd(int fd, + ErlNifResourceType *nif_type, + efile_data_t **d) { + if (fcntl(fd, F_GETFL) != -1 || errno != EBADF) { + efile_unix_t *u; + + u = (efile_unix_t*)enif_alloc_resource(nif_type, sizeof(efile_unix_t)); + u->fd = fd; + + EFILE_INIT_RESOURCE(&u->common, EFILE_MODE_FROM_ALREADY_OPEN_FD); + (*d) = &u->common; + + return 0; + } + (*d) = NULL; + return errno; +} + int efile_close(efile_data_t *d, posix_errno_t *error) { efile_unix_t *u = (efile_unix_t*)d; int fd; diff --git a/erts/emulator/nifs/win32/win_prim_file.c b/erts/emulator/nifs/win32/win_prim_file.c index a6c08c4162..d3d38049bd 100644 --- a/erts/emulator/nifs/win32/win_prim_file.c +++ b/erts/emulator/nifs/win32/win_prim_file.c @@ -84,9 +84,20 @@ typedef struct { efile_data_t common; HANDLE handle; + /* The following field is only used when the handle has been + obtained from an already existing file descriptor (i.e., + prim_file:file_desc_to_ref/2). common.modes is set to + EFILE_MODE_FROM_ALREADY_OPEN_FD when that is the case. It is + needed because we can't close using handle in that case. */ + int fd; } efile_win_t; static int windows_to_posix_errno(DWORD last_error); +static void tmp_nop_invalid_parameter_handler(const wchar_t* expression, + const wchar_t* function, + const wchar_t* file, + unsigned int line, + uintptr_t pReserved); static int has_invalid_null_termination(const ErlNifBinary *path) { const WCHAR *null_pos, *end_pos; @@ -491,10 +502,59 @@ posix_errno_t efile_open(const efile_path_t *path, enum efile_modes_t modes, } } +static void tmp_nop_invalid_parameter_handler(const wchar_t* expression, + const wchar_t* function, + const wchar_t* file, + unsigned int line, + uintptr_t pReserved) { + (void)expression; + (void)function; + (void)file; + (void)line; + (void)pReserved; +} + +posix_errno_t efile_from_fd(int fd, + ErlNifResourceType *nif_type, + efile_data_t **d) { + HANDLE handle; + + _invalid_parameter_handler old_handler; + + /* Temporarily disable the parameter handler so we don't crash */ + old_handler = + _set_thread_local_invalid_parameter_handler(tmp_nop_invalid_parameter_handler); + + handle = (HANDLE)_get_osfhandle(fd); + + /* Enable old parameter handler again */ + _set_thread_local_invalid_parameter_handler(old_handler); + + if(handle != INVALID_HANDLE_VALUE && handle != ((HANDLE)-2)) { + efile_win_t *w; + + w = (efile_win_t*)enif_alloc_resource(nif_type, sizeof(efile_win_t)); + w->handle = handle; + w->fd = fd; + + EFILE_INIT_RESOURCE(&w->common, EFILE_MODE_FROM_ALREADY_OPEN_FD); + (*d) = &w->common; + + return 0; + } else { + return EBADF; + } +} + int efile_close(efile_data_t *d, posix_errno_t *error) { efile_win_t *w = (efile_win_t*)d; HANDLE handle; - + int from_already_open_fd = + w->common.modes & EFILE_MODE_FROM_ALREADY_OPEN_FD; + int fd; + if (from_already_open_fd) { + fd = w->fd; + } ASSERT(enif_thread_type() == ERL_NIF_THR_DIRTY_IO_SCHEDULER); ASSERT(erts_atomic32_read_nob(&d->state) == EFILE_STATE_CLOSED); ASSERT(w->handle != INVALID_HANDLE_VALUE); @@ -504,7 +564,12 @@ int efile_close(efile_data_t *d, posix_errno_t *error) { enif_release_resource(d); - if(!CloseHandle(handle)) { + if(from_already_open_fd) { + if(_close(fd) == -1) { + *error = EBADF; + return 0; + } + } else if(!CloseHandle(handle)) { *error = windows_to_posix_errno(GetLastError()); return 0; } @@ -558,7 +623,8 @@ static void shift_iov(SysIOVec **iov, int *iovlen, DWORD shift) { typedef BOOL (WINAPI *io_op_t)(HANDLE, LPVOID, DWORD, LPDWORD, LPOVERLAPPED); static Sint64 internal_sync_io(efile_win_t *w, io_op_t operation, - SysIOVec *iov, int iovlen, OVERLAPPED *overlapped) { + SysIOVec *iov, int iovlen, OVERLAPPED *overlapped, + int is_read) { Sint64 bytes_processed = 0; @@ -574,10 +640,17 @@ static Sint64 internal_sync_io(efile_win_t *w, io_op_t operation, &block_bytes_processed, overlapped); last_error = GetLastError(); - if(!succeeded && (last_error != ERROR_HANDLE_EOF)) { - w->common.posix_errno = windows_to_posix_errno(last_error); - return -1; - } else if(block_bytes_processed == 0) { + if(is_read && !succeeded) { + if(last_error == ERROR_BROKEN_PIPE) { + /* Pipes gives ERROR_BROKEN_PIPE instead of EOF when the + write end has been closed */ + return bytes_processed; + } else if(last_error != ERROR_HANDLE_EOF) { + w->common.posix_errno = windows_to_posix_errno(last_error); + return -1; + } + } + if(block_bytes_processed == 0) { /* EOF */ return bytes_processed; } @@ -595,7 +668,7 @@ static Sint64 internal_sync_io(efile_win_t *w, io_op_t operation, Sint64 efile_readv(efile_data_t *d, SysIOVec *iov, int iovlen) { efile_win_t *w = (efile_win_t*)d; - return internal_sync_io(w, ReadFile, iov, iovlen, NULL); + return internal_sync_io(w, ReadFile, iov, iovlen, NULL, 1); } Sint64 efile_writev(efile_data_t *d, SysIOVec *iov, int iovlen) { @@ -614,7 +687,7 @@ Sint64 efile_writev(efile_data_t *d, SysIOVec *iov, int iovlen) { overlapped = NULL; } - return internal_sync_io(w, WriteFile, iov, iovlen, overlapped); + return internal_sync_io(w, WriteFile, iov, iovlen, overlapped, 0); } Sint64 efile_preadv(efile_data_t *d, Sint64 offset, SysIOVec *iov, int iovlen) { @@ -626,7 +699,7 @@ Sint64 efile_preadv(efile_data_t *d, Sint64 offset, SysIOVec *iov, int iovlen) { overlapped.OffsetHigh = (offset >> 32) & 0xFFFFFFFF; overlapped.Offset = offset & 0xFFFFFFFF; - return internal_sync_io(w, ReadFile, iov, iovlen, &overlapped); + return internal_sync_io(w, ReadFile, iov, iovlen, &overlapped, 1); } Sint64 efile_pwritev(efile_data_t *d, Sint64 offset, SysIOVec *iov, int iovlen) { @@ -638,7 +711,7 @@ Sint64 efile_pwritev(efile_data_t *d, Sint64 offset, SysIOVec *iov, int iovlen) overlapped.OffsetHigh = (offset >> 32) & 0xFFFFFFFF; overlapped.Offset = offset & 0xFFFFFFFF; - return internal_sync_io(w, WriteFile, iov, iovlen, &overlapped); + return internal_sync_io(w, WriteFile, iov, iovlen, &overlapped, 0); } int efile_seek(efile_data_t *d, enum efile_seek_t seek, Sint64 offset, Sint64 *new_position) { diff --git a/erts/etc/common/erlexec.c b/erts/etc/common/erlexec.c index c24c066e9e..e9d18964d6 100644 --- a/erts/etc/common/erlexec.c +++ b/erts/etc/common/erlexec.c @@ -18,11 +18,6 @@ * %CopyrightEnd% */ -/* - * This is a C version of the erl.exec Bourne shell script, including - * additions required for Windows NT. - */ - #include "etc_common.h" #include "erl_driver.h" @@ -651,6 +646,16 @@ int main(int argc, char **argv) } while ((i+1) < argc && argv[i+1][0] != '-' && argv[i+1][0] != '+'); } #endif + else if (strcmp(argv[i], "-configfd") == 0) { + if (i+1 >= argc) + usage("-configfd"); + if ( strcmp(argv[i+1], "0") != 0 ) { + add_arg(argv[i]); + } else { + add_args("-noshell", "-noinput", NULL); + add_arg(argv[i]); + } + } else { add_arg(argv[i]); } diff --git a/erts/preloaded/src/init.erl b/erts/preloaded/src/init.erl index 6aa73587fe..a5ead55d47 100644 --- a/erts/preloaded/src/init.erl +++ b/erts/preloaded/src/init.erl @@ -50,7 +50,7 @@ -export([restart/1,restart/0,reboot/0,stop/0,stop/1, get_status/0,boot/1,get_arguments/0,get_plain_arguments/0, - get_argument/1,script_id/0]). + get_argument/1,script_id/0,script_name/0]). %% for the on_load functionality; not for general use -export([run_on_load_handlers/0]). @@ -58,7 +58,8 @@ %% internal exports -export([fetch_loaded/0,ensure_loaded/1,make_permanent/2, notify_when_started/1,wait_until_started/0, - objfile_extension/0, archive_extension/0,code_path_choice/0]). + objfile_extension/0, archive_extension/0,code_path_choice/0, + get_configfd/1, set_configfd/2]). -include_lib("kernel/include/file.hrl"). @@ -73,7 +74,9 @@ status = {starting, starting} :: {internal_status(), term()}, script_id = [], loaded = [], - subscribed = []}). + subscribed = [], + configfdid_to_config = #{} :: #{} | #{integer() := term()}, + script_name = {[],[]} :: {string(), string()}}). -type state() :: #state{}. %% Data for eval_script/2. @@ -95,6 +98,15 @@ debug(false, _) -> ok; debug(_, T) -> erlang:display(T). +-spec get_configfd(integer()) -> none | term(). +get_configfd(ConfigFdId) -> + request({get_configfd, ConfigFdId}). + +-spec set_configfd(integer(), term()) -> 'ok'. +set_configfd(ConfigFdId, Config) -> + request({set_configfd, ConfigFdId, Config}), + ok. + -spec get_arguments() -> Flags when Flags :: [{Flag :: atom(), Values :: [string()]}]. get_arguments() -> @@ -116,6 +128,30 @@ get_argument(Arg) -> script_id() -> request(script_id). +%% Returns the path to the boot script. The path can only be relative +%% (i.e., not absolute) if prim_file:get_cwd() returned an error tuple +%% during boot. filename:absname/2 is not available during boot so we +%% construct the path here instead of during boot +-spec script_name() -> string(). +script_name() -> + {BootCWD, ScriptPath} = request(get_script_name), + case BootCWD of + [] -> + ScriptPath; + _ -> + %% Makes the path absolute if ScriptPath is a relative + %% path + filename:absname(ScriptPath, BootCWD) + end. + +%% Module internal function to set the script name during boot +-spec set_script_name(BootCurrentWorkingDir, ScriptPath) -> 'ok' when + BootCurrentWorkingDir :: string(), + ScriptPath :: string(). +set_script_name(BootCurrentWorkingDir, ScriptPath) -> + request({set_script_name, {BootCurrentWorkingDir, ScriptPath}}), + ok. + bs2as(L0) when is_list(L0) -> map(fun b2a/1, L0); bs2as(L) -> @@ -459,7 +495,9 @@ do_handle_msg(Msg,State) -> status = Status, script_id = Sid, args = Args, - subscribed = Subscribed} = State, + subscribed = Subscribed, + configfdid_to_config = ConfigFdIdToConfig, + script_name = ScriptName} = State, case Msg of {From,get_plain_arguments} -> From ! {init,Args}; @@ -485,6 +523,24 @@ do_handle_msg(Msg,State) -> end; {From, {ensure_loaded, _}} -> From ! {init, not_allowed}; + {From, {get_configfd, ConfigFdId}} -> + case ConfigFdIdToConfig of + #{ConfigFdId := Config} -> + From ! {init, Config}; + _ -> + From ! {init, none} + end; + {From, {set_configfd, ConfigFdId, Config}} -> + From ! {init, ok}, + NewConfigFdIdToConfig = ConfigFdIdToConfig#{ConfigFdId => Config}, + NewState = State#state{configfdid_to_config = NewConfigFdIdToConfig}, + {new_state, NewState}; + {From, get_script_name} -> + From ! {init, ScriptName}; + {From, {set_script_name, NewScriptName}} -> + From ! {init, ok}, + NewState = State#state{script_name = NewScriptName}, + {new_state, NewState}; X -> case whereis(user) of %% io_requests may end up here from various processes that have @@ -865,6 +921,7 @@ path_flags(Flags) -> get_boot(BootFile0,Root) -> BootFile = BootFile0 ++ ".boot", + case get_boot(BootFile) of {ok, CmdList} -> CmdList; @@ -888,7 +945,16 @@ get_boot(BootFile) -> case binary_to_term(Bin) of {script,Id,CmdList} when is_list(CmdList) -> init ! {self(),{script_id,Id}}, % ;-) - {ok, CmdList}; + CWD = + case prim_file:get_cwd() of + {ok, TheCWD} -> TheCWD; + {error, _} -> [] + end, + %% The filename module is not available during + %% boot so we call filename:absname/2 in + %% init:script_name/0 instead. + set_script_name(CWD, BootFile), + {ok, CmdList}; _ -> error end; diff --git a/erts/preloaded/src/prim_file.erl b/erts/preloaded/src/prim_file.erl index 046d6ffbad..3a6a573b89 100644 --- a/erts/preloaded/src/prim_file.erl +++ b/erts/preloaded/src/prim_file.erl @@ -27,6 +27,9 @@ pread/2, pread/3, pwrite/2, pwrite/3]). %% OTP internal. + +-export([file_desc_to_ref/2]). + -export([ipread_s32bu_p32bu/3, sendfile/8, altname/1, get_handle/1]). -export([read_file/1, write_file/2]). @@ -122,6 +125,14 @@ open(Name, Modes) -> error:badarg -> {error, badarg} end. +file_desc_to_ref(FileDescriptorId, Modes) -> + try file_desc_to_ref_nif(FileDescriptorId) of + {ok, Ref} -> {ok, make_fd(Ref, Modes)}; + {error, Reason} -> {error, Reason} + catch + error:badarg -> {error, badarg} + end. + make_fd(FRef, Modes) -> #file_descriptor{module = ?MODULE, data = build_fd_data(FRef, Modes) }. @@ -474,6 +485,8 @@ fill_fd_option_map([_Ignored | Modes], Map) -> open_nif(_Name, _Modes) -> erlang:nif_error(undef). +file_desc_to_ref_nif(_FD) -> + erlang:nif_error(undef). close_nif(_FileRef) -> erlang:nif_error(undef). read_nif(_FileRef, _Size) -> diff --git a/lib/kernel/doc/src/config.xml b/lib/kernel/doc/src/config.xml index adb3241306..ac354b5d1f 100644 --- a/lib/kernel/doc/src/config.xml +++ b/lib/kernel/doc/src/config.xml @@ -32,15 +32,27 @@ <filesummary>Configuration file.</filesummary> <description> <p>A <em>configuration file</em> contains values for configuration - parameters for the applications in the system. The <c>erl</c> - command-line argument <c>-config Name</c> tells the system to use - data in the system configuration file <c>Name.config</c>.</p> - <p>Configuration parameter values in the configuration file - override the values in the application resource files (see - <seefile marker="app"><c>app(4)</c></seefile>). - The values in the configuration file can be - overridden by command-line flags (see - <seecom marker="erts:erl"><c>erts:erl(1)</c></seecom>).</p> + parameters for the applications in the system. The <c>erl</c> + command-line argument <seecom marker="erts:erl#config"><c>-config + Name</c></seecom> tells the system to use data in the system configuration + file <c>Name.config</c>.</p> + <p>The erl command-line argument <seecom + marker="erts:erl#configfd"><c>-configfd</c></seecom> works the + same way as the <c>-config</c> option but specifies a file + descriptor to read configuration data from instead of a file.</p> + <p>The configuration data from configuration files and file + descriptors are read in the same order as they are given on the + command line. For example, <c>erl -config a -configfd 3 -config b + -configfd 4</c> would cause the system to read configuration data + in the following order <c>a.config</c>, file descriptor <c>3</c>, + <c>b.config</c>, and file descriptor <c>4</c>. If a configuration + parameter is specified more than once in the given files and file + descriptors, the last one overrides the previous ones.</p> + <p>Configuration parameter values in a configuration file or file + descriptor override the values in the application resource files + (see <seefile marker="app"><c>app(4)</c></seefile>). The values in + the configuration file are always overridden by command-line flags + (see <seecom marker="erts:erl"><c>erts:erl(1)</c></seecom>).</p> <p>The value of a configuration parameter is retrieved by calling <c>application:get_env/1,2</c>.</p> </description> @@ -74,50 +86,79 @@ root installation directory and <c>Vsn</c> is the release version.</p> <p>Release handling relies on this assumption. When installing a new release version, the new <c>sys.config</c> is read and used - to update the application configurations.</p> + to update the application's configurations.</p> <p>This means that specifying another <c>.config</c> file, or more - <c>.config</c> files, leads to inconsistent update of application - configurations. There is, however, a syntax for - <c>sys.config</c> that allows pointing out other - <c>.config</c> files:</p> + <c>.config</c> files, leads to an inconsistent update of application + configurations. There is, however, a way to point out other config + files from a <c>sys.config</c>. How to do this is described in + the next section.</p> + </section> + + <section> + <title>Including Files from sys.config and -configfd Configurations</title> + + <p>There is a way to include other configuration files from a + <c>sys.config</c> file and from a configuration that comes + from a file descriptor that has been pointed out with the <seecom + marker="erts:erl#configfd"><c>-configfd</c></seecom> command-line + argument.</p> + + <p>The syntax for including files can be described by the + <seeguide marker="system/reference_manual:typespec">Erlang type + language</seeguide> like this:</p> <code type="none"> -[{Application, [{Par, Val}]} | File].</code> +[{Application, [{Par, Val}]} | IncludeFile].</code> <taglist> - <tag><c>File = string()</c></tag> - <item>Name of another <c>.config</c> file. - Extension <c>.config</c> can be omitted. It is - recommended to use absolute paths. If a relative path is used, - <c>File</c> is searched, first, relative from <c>sys.config</c> directory, then relative - to the current working directory of the emulator, for backward compatibility. - This allow to use a <c>sys.config</c> pointing out other <c>.config</c> files in a release - or in a node started manually using <c>-config ...</c> with same result whatever - the current working directory. + <tag><c>IncludeFile = string()</c></tag> + <item>Name of a <c>.config</c> file. The extension + <c>.config</c> can be omitted. It is recommended to use absolute + paths. If a relative path is used in a <c>sys.config</c>, + <c>IncludeFile</c> is searched, first, relative to the + <c>sys.config</c> directory, then relative to the current + working directory of the emulator. If a relative path is used + in a <c>-configfd</c> configuration, <c>IncludeFile</c> is + searched, first, relative to the dictionary containing the + <seefile marker="sasl:script">boot script</seefile> (see also + the <seecom marker="erts:erl#boot"><c>-boot</c></seecom> + command-line argument) for the emulator, then relative to the + current working directory of the emulator. This makes it + possible to use <c>sys.config</c> for pointing out other + <c>.config</c> files in a release or in a node started manually + using <c>-config</c> or <c>-configfd</c> with the same result + whatever the current working directory is. </item> </taglist> - <p>When traversing the contents of <c>sys.config</c> and a filename - is encountered, its contents are read and merged with the result - so far. When an application configuration tuple - <c>{Application, Env}</c> is found, it is merged with the result - so far. Merging means that new parameters are added and existing - parameter values overwritten.</p> + <p>When traversing the contents of a <c>sys.config</c> or a + <c>-configfd</c> configuration and a filename is encountered, its + contents are read and merged with the result so far. When an + application configuration tuple <c>{Application, Env}</c> is + found, it is merged with the result so far. Merging means that new + parameters are added and existing parameter values + are overwritten.</p> <p><em>Example:</em></p> <code type="none"> sys.config: -[{myapp,[{par1,val1},{par2,val2}]}, - "/home/user/myconfig"]. +["/home/user/myconfig1" + {myapp,[{par1,val1},{par2,val2}]}, + "/home/user/myconfig2"]. + +myconfig1.config: + +[{myapp,[{par0,val0},{par1,val0},{par2,val0}]}]. -myconfig.config: +myconfig2.config: [{myapp,[{par2,val3},{par3,val4}]}].</code> <p>This yields the following environment for <c>myapp</c>:</p> <code type="none"> -[{par1,val1},{par2,val3},{par3,val4}]</code> - <p>The behavior if a file specified in <c>sys.config</c> does not - exist, or is erroneous, is backwards compatible. - Starting the runtime system will fail. Installing a new release - version will not fail, but an error message is returned and - the erroneous file is ignored.</p> +[{par0,val0},{par1,val1},{par2,val3},{par3,val4}]</code> + <p>The run-time system will abort before staring up if an include file + specified in <c>sys.config</c> or a <c>-configfd</c> configuration + does not exist, or is erroneous. However, installing a new release + version will not fail if there is an error while loading an + include file, but an error message is returned and the erroneous + file is ignored.</p> </section> <section> diff --git a/lib/kernel/src/application_controller.erl b/lib/kernel/src/application_controller.erl index cb65574d81..2516fbae74 100644 --- a/lib/kernel/src/application_controller.erl +++ b/lib/kernel/src/application_controller.erl @@ -524,6 +524,12 @@ init(Init, Kernel) -> lists:flatten(io_lib:format("error in config file " "~tp (~w): ~ts", [File, Line, Str])), + Init ! {ack, self(), {error, to_string(ReasonStr)}}; + {error, {file_descriptor, FDString, Line, Str}} -> + ReasonStr = + lists:flatten(io_lib:format("error in config read from file descriptor " + "~tp (~w): ~ts", + [FDString, Line, Str])), Init ! {ack, self(), {error, to_string(ReasonStr)}} end. @@ -1799,48 +1805,109 @@ do_config_diff([{Env, Value} | AppEnvNow], AppEnvBefore, {Changed, New}) -> end. +conf_param_to_conf({config, FileName}) -> + BFName = filename:basename(FileName,".config"), + FName = filename:join(filename:dirname(FileName), + BFName ++ ".config"), + case load_file(FName) of + {ok, NewEnv} -> + %% OTP-4867 sys.config may now contain names of other + %% .config files as well as configuration parameters. + %% Therefore read and merge contents. + if + BFName =:= "sys" -> + DName = filename:dirname(FName), + {ok, SysEnv, Errors} = + check_conf_sys(NewEnv, [], [], DName), + %% Report first error, if any, and terminate + %% (backwards compatible behaviour) + case Errors of + [] -> + SysEnv; + [{error, {SysFName, Line, Str}}|_] -> + throw({error, {SysFName, Line, Str}}) + end; + true -> + NewEnv + end; + {error, {Line, _Mod, Str}} -> + throw({error, {FName, Line, Str}}) + end; +conf_param_to_conf({configfd, FileDescStrP}) -> + FileDescStr = unicode:characters_to_nfc_list(FileDescStrP), + IsDigit = fun(C) -> lists:member(C, "0123456789") end, + {FdString, FileDescType} = lists:splitwith(IsDigit, FileDescStr), + FileDesc = + try list_to_integer(FdString) + catch + error:badarg -> + throw({error, {file_descriptor, + FileDescStr, + invalid_file_desc, + "The given file descriptor has incorrect format. " + "The format should be \"FileDescId[.FileType]\". " + "Examples: 3 or 3.config"}}) + end, + case lists:member(FileDescType, [".config", ""]) of + false -> + throw({error, {file_descriptor, + FileDescStr, + invalid_file_desc, + io_lib:format("Cannot parse file descriptor of type: ~ts", + [FileDescType])}}); + true -> ok + end, + case load_file_descriptor(FileDesc) of + {ok, NewEnv} -> + %% Load config parameters from included config files + BootScript = init:script_name(), + DName = filename:dirname(BootScript), + {ok, SysEnv, Errors} = + check_conf_sys(NewEnv, [], [], DName), + case Errors of + [] -> + SysEnv; + [{error, {SysFName, Line, Str}}|_] -> + throw({error, {SysFName, Line, Str}}) + end; + {error, {Line, _Mod, Str}} -> + throw({error, {file_descriptor, + FileDescStr, + Line, + Str}}) + end. + +config_param_to_list({Type, [First | _] = ConfigVals}) + when + is_list(First), + Type =:= config orelse Type =:= configfd -> + [{Type, Val} || Val <- ConfigVals]; +config_param_to_list({config, Val}) -> + [{config, Val}]; +config_param_to_list({configfd, Val}) -> + [{configfd, Val}]; +config_param_to_list(_) -> + []. + + + %%----------------------------------------------------------------- %% Read the .config files. %%----------------------------------------------------------------- check_conf() -> - case init:get_argument(config) of - {ok, Files} -> - {ok, lists:foldl( - fun(File, Env) -> - BFName = filename:basename(File,".config"), - FName = filename:join(filename:dirname(File), - BFName ++ ".config"), - case load_file(FName) of - {ok, NewEnv} -> - %% OTP-4867 - %% sys.config may now contain names of - %% other .config files as well as - %% configuration parameters. - %% Therefore read and merge contents. - if - BFName =:= "sys" -> - DName = filename:dirname(FName), - {ok, SysEnv, Errors} = - check_conf_sys(NewEnv, [], [], DName), - - %% Report first error, if any, and - %% terminate - %% (backwards compatible behaviour) - case Errors of - [] -> - merge_env(Env, SysEnv); - [{error, {SysFName, Line, Str}}|_] -> - throw({error, {SysFName, Line, Str}}) - end; - true -> - merge_env(Env, NewEnv) - end; - {error, {Line, _Mod, Str}} -> - throw({error, {FName, Line, Str}}) - end - end, [], lists:append(Files))}; - _ -> {ok, []} - end. + ConfigParameters = + lists:flatmap( + fun config_param_to_list/1, + init:get_arguments()), + MergedConf = + lists:foldl(fun (ConfigParameter, Env) -> + NewEnv = conf_param_to_conf(ConfigParameter), + merge_env(Env, NewEnv) + end, + [], + ConfigParameters), + {ok, MergedConf}. + check_conf_sys(Env) -> check_conf_sys(Env, [], [], []). @@ -1886,6 +1953,40 @@ load_file(File) -> {error, {none, open_file, "configuration file not found"}} end. +load_file_descriptor(FileDescriptorId) -> + %% We do not want to read from the same file descriptor again if + %% init:restart is called so we use the old config obtained from + %% the file descriptor + case init:get_configfd(FileDescriptorId) of + none -> + WarningIntervalMs = 20000, % 20 seconds + MaxConfSizeBytes = 1024*1024*128, % 134 MB + case read_fd_until_end_and_close(FileDescriptorId, + WarningIntervalMs, + MaxConfSizeBytes) of + {ok, Bin} -> + %% Make sure that there is some whitespace at the end of the string + %% (so that reading a file with no NL following the "." will work). + case file_binary_to_list(Bin) of + {ok, String} -> + case scan_file(String ++ " ") of + {ok, Config} -> + init:set_configfd(FileDescriptorId, Config), + {ok, Config}; + Error -> + Error + end; + error -> + {error, {none, scan_file, "bad encoding"}} + end; + {error, Reason} -> + {error, {none, + read_from_file_descriptor, + io_lib:format("Could not read from file descriptor: ~s", [Reason])}} + end; + Config -> {ok, Config} + end. + scan_file(Str) -> case erl_scan:tokens([], Str, 1) of {done, {ok, Tokens, _}, Left} -> @@ -1910,6 +2011,61 @@ scan_file(Str) -> {error, {none, load_file, "no ending <dot> found"}} end. +read_fd_until_end_and_close(FileDescriptorId, + TimeBetweenWarningsMilliseconds, + MaxSizeBytes) -> + ReadLimit = 1024, + Read = + fun Read(Ref, _Fd, _Data, DataSize) when DataSize > MaxSizeBytes -> + Reason = io_lib:format("Max size ~w bytes exceeded", + [MaxSizeBytes]), + {Ref, error, Reason}; + Read(Ref, Fd, Data, DataSize) -> + case file:read(Fd, ReadLimit) of + eof -> + {Ref, ok, erlang:iolist_to_binary(Data)}; + {ok, Bin} -> + Read(Ref, Fd, [Data, Bin], DataSize + byte_size(Bin)); + {error, Reason} -> + {Ref, error, Reason} + end + end, + Receiver = self(), + Ref = make_ref(), + Reader = + spawn( + fun() -> + case prim_file:file_desc_to_ref(FileDescriptorId, [read]) of + {ok, Fd} -> + Res = Read(Ref, Fd, [], 0), + prim_file:close(Fd), + Receiver ! Res; + {error, _} -> + Receiver ! {Ref, + error, + "Invalid file descriptor"} + end + end), + Reader ! {reader_ref, Ref}, + (fun GetResult() -> + receive + {Ref, ok, Data} -> + {ok, erlang:iolist_to_binary(Data)}; + {Ref, error, Reason} -> + {error, Reason} + after TimeBetweenWarningsMilliseconds -> + Msg = + io_lib:format( + "Slow -configfd file descriptor ~p. " + "The system will continue to read from " + "the file descriptor and will be blocked " + "until end of file is received.", + [FileDescriptorId]), + ?LOG_WARNING(Msg), + GetResult() + end + end)(). + only_ws([C|Cs]) when C =< $\s -> only_ws(Cs); only_ws([$%|Cs]) -> only_ws(strip_comment(Cs)); % handle comment only_ws([_|_]) -> false; diff --git a/lib/kernel/test/application_SUITE.erl b/lib/kernel/test/application_SUITE.erl index 65c691b630..4af0c942b5 100644 --- a/lib/kernel/test/application_SUITE.erl +++ b/lib/kernel/test/application_SUITE.erl @@ -40,7 +40,8 @@ ensure_started/1, ensure_all_started/1, shutdown_func/1, do_shutdown/1, shutdown_timeout/1, shutdown_deadlock/1, config_relative_paths/1, handle_many_config_files/1, - format_log_1/1, format_log_2/1]). + format_log_1/1, format_log_2/1, + configfd_bash/1, configfd_port_program/1]). -define(TESTCASE, testcase_name). -define(testcase, proplists:get_value(?TESTCASE, Config)). @@ -60,7 +61,8 @@ all() -> set_env, set_env_persistent, set_env_errors, {group, distr_changed}, config_change, shutdown_func, shutdown_timeout, shutdown_deadlock, config_relative_paths, optional_applications, - persistent_env, handle_many_config_files, format_log_1, format_log_2]. + persistent_env, handle_many_config_files, format_log_1, format_log_2, + configfd_bash, configfd_port_program]. groups() -> [{reported_bugs, [], @@ -1949,6 +1951,293 @@ distr_changed_tc2(Conf) when is_list(Conf) -> ok. +get_relative_path(AbsolutePath, RelativeTo) -> + AbsolutePathList = filename:split(AbsolutePath), + RelativeToList = filename:split(RelativeTo), + CommonPath = + (fun GetCommonPath([], _, Acc) -> + lists:reverse(Acc); + GetCommonPath(_, [], Acc) -> + lists:reverse(Acc); + GetCommonPath([A | _], [B | _], Acc) + when A =/= B -> + lists:reverse(Acc); + GetCommonPath([N | Rest1], [N | Rest2], Acc) -> + GetCommonPath(Rest1, Rest2, [N | Acc]) + end)(AbsolutePathList, RelativeToList, []), + CommonPathLength = length(CommonPath), + RelPathEnd = lists:nthtail(CommonPathLength, AbsolutePathList), + NrOfDowns = length(RelativeToList) - CommonPathLength, + filename:join(lists:duplicate(NrOfDowns, "..") ++ RelPathEnd). + +do_configfd_test_port_program(ErlProgram) -> + PrintLogLevelString = + "io_lib:format(\"~p\",[element(2, application:get_env(kernel, logger_level))])", + DataDir = filename:join(filename:dirname(code:which(?MODULE)), "application_SUITE_data"), + OutFilePath = filename:join(DataDir, "do_configfd_test_port.out"), + ToEval = + lists:flatten( + io_lib:format("file:write_file(\"~s\", ~s),erlang:halt()", + [OutFilePath, + PrintLogLevelString])), + Port = erlang:open_port( + {spawn_executable, ErlProgram}, + [{args, ["-configfd", + "0", + "-eval", + ToEval]}, + use_stdio, + stderr_to_stdout]), + Port ! {self(),{command,"[{kernel, [{logger_level, warning}]}]."}}, + Port ! {self(),close}, + (fun Read() -> + receive + {Port,closed} -> ok; + {Port, Message} -> + io:format("Got unexpected message: ~p", Message), + Read() + end + end)(), + %% Check that the config file was read correctly in the port + %% program + ok = + (fun TryRead(0) -> + cannot_find_file; + TryRead(TriesLeft) -> + case file:read_file(OutFilePath) of + {ok, <<"warning">>} -> ok; + Error -> + %% It might take some time for the file to be + %% written to disk after we have closed the + %% config file descriptor + io:format("INFO: File not written yet, trying again (~p)", [Error]), + timer:sleep(250), + TryRead(TriesLeft -1) + end + end)(40), + ok = file:delete(OutFilePath). + +quote_sub_strings(String) -> + lists:flatmap( + fun($") -> + "\\\""; + (C) -> [C] + end, + lists:flatten(String)). + +do_configfd_test_bash() -> + DataDir = filename:join(filename:dirname(code:which(?MODULE)), "application_SUITE_data"), + TestConfigPath1 = filename:join(DataDir, "testconfigfd1.config"), + TestConfigPath2 = filename:join(DataDir, "testconfigfd2.config"), + RunInBash = + fun(String) -> + Command = + lists:flatten(io_lib:format("bash -c \"~s\"", + [quote_sub_strings(String)])), + Res = os:cmd(Command), + io:format("Command:~n"), + io:format("~s~n", [Command]), + io:format("Result:~n"), + io:format("~s~n", [Res]), + Res + end, + PrintLogLevelString = + "io:format(\"~p\",[element(2, application:get_env(kernel, logger_level))])", + %% Single config from file descriptor + "warning" = + RunInBash( + io_lib:format( + "erl " + "-noshell " + "-configfd 3 " + "-eval " + "'~s,erlang:halt()' " + "3< \"~s\"", + [PrintLogLevelString, + TestConfigPath1])), + %% Single config with .config sufix + "warning" = + RunInBash( + io_lib:format( + "erl " + "-noshell " + "-configfd 3.config " + "-eval " + "'~s,erlang:halt()' " + "3< \"~s\"", + [PrintLogLevelString, + TestConfigPath1])), + %% Single config from file descriptor (stdin) + %% This should automatically turn on -noinput + "warning" = + RunInBash( + io_lib:format( + "erl " + "-configfd 0 " + "-eval " + "'~s,erlang:halt()' " + "0< \"~s\"", + [PrintLogLevelString, + TestConfigPath1])), + %% Configs from two different file descriptors + "error" = + RunInBash( + io_lib:format( + "erl " + "-noshell " + "-configfd 4 " + "-configfd 5 " + "-eval " + "'~s,erlang:halt()' " + "4< \"~s\" " + "5< \"~s\" ", + [PrintLogLevelString, + TestConfigPath1, + TestConfigPath2])), + %% Configs from two different file descriptors single parameter + "error" = + RunInBash( + io_lib:format( + "erl " + "-noshell " + "-configfd 4 5 " + "-eval " + "'~s,erlang:halt()' " + "4< \"~s\" " + "5< \"~s\" ", + [PrintLogLevelString, + TestConfigPath1, + TestConfigPath2])), + lists:foreach( + fun(Path) -> + "warning" = + RunInBash( + io_lib:format( + "erl " + "-noshell " + "-configfd 3 " + "-eval " + "'~s,erlang:halt()' " + "3< <(echo '[\"~s\"].') ", + [PrintLogLevelString, + Path])) + end, + [%% Absolute paths + TestConfigPath1, + %% Without suffix + filename:join(filename:dirname(TestConfigPath1), + filename:basename(TestConfigPath1, ".config")), + %% Relative To CWD + get_relative_path(TestConfigPath1, erlang:element(2, file:get_cwd())) + ] ++ + case filename:pathtype(init:script_name()) of + absolute -> + %% Relative to the boot script directory + [get_relative_path(TestConfigPath1, + filename:dirname(init:script_name()))]; + _ -> + io:format("Skip include relative to boot script dir test. " + "init:script_name() returned a relative path." + "init:script_name() can return a relative path if" + "prim_file:get_pwd() fails during boot."), + [] + end + ), + %% init:restart() should work + "errorerror" = + RunInBash( + io_lib:format( + "erl " + "-noshell " + "-configfd 3 " + "-eval " + "'~s,init:restart(),~s,erlang:halt()' " + "3< \"~s\" ", + [PrintLogLevelString, + PrintLogLevelString, + TestConfigPath2])), + %% Check that invalid file descriptor gives error + true = + ("magic42" =/= + RunInBash( + "erl " + "-noshell " + "-configfd invalid " + "-eval " + "'io:format(\"magic42\"),erlang:halt()' ")), + %% Check that an incorrect suffix gives error + true = + ("magic42" =/= + RunInBash( + io_lib:format( + "erl " + "-noshell " + "-configfd 3.badsuffix " + "-eval " + "'io:format(\"magic42\"),erlang:halt()' " + "3< \"~s\"", + [TestConfigPath1]))), + %% Check that an output only file descriptor gives error + true = + ("magic42" =/= + RunInBash("erl " + "-noshell " + "-configfd 3 " + "-eval " + "'io:format(\"magic42\"),erlang:halt()' " + "3> /dev/null ")), + %% Check that file descriptor with a huge amount of data fails + case application:start(os_mon) of + ok -> case proplists:get_value(system_total_memory, + memsup:get_system_memory_data()) of + Memory when is_integer(Memory), + Memory > 16*1024*1024*1024 -> + application:stop(os_mon), + true = + ("magic42" =/= + RunInBash( + "erl " + "-noshell " + "-configfd 3 " + "-eval " + "'io:format(\"magic42\"),erlang:halt()' " + "3< <(erl -noshell -eval '(fun W(D) -> io:put_chars(D), W([D,D]) end)(<<\"00000000000000000\">>)') ")); + _ -> + io:format("Skipped huge file check to avoid flaky test on machine with less than 8GB of memory") + end; + _ -> + io:format("Skipped because we could not start os_mon") + end, + ok. + +%% Test that one can get configuration from file descriptor with the +%% -configfd option +configfd_bash(Conf) when is_list(Conf) -> + case os:type() of + {unix,_} -> + case os:cmd("bash -c \"echo -n yes_bash_shell_exists\"") of + "yes_bash_shell_exists" -> + do_configfd_test_bash(); + _ -> + {skip,"Runs only when there is a bash shell"} + end; + _ -> {skip,"Runs only on UNIX systems"} + end. + +%% This test should work on all platforms +configfd_port_program(Conf) when is_list(Conf) -> + ErlProgram = + case os:find_executable("erl") of + false -> os:find_executable("erl.exe"); + Path -> Path + end, + case ErlProgram of + false -> + {skip,"Cannot find erl program"}; + ErlProgramPath -> + do_configfd_test_port_program(ErlProgramPath) + end. + %%%----------------------------------------------------------------- diff --git a/lib/kernel/test/application_SUITE_data/testconfigfd1.config b/lib/kernel/test/application_SUITE_data/testconfigfd1.config new file mode 100644 index 0000000000..0091788936 --- /dev/null +++ b/lib/kernel/test/application_SUITE_data/testconfigfd1.config @@ -0,0 +1 @@ +[{kernel, [{logger_level, warning}]}]. \ No newline at end of file diff --git a/lib/kernel/test/application_SUITE_data/testconfigfd2.config b/lib/kernel/test/application_SUITE_data/testconfigfd2.config new file mode 100644 index 0000000000..30b85b0a53 --- /dev/null +++ b/lib/kernel/test/application_SUITE_data/testconfigfd2.config @@ -0,0 +1 @@ +[{kernel, [{logger_level, error}]}]. \ No newline at end of file -- 2.31.1
Locations
Projects
Search
Status Monitor
Help
OpenBuildService.org
Documentation
API Documentation
Code of Conduct
Contact
Support
@OBShq
Terms
openSUSE Build Service is sponsored by
The Open Build Service is an
openSUSE project
.
Sign Up
Log In
Places
Places
All Projects
Status Monitor
